fix: capture only real timeline entries in export scripts#85
Open
JinZhongcun wants to merge 1 commit into
Open
fix: capture only real timeline entries in export scripts#85JinZhongcun wants to merge 1 commit into
JinZhongcun wants to merge 1 commit into
Conversation
The export scripts hunt for any object that looks like a tweet (rest_id + legacy/core). User objects nested under core.user_results.result also match that shape, so they get captured as bookmarks with an empty author, text and timestamp, and are then imported into the database as empty entries. Quoted/embedded tweets from other timeline modules can leak in the same way. Capture tweets only from real timeline entries (entryId "tweet-<id>", used by both the bookmarks and likes timelines) instead of pattern-matching arbitrary objects. In a live export of ~2,000 bookmark items, the old matcher produced 241 junk entries (~12%); with entry-based capture, zero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The bookmarklet and console export scripts hunt for any object shaped like a tweet (
rest_id+legacy/core). User objects nested undercore.user_results.resultmatch that shape too, so they get captured as bookmarks with empty author/text/timestamp and are then imported into the database as empty entries. Quoted/embedded tweets can leak in the same way.Changes
entryId: "tweet-<id>", the format used by both the bookmarks and likes timelines) instead of pattern-matching arbitrary objects (BOOKMARKLET_SCRIPTandCONSOLE_SCRIPTinapp/import/page.tsx)isTweetObj→isTweetEntry; thequoted_status_resultskip is no longer needed since only entry containers are matchedRelated Issues
None
Checklist
npx tsc --noEmit— no new errors introduced (the 2 pre-existing errors inapp/api/import/x-oauth/fetch/route.tsare present onmainas well)