Skip to content

perf(sqlx-cli): avoid rewriting the entire .sqlx cache on prepare#4237

Open
fhsgoncalves wants to merge 1 commit intolaunchbadge:mainfrom
fhsgoncalves:perf/prepare-prune
Open

perf(sqlx-cli): avoid rewriting the entire .sqlx cache on prepare#4237
fhsgoncalves wants to merge 1 commit intolaunchbadge:mainfrom
fhsgoncalves:perf/prepare-prune

Conversation

@fhsgoncalves
Copy link
Copy Markdown
Contributor

Does your PR solve an issue?

N/A (no issue filed).

Is this a breaking change?

No.

Summary

  • stop deleting all query-*.json files in .sqlx on every cargo sqlx prepare
  • generate query data into a temporary cache directory and then sync only changed files into .sqlx
  • prune stale query cache files while preserving unchanged entries to reduce filesystem churn

Motivation: I'm working on a large monorepo, with ~1500 sqlx cached query files, and every time I run cargo sqlx prepare --workspace, it deletes all the cached files, the IDE/lsp shows lots of compilation errors, and I need to wait several seconds (sometimes minutes) to have all the files back. This PR fixes that

Copy link
Copy Markdown
Contributor

@jplatte jplatte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the commit message say perf? This doesn't actually help performance in any way, right?

Comment thread sqlx-cli/src/prepare.rs

let prepare_dir = ctx.prepare_dir()?;
run_prepare_step(ctx, &prepare_dir)?;
let cache_dir = ctx.metadata.target_directory().join("sqlx-prepare");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should probably be a subdirectory of .sqlx instead, to avoid any cross-filesystem copying in case target is a symlink to a different fs. Maybe .sqlx/staging? You could put a .gitignore in .sqlx automatically to not have it tracked by git (though maybe this would be annoying for people on other version control systems?).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, do we already have this kind of temporary folder somewhere here in sqlx?
Using .sqlx looks weird for me, since it is a folder to be versioned, and may impact people not using git indeed, or already having an own custom .gitignore inside .sqlx/ folder.

About using the target directory: why that would be an issue with cross-filesystem copying? It should work, right?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-fs copies are slower and more error-prone because they're not atomic. I could see things breaking if in parallel with sqlx prepare, rust-analyzer executes one of the macros or something.

@fhsgoncalves
Copy link
Copy Markdown
Contributor Author

Why does the commit message say perf? This doesn't actually help performance in any way, right?

It is because the dev-loop when using cargo sqlx prepare will have better performance -> the IDE / rust analyzer will not try to recompile right away because of the missing query caches while sqlx prepare runs.
But I can change the commit message or PR title, no problem! Just tell me what works best 😄

@jplatte
Copy link
Copy Markdown
Contributor

jplatte commented Apr 17, 2026

I'm not actually a maintainer, so I can't make any definitive statements about commit message style. Was just asking out of curiosity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants