performance experiments - #4022

Draft

cody-littley wants to merge 74 commits into

mainfrom

cjl/snapshot-experiments

Commits on Aug 10, 2026

wire snapshots into flatKV
Cody Littley
committed
9426bb3

Commits on Aug 11, 2026

allow iterator to be held longer
Cody Littley
committed
596f3db
don't paper over an error with a log
Cody Littley
committed
e022e8e
doc cleanup, unit test fix
Cody Littley
committed
5a32ee5
document startup edge case
Cody Littley
committed
a212f4a
fix unit test dead code
Cody Littley
committed
0cc5f91
fix snapshot metrics
Cody Littley
committed
aec629c
fix docs
Cody Littley
committed
c65ee19
fix flaky test
Cody Littley
committed
ce4a3f5
small fixes
Cody Littley
committed
521a32f

Commits on Aug 12, 2026

fix bug
Cody Littley
committed
9029ea1
minor fixes
Cody Littley
committed
fae5d04
Merge branch 'main' into cjl/snapshot-integration-2
Cody Littley
committed
c44b7dd
snapshot writer on background thread
Cody Littley
committed
9b63f3b

Commits on Aug 13, 2026

background hashing, not yet had a human review pass
Cody Littley
committed
2b5e17a
test work
Cody Littley
committed
bceb98f
don't flush the WAL
Cody Littley
committed
89136b9
tune block preprocessing
Cody Littley
committed
fd5065d
new tests
Cody Littley
committed
f51d0df
more efficient serialization
Cody Littley
committed
16bf1f3
snapshot engine tuning/rewrite
Cody Littley
committed
f17ad76
more snapshot engine write optimizations
Cody Littley
committed
c856d15
minor tweaks
Cody Littley
committed
164b952

Commits on Aug 14, 2026

fix cache dashboard
Cody Littley
committed
115d6c0
2000 transactions per block
Cody Littley
committed
55b8bc4
new test scenario
Cody Littley
committed
4c3e2a5
update block size
Cody Littley
committed
ef45016
optimize batch get
Cody Littley
committed
b89bab9
more optimizations for batch get
Cody Littley
committed
79e97cb

Commits on Aug 20, 2026

remove shard locks
Cody Littley
committed
198a10d
fan out threads for commit
Cody Littley
committed
74e6778
move allocations out of shard commit lock
Cody Littley
committed
9159c7c
revert fanout
Cody Littley
committed
8ba160b
fan out to different DBs but not to shards, revert changes that didn't help
Cody Littley
committed
2c47604
5k transactions per block
Cody Littley
committed
35169e4
improve cryptosim harness perf
Cody Littley
committed
a51d0a4
Revert "improve cryptosim harness perf"

Cody Littley
and
claude
committed
c0c1ac6
tweak cryptosim harness
Cody Littley
committed
2625c9b
remove mutex
Cody Littley
committed
e241693
approximate LRU
Cody Littley
committed
17481cf

Commits on Aug 21, 2026

don't take wlock for hashing
Cody Littley
committed
5dbe66d
fewer allocations
Cody Littley
committed
9e58fb3
turn off cache metrics
Cody Littley
committed
7cdb4aa
tweaks
Cody Littley
committed
ec5da7c
parallel preprocess
Cody Littley
committed
9498e10
revert preprocess threading
Cody Littley
committed
c147d63
batch allocate keys in ApplyChangeSets()
Cody Littley
committed
b8fba81
re-enable snapshot metrics, add pprof support
Cody Littley
committed
d85427c
more shards
Cody Littley
committed
6b579bb
shard count 16
Cody Littley
committed
dc58c4c
32 shards + fanout
Cody Littley
committed
bfc0f27
experimental fanout
Cody Littley
committed
8b1a2f5

Commits on Aug 24, 2026

refactor hashing
Cody Littley
committed
504c2c2
move account merging+reads off the main thread
Cody Littley
committed
fa3a6b3
optimze merge account phase
Cody Littley
committed
26d3922
chunk up GC
Cody Littley
committed
bb5aa2d
more performant block generation
Cody Littley
committed
69bc825
more efficient work sending to executors
Cody Littley
committed
6210c23
experimental build
Cody Littley
committed
3fa62dd
remove GC chunking
Cody Littley
committed
b9554e5
fan out GC
Cody Littley
committed
db4ffcd
better back pressure, more metrics
Cody Littley
committed
eac9b2b
tune pebble, new metrics
Cody Littley
committed
8daf4fa
tweak pebble config
Cody Littley
committed
f08cae3
tune pebble
Cody Littley
committed
ec7d38c
pre-sort values, revert pebble config tweaks
Cody Littley
committed
2990212
better pre-sorting
Cody Littley
committed
4648be2
moar transactions per block
Cody Littley
committed
b394520

Commits on Aug 25, 2026

parallelize apply_change_sets_prepare
Cody Littley
committed
a01dbc5
wire in change
Cody Littley
committed
77c1cc6
benchmarks
Cody Littley
committed
79b3f04
faster block creation
Cody Littley
committed
359cd78
memiavl stuff
Cody Littley
committed
2ae80fb
cryptosim progress
Cody Littley
committed
f8c7d84