Skip to content

[Fix] ConcurrentLongHashMap throw ArrayIndexOutOfBoundsException#4771

Open
void-ptr974 wants to merge 7 commits intoapache:masterfrom
void-ptr974:fix_clhm_aoob
Open

[Fix] ConcurrentLongHashMap throw ArrayIndexOutOfBoundsException#4771
void-ptr974 wants to merge 7 commits intoapache:masterfrom
void-ptr974:fix_clhm_aoob

Conversation

@void-ptr974
Copy link
Copy Markdown
Contributor

@void-ptr974 void-ptr974 commented May 1, 2026

Fix #4684

Bug

ConcurrentLongHashMap.Section publishes keys, values, capacity as three separate volatile fields. During rehash a reader can observe a torn snapshot — e.g. new keys (length 8)
with old values (length 4) — then compute bucket = signSafeMod(hash, values.length) ∈ [0,8) and index keys[bucket] against length 4, throwing ArrayIndexOutOfBoundsException at
Section.get. The exception fires before the optimistic-read validate(stamp) check, so the existing fallback can't recover.

Reproduce in ≈ 2 minutes

# Revert just the production file to upstream master, keeping this PR's tests + benchmark:
git checkout origin/master -- bookkeeper-server/src/main/java/org/apache/bookkeeper/util/collections/ConcurrentLongHashMap.java

mvn -pl bookkeeper-server -DskipTests install
mvn -pl microbenchmarks -DskipTests package

java -jar microbenchmarks/target/benchmarks.jar \
    "ConcurrentLongHashMapBenchmark.concurrentExpandShrink" \
    -wi 1 -i 1 -w 3s -r 3s -f 1 -p implementation=clhm

ArrayIndexOutOfBoundsException fires within the first warmup iteration. 10/10 reproductions across 10 fresh JVMs on the unpatched code, 0/10 after this PR.

Fix

Bundle keys, values, capacity into one immutable inner class published via a single volatile reference:

private static final class Table<V> {
    final long[] keys;
    final V[] values;
    final int capacity;
}
private volatile Table<V> table;

Final-field semantics plus one volatile-acquire-release pair give the reader a consistent triple from one read. The partial-publish window is impossible by construction.

Tests added

bookkeeper-server/src/test/.../ConcurrentLongHashMapTest:

  • testNoLostGetAfterPublish — writer puts k then publishes k via a volatile counter; every reader observing the counter must see a non-null value for k.
  • testCorrectnessAgainstConcurrentHashMap — 8 threads × 50K random ops mirrored to ConcurrentHashMap; per-call return values and final state must match.
  • testConcurrentMultiWriterExpandShrink — 8 writers × 8 readers on a single section with autoShrink; no torn (k,v) pair is allowed to surface.
  • testForEachDuringWritesforEach during concurrent puts/removes must not throw or expose internal sentinels.
  • testConcurrentExpandAndShrinkAndGet — strengthened existing test.
mvn -pl bookkeeper-server -Dtest=ConcurrentLongHashMapTest test

23/23 pass in 4s.

A JMH benchmark (ConcurrentLongHashMapBenchmark in microbenchmarks) is included so this race can be regression-tested on every JDK / hardware change.

Master Issue: #4684

When concurrent read write access the map, The key array and value array are not publish at the same time when shrink or expand.

This fix encapsulate the key and value in the same field to avoid this happen
@void-ptr974 void-ptr974 changed the title fix ConcurrentLongHashMap throw ArrayIndexOutOfBoundsException [Fix] ConcurrentLongHashMap throw ArrayIndexOutOfBoundsException May 1, 2026
When concurrent read write access the map, The key array and value array are not publish at the same time when shrink or expand.

This fix encapsulate the key and value in the same field to avoid this happen
When concurrent read write access the map, The key array and value array are not publish at the same time when shrink or expand.

This fix encapsulate the key and value in the same field to avoid this happen
When concurrent read write access the map, The key array and value array are not publish at the same time when shrink or expand.

This fix encapsulate the key and value in the same field to avoid this happen
When concurrent read write access the map, The key array and value array are not publish at the same time when shrink or expand.

This fix encapsulate the key and value in the same field to avoid this happen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ConcurrentLongHashMap may throw an ArrayIndexOutOfBoundsException

2 participants