Add weight-based eviction to ConcurrentLfu

[bitfaster]

Summary

Adds opt-in weight-based eviction to ConcurrentLfu, faithfully matching Caffeine's weighted W-TinyLFU. A user supplies an IWeigher<K, V>; the cache is then bounded by total weight instead of item count.

var lfu = new ConcurrentLfuBuilder<string, byte[]>()
    .WithCapacity(1_000_000)              // maximum total weight
    .WithWeigher(new ByteArrayWeigher())  // IWeigher<K,V>.Weigh => value.Length
    .Build();

WithWeigher composes with WithEvents and WithExpireAfterWrite/WithExpireAfterAccess/WithExpireAfter.

Design

• No new generic. Weighting is carried by the existing INodePolicy slot via two new policy structs (WeightedAccessOrderPolicy, WeightedExpireAfterPolicy). ConcurrentLfuCore/TimerWheel arities are unchanged.
• Zero cost to the default (unweighted) path. Weighted branches are selected by static readonly bool IsWeighted = default(P).IsWeighted and JIT-elided in the count case. Weight is stored on weighted node subclasses, so unweighted node layout is untouched.
• policyWeight-authoritative accounting maintains the invariant weightedSize == Σ policyWeight, which is robust to BitFaster's buffered/duplicated writes.
• Ports Caffeine's weighted evictFromWindow/evictFromMain (zero-weight skips, oversize-candidate eviction, frequency admission with the >=6/1-in-128 anti-hashflood jitter), oversize-on-add, conditional weight-change handling, zero-weight survival, and the absolute-weight hill-climb (increase/decreaseWindow moving whole nodes within quota).
• IBoundedPolicy.Capacity stays int (the weight budget); internal accounting is long. With a weigher, zero/low-weight entries mean Count can exceed Capacity (by design).

Scope

In: ConcurrentLfu (with/without events) and ConcurrentTLfu (weight composed with time expiry). Deferred: FastConcurrentLfu standalone exposure and the scoped/atomic/async builder wrappers.

Testing

• Full suite: 1563 pass, no regressions; builds on all TFMs (netstandard2.0, netcoreapp3.1, net6.0, net10.0, net48).
• New coverage: weighted accounting/eviction/oversize/zero-weight/weight-change/promotion invariants, the hill-climb adaptation, all builder combinations, the shared cache suite run against the weighted+events wrapper, and a 4-thread concurrency soak that reuses the integrity checker and asserts the weighted-size invariant under load.
• A rubber-duck review caught a critical bug (probation→protected read promotion was using the unweighted path, corrupting mainProtectedWeightedSize) — fixed and regression-tested — plus three weighted+expiry hazards (deschedule-on-remove, skip AfterWrite on evict, idempotent discount).

Note

TimerWheelTests.WhenRescheduledLaterNodeIsMoved is a pre-existing, date/wall-clock-dependent test (uses real Duration.SinceEpoch()); it fails on a clean main independent of this change.

Draft

Opened as a draft for review of the approach and API shape before finalizing.

[coveralls]

coverage: 97.363% (-1.9%) from 99.226% — users/alexpeck/weightedlfu into main

[ben-manes]

fwiw, this paper argued for a different weighted admission scheme than Caffeine's (compares an accumulation).

My concerns with the approach was that under concurrency this becomes harder to reason about because the cache's state is constantly changing. There is still a need for an evaluation loop to handle races, e.g. the entry's weight might change mid evaluation or it could be explicitly removed. Since weight-based caches are less common for on-heap application caches (hard to size objects, smaller, use less memory) it seemed like an optimization that could be deferred. The approach would make more sense for persisted or dedicated remote cache servers which might hold hundreds of millions of entries, but also have vastly different needs and design goals. You might find something interesting to borrow as I liked the idea conceptually but there was a marginal hit rate increase and no real world advocate.