spsc-ring-buffer

A lock-free, wait-free Single-Producer Single-Consumer (SPSC) ring buffer written in C++20, optimised for low-latency High Frequency Trading (HFT) workloads on x86-64.

Key design: cached index optimisation

Most SPSC implementations perform an atomic load of the opposite index on every push and pop. Because the producer and consumer run on separate cores, each such load triggers a cross-core cache-line coherence request — even when the queue is neither full nor empty.

This implementation eliminates those coherence requests in the common case by giving each side a private, non-atomic cached copy of the other side's index:

Producer cache line (64 B): [ head_ (atomic) | tail_cached_ (non-atomic) | padding ]
Consumer cache line (64 B): [ tail_ (atomic) | head_cached_ (non-atomic) | padding ]
Data array               :  aligned to 64 B separately

• The producer reads tail_cached_ first. It only falls back to tail_.load(acquire) when the queue appears full.
• The consumer reads head_cached_ first. It only falls back to head_.load(acquire) when the queue appears empty.
• In the common case (queue neither full nor empty): zero cross-core atomic reads.

Features

Feature	Detail
Language	C++20, header-only (include/spsc_queue.hpp)
Platform	x86-64 (_mm_pause in spin loops)
Capacity	Compile-time template parameter, must be a power of two
Type constraint	std::is_trivially_copyable<T> — direct memory copy, no ctor/dtor overhead
API	try_push / try_pop (wait-free) and push / pop (blocking spin)
Cache alignment	alignas(64) on producer line, consumer line, and data array
Prefetching	__builtin_prefetch on the next slot in both push and pop
Branch hints	[[likely]] / [[unlikely]] on all hot branches
Compiler hints	__attribute__((always_inline)) on all hot-path methods

Memory ordering

Operation	Ordering	Reason
Own index load	relaxed	Only this thread writes it; no coherence needed
Opposite index load (refresh)	acquire	Synchronises with the other thread's release store
Own index store	release	Makes the data write visible before the index update

This is the minimal correct ordering for SPSC — no fences, no seq_cst.

API

// Non-blocking — returns false immediately if full / empty.
[[nodiscard]] bool try_push(const T& item) noexcept;
[[nodiscard]] bool try_pop(T& item)        noexcept;

// Blocking spin — primary HFT pattern (dedicated pinned thread).
void push(const T& item) noexcept;
T    pop()               noexcept;

// Approximate size queries (not synchronised between threads).
[[nodiscard]] std::size_t size()     const noexcept;
[[nodiscard]] bool        empty()    const noexcept;
[[nodiscard]] bool        full()     const noexcept;
[[nodiscard]] static constexpr std::size_t capacity() noexcept;

Usage

#include "spsc_queue.hpp"

// Capacity must be a power of two.
SpscQueue<uint64_t, 1 << 14> queue;

// Producer thread
queue.push(42);

// Consumer thread
uint64_t value = queue.pop();

Build

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build
./build/bench

ThreadSanitizer (correctness check)

cmake -B build_tsan -DCMAKE_BUILD_TYPE=Debug -DTSAN=ON
cmake --build build_tsan
./build_tsan/bench

Benchmarks

Two benchmarks are included in bench/bench.cpp:

1. Throughput — producer pushes 50 million uint64_t values; consumer pops them all. Reports ops/sec.
2. Round-trip latency — ping-pong between two threads via two queues. Reports min / mean / median / p99 / p99.9 / max latency in nanoseconds.

Both benchmarks pin threads using pthread_setaffinity_np. Two logical CPUs on separate physical cores are selected automatically at startup by parsing /proc/cpuinfo — no manual configuration needed. Latency is measured with rdtsc (the TSC frequency is auto-calibrated at startup).

Requirements

• GCC ≥ 12 or Clang ≥ 15 (C++20 support, std::has_single_bit)
• Linux (uses pthread_setaffinity_np in the benchmark)
• x86-64 CPU

References

• Charles Frasch — Single Producer Single Consumer Lock-free FIFO From the Ground Up (CppCon 2023). The cached index optimisation and memory ordering strategy are rooted in the ideas presented in this talk.