Skip to content

Add native SIMD testing and benchmark module - #708

Open
r-devulap wants to merge 6 commits into
datastax:mainfrom
r-devulap:simd-testing
Open

Add native SIMD testing and benchmark module#708
r-devulap wants to merge 6 commits into
datastax:mainfrom
r-devulap:simd-testing

Conversation

@r-devulap

Copy link
Copy Markdown
Contributor

This pull request adds comprehensive native C++ unit tests and benchmarks for the SIMD kernel dispatcher and vector similarity functions, and integrates them into the build and CI pipeline. The most important changes are:

Testing and Benchmarking Enhancements:

  • Adds a new test_simd_kernels executable using Google Test, with tests for ISA detection and SIMD kernel correctness, and integrates it into the Meson build and GitHub Actions CI workflow.
  • Adds a new bench_simd_kernels executable using Google Benchmark to micro-benchmark cosine_f32, dot_product_f32, and euclidean_f32 functions at realistic embedding dimensions.

Build System and Code Organization:

  • Moves the core SIMD kernel and dispatcher source files into a src/ subdirectory and updates the Meson build rules and include paths accordingly. The native code layout has been reorganized in preparation for modularizing it into a standalone library, improving structure and maintainability.

CI Improvements:

  • Updates the GitHub Actions workflow to install libgtest-dev, build, and run the new native C++ tests for all relevant ISA tiers, ensuring dispatcher correctness across hardware.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Before you submit for review:

  • Does your PR follow guidelines from CONTRIBUTIONS.md?
  • Did you summarize what this PR does clearly and concisely?
  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you rebase your branch onto the latest main for regression testing and PR submission?
  • Did you trigger regression testing via Run Bench Main and review results?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?
  • Did you add documentation for this feature to the release notes directory?

If you did not complete any of these, then please explain below.

@r-devulap r-devulap changed the title Add SIMD native testing and benchmark module Add native SIMD testing and benchmark module Aug 3, 2026
@r-devulap
r-devulap force-pushed the simd-testing branch 2 times, most recently from e70b8ea to d24a6f2 Compare August 3, 2026 07:14
Covers cosine_f32, dot_product_f32, euclidean_f32, elementwise ops, and
CPU-feature dispatch across SSE42/AVX2/AVX512 lane widths and tail paths.
- New benchmarks/bench_similarity_f32.cpp: parameterised benchmarks for
  cosine_f32, dot_product_f32, and euclidean_f32 over array sizes
  128, 256, 512, 1024, 1536, and 3072.
- meson.build: add optional google-benchmark dependency (required: false);
  builds bench_simd_kernels executable only when the library is present.
- Install libgtest-dev alongside meson/ninja so GTest is found by meson.
- Build test_simd_kernels with 'ninja -C build test_simd_kernels' after
  checkout (before the JDK setup, since it is a pure C++ step).
- Run the binary once per matrix.max_isa leg:
    avx512f  → no JVECTOR_MAX_ISA cap (auto-detects best ISA on runner)
    avx2     → JVECTOR_MAX_ISA=avx2
    sse42    → JVECTOR_MAX_ISA=sse42
…to the jar

replace all hardcoded relative ../../../../ paths with variables rooted
at $(git rev-parse --show-toplevel), so the script works correctly regardless
of the working directory it is invoked from.

@MarkWolters MarkWolters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks good, very thorough. I think this will make a good addition to our test suite. Please just update the comments in bench_similarity_f32.cpp and/or add a README and I think it's good to go.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks like it is probably fine, but for future reference it would be easier to verify if this was a branch and not coming from a fork. Since it's a fork I can't just run the gha and verify the results, which would make it easier to be comfortable with approval. Not a showstopper but something to keep in mind for future PRs.

// 128, 256, 512, 1024, 1536, 3072
//
// Build (requires google-benchmark installed or available via pkg-config):
// meson setup build && ninja -C build bench_simd_kernels

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work for me. I cloned the fork but this command fails with the following

ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ meson setup build && ninja -C build bench_simd_kernels
Directory already configured.

Just run your build command (e.g. ninja) and Meson will regenerate as necessary.
Run "meson setup --reconfigure to force Meson to regenerate.

If build failures persist, run "meson setup --wipe" to rebuild from scratch
using the same options as passed when configuring the build.
ninja: Entering directory `build'
ninja: error: unknown target 'bench_simd_kernels'
ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ git status
On branch simd-testing
Your branch is up to date with 'origin/simd-testing'.

ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ meson setup build --wipe
The Meson build system
Version: 1.3.2
Source dir: /home/ubuntu/jvector/jvector-native/src/main/native
Build dir: /home/ubuntu/jvector/jvector-native/src/main/native/build
Build type: native build
Project name: jvector_simd_kernels
Project version: 0.1.0
C++ compiler for the host machine: c++ (gcc 13.3.0 "c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0")
C++ linker for the host machine: c++ ld.bfd 2.42
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: YES (/usr/bin/pkg-config) 1.8.1
Run-time dependency gtest_main found: YES 1.14.0
Found CMake: /usr/bin/cmake (3.28.3)
Run-time dependency benchmark found: NO (tried pkgconfig and cmake)
Build targets in project: 7

Found ninja-1.11.1 at /usr/bin/ninja
ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ ninja -C build bench_simd_kernels
ninja: Entering directory `build'
ninja: error: unknown target 'bench_simd_kernels'


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh it's test_simd_kernels, not bench_simd_kernels. Just needs an updated comment(s). Might be worth including a README that lists the steps, I had to walk through the github workflow to understand what to do.

# fail on error
set -e
# print commands as they are executed
set +x

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want set -x here, set +x turns off debugging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants