Add native SIMD testing and benchmark module - #708
Conversation
|
Before you submit for review:
If you did not complete any of these, then please explain below. |
e70b8ea to
d24a6f2
Compare
Covers cosine_f32, dot_product_f32, euclidean_f32, elementwise ops, and CPU-feature dispatch across SSE42/AVX2/AVX512 lane widths and tail paths.
- New benchmarks/bench_similarity_f32.cpp: parameterised benchmarks for cosine_f32, dot_product_f32, and euclidean_f32 over array sizes 128, 256, 512, 1024, 1536, and 3072. - meson.build: add optional google-benchmark dependency (required: false); builds bench_simd_kernels executable only when the library is present.
- Install libgtest-dev alongside meson/ninja so GTest is found by meson.
- Build test_simd_kernels with 'ninja -C build test_simd_kernels' after
checkout (before the JDK setup, since it is a pure C++ step).
- Run the binary once per matrix.max_isa leg:
avx512f → no JVECTOR_MAX_ISA cap (auto-detects best ISA on runner)
avx2 → JVECTOR_MAX_ISA=avx2
sse42 → JVECTOR_MAX_ISA=sse42
…to the jar replace all hardcoded relative ../../../../ paths with variables rooted at $(git rev-parse --show-toplevel), so the script works correctly regardless of the working directory it is invoked from.
7164b35 to
4578555
Compare
MarkWolters
left a comment
There was a problem hiding this comment.
code looks good, very thorough. I think this will make a good addition to our test suite. Please just update the comments in bench_similarity_f32.cpp and/or add a README and I think it's good to go.
There was a problem hiding this comment.
I think this looks like it is probably fine, but for future reference it would be easier to verify if this was a branch and not coming from a fork. Since it's a fork I can't just run the gha and verify the results, which would make it easier to be comfortable with approval. Not a showstopper but something to keep in mind for future PRs.
| // 128, 256, 512, 1024, 1536, 3072 | ||
| // | ||
| // Build (requires google-benchmark installed or available via pkg-config): | ||
| // meson setup build && ninja -C build bench_simd_kernels |
There was a problem hiding this comment.
This doesn't work for me. I cloned the fork but this command fails with the following
ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ meson setup build && ninja -C build bench_simd_kernels
Directory already configured.
Just run your build command (e.g. ninja) and Meson will regenerate as necessary.
Run "meson setup --reconfigure to force Meson to regenerate.
If build failures persist, run "meson setup --wipe" to rebuild from scratch
using the same options as passed when configuring the build.
ninja: Entering directory `build'
ninja: error: unknown target 'bench_simd_kernels'
ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ git status
On branch simd-testing
Your branch is up to date with 'origin/simd-testing'.
ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ meson setup build --wipe
The Meson build system
Version: 1.3.2
Source dir: /home/ubuntu/jvector/jvector-native/src/main/native
Build dir: /home/ubuntu/jvector/jvector-native/src/main/native/build
Build type: native build
Project name: jvector_simd_kernels
Project version: 0.1.0
C++ compiler for the host machine: c++ (gcc 13.3.0 "c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0")
C++ linker for the host machine: c++ ld.bfd 2.42
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: YES (/usr/bin/pkg-config) 1.8.1
Run-time dependency gtest_main found: YES 1.14.0
Found CMake: /usr/bin/cmake (3.28.3)
Run-time dependency benchmark found: NO (tried pkgconfig and cmake)
Build targets in project: 7
Found ninja-1.11.1 at /usr/bin/ninja
ubuntu@ip-172-31-6-106:~/jvector/jvector-native/src/main/native$ ninja -C build bench_simd_kernels
ninja: Entering directory `build'
ninja: error: unknown target 'bench_simd_kernels'
There was a problem hiding this comment.
Oh it's test_simd_kernels, not bench_simd_kernels. Just needs an updated comment(s). Might be worth including a README that lists the steps, I had to walk through the github workflow to understand what to do.
| # fail on error | ||
| set -e | ||
| # print commands as they are executed | ||
| set +x |
There was a problem hiding this comment.
I think you want set -x here, set +x turns off debugging
This pull request adds comprehensive native C++ unit tests and benchmarks for the SIMD kernel dispatcher and vector similarity functions, and integrates them into the build and CI pipeline. The most important changes are:
Testing and Benchmarking Enhancements:
test_simd_kernelsexecutable using Google Test, with tests for ISA detection and SIMD kernel correctness, and integrates it into the Meson build and GitHub Actions CI workflow.bench_simd_kernelsexecutable using Google Benchmark to micro-benchmarkcosine_f32,dot_product_f32, andeuclidean_f32functions at realistic embedding dimensions.Build System and Code Organization:
CI Improvements:
libgtest-dev, build, and run the new native C++ tests for all relevant ISA tiers, ensuring dispatcher correctness across hardware.