perf(simd.h): fill in missing NEON paths in simd.h - #5412
Open
lgritz wants to merge 1 commit into
Open
Conversation
Audit of simd.h for places where the SSE branch had an obvious aarch64
counterpart but NEON fell through to the scalar path.
The biggest find is exp() and log(): both were gated on OIIO_SIMD_SSE even
though their bodies are written entirely in terms of our own vector ops and
contain no raw _mm_ intrinsics at all, so NEON was evaluating them one lane
at a time. Widening the gate makes them 2.5x faster.
Also newly vectorized on NEON:
* vint4/vfloat4 loads from short, unsigned short, char, unsigned char --
these were SSE4-or-scalar, now vmovl/vcvt (1.6x-2.5x, and it carries up
to the 8- and 16-wide types, which decompose into vfloat4).
* ceil, floor and ifloor for vfloat4 and vfloat3 (vrndpq/vrndmq/vcvtmq).
round already had a NEON branch; these were simply missed.
* round for vfloat3, and vreduce_add for vfloat3.
* transpose and AxBxCxDx for vfloat4 and vint4 (vtrnq + vcombine).
* matrix44 * vfloat4, where vpaddq_f32 is the exact equivalent of the
_mm_hadd_ps the SSE3 path uses.
* msub, nmadd and nmsub. madd already had a NEON branch; the other three
did not, and nmadd was compiling to a separate fmul and fsub rather than
a single fused instruction. Switched madd from vmlaq to vfmaq while here
so all four match the fused semantics of the _mm_fmadd_ps path (clang
was already lowering vmlaq to fmla, so codegen is unchanged).
* vfloat4::load_pairs, and vfloat4::load(const half*), which was widening
to 32 bits only to immediately narrow back to 16.
fast_rint in fmath.h had the same shape of gap: it was gated on SSE4 for the
single-instruction path, but aarch64 lowers std::rint to frintx just as well.
cpu_has_neon() is new, so that the hw:simd attribute reports "neon" on ARM
rather than an empty string -- cpuid() returns all zeros off x86, so every
existing query answered false there.
Two things this deliberately does not do. A NEON path for the general
shuffle<i0,i1,i2,i3> using vqtbl1q_u8 was tried and dropped: clang already
pattern-matches the scalar form into rev64/ext/zip, sometimes more cheaply
than a table lookup, and the benchmark showed no difference. And nothing was
added for the 8- and 16-wide gather/scatter fallbacks, since aarch64 has no
gather instruction.
simd_test on an M4: 428 of 649 benchmarks improved by more than 10%, none
regressed by more than 10%.
Assisted-by: Claude Code / claude-opus-5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Audit of simd.h for places where the SSE branch had an obvious aarch64 counterpart but NEON fell through to the scalar path.
The biggest find is exp() and log(): both were gated on OIIO_SIMD_SSE even though their bodies are written entirely in terms of our own vector ops and contain no raw
_mm_intrinsics at all, so NEON was evaluating them one lane at a time. Widening the gate makes them 2.5x faster.Also newly vectorized on NEON:
fast_rint in fmath.h had the same shape of gap: it was gated on SSE4 for the single-instruction path, but aarch64 lowers std::rint to frintx just as well.
cpu_has_neon() is new, so that the hw:simd attribute reports "neon" on ARM rather than an empty string -- cpuid() returns all zeros off x86, so every existing query answered false there.
Two things this deliberately does not do. A NEON path for the general
shuffle<i0,i1,i2,i3>using vqtbl1q_u8 was tried and dropped: clang already pattern-matches the scalar form into rev64/ext/zip, sometimes more cheaply than a table lookup, and the benchmark showed no difference. And nothing was added for the 8- and 16-wide gather/scatter fallbacks, since aarch64 has no gather instruction.simd_test on an M4: 428 of 649 benchmarks improved by more than 10%, none regressed by more than 10%.
Assisted-by: Claude Code / claude-opus-5