Skip to content

perf(simd.h): fill in missing NEON paths in simd.h - #5412

Open
lgritz wants to merge 1 commit into
AcademySoftwareFoundation:mainfrom
lgritz:lg-neon
Open

perf(simd.h): fill in missing NEON paths in simd.h#5412
lgritz wants to merge 1 commit into
AcademySoftwareFoundation:mainfrom
lgritz:lg-neon

Conversation

@lgritz

@lgritz lgritz commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Audit of simd.h for places where the SSE branch had an obvious aarch64 counterpart but NEON fell through to the scalar path.

The biggest find is exp() and log(): both were gated on OIIO_SIMD_SSE even though their bodies are written entirely in terms of our own vector ops and contain no raw _mm_ intrinsics at all, so NEON was evaluating them one lane at a time. Widening the gate makes them 2.5x faster.

Also newly vectorized on NEON:

  • vint4/vfloat4 loads from short, unsigned short, char, unsigned char -- these were SSE4-or-scalar, now vmovl/vcvt (1.6x-2.5x, and it carries up to the 8- and 16-wide types, which decompose into vfloat4).
  • ceil, floor and ifloor for vfloat4 and vfloat3 (vrndpq/vrndmq/vcvtmq). round already had a NEON branch; these were simply missed.
  • round for vfloat3, and vreduce_add for vfloat3.
  • transpose and AxBxCxDx for vfloat4 and vint4 (vtrnq + vcombine).
  • matrix44 * vfloat4, where vpaddq_f32 is the exact equivalent of the _mm_hadd_ps the SSE3 path uses.
  • msub, nmadd and nmsub. madd already had a NEON branch; the other three did not, and nmadd was compiling to a separate fmul and fsub rather than a single fused instruction. Switched madd from vmlaq to vfmaq while here so all four match the fused semantics of the _mm_fmadd_ps path (clang was already lowering vmlaq to fmla, so codegen is unchanged).
  • vfloat4::load_pairs, and vfloat4::load(const half*), which was widening to 32 bits only to immediately narrow back to 16.

fast_rint in fmath.h had the same shape of gap: it was gated on SSE4 for the single-instruction path, but aarch64 lowers std::rint to frintx just as well.

cpu_has_neon() is new, so that the hw:simd attribute reports "neon" on ARM rather than an empty string -- cpuid() returns all zeros off x86, so every existing query answered false there.

Two things this deliberately does not do. A NEON path for the general shuffle<i0,i1,i2,i3> using vqtbl1q_u8 was tried and dropped: clang already pattern-matches the scalar form into rev64/ext/zip, sometimes more cheaply than a table lookup, and the benchmark showed no difference. And nothing was added for the 8- and 16-wide gather/scatter fallbacks, since aarch64 has no gather instruction.

simd_test on an M4: 428 of 649 benchmarks improved by more than 10%, none regressed by more than 10%.

Assisted-by: Claude Code / claude-opus-5

Audit of simd.h for places where the SSE branch had an obvious aarch64
counterpart but NEON fell through to the scalar path.

The biggest find is exp() and log(): both were gated on OIIO_SIMD_SSE even
though their bodies are written entirely in terms of our own vector ops and
contain no raw _mm_ intrinsics at all, so NEON was evaluating them one lane
at a time. Widening the gate makes them 2.5x faster.

Also newly vectorized on NEON:

  * vint4/vfloat4 loads from short, unsigned short, char, unsigned char --
    these were SSE4-or-scalar, now vmovl/vcvt (1.6x-2.5x, and it carries up
    to the 8- and 16-wide types, which decompose into vfloat4).
  * ceil, floor and ifloor for vfloat4 and vfloat3 (vrndpq/vrndmq/vcvtmq).
    round already had a NEON branch; these were simply missed.
  * round for vfloat3, and vreduce_add for vfloat3.
  * transpose and AxBxCxDx for vfloat4 and vint4 (vtrnq + vcombine).
  * matrix44 * vfloat4, where vpaddq_f32 is the exact equivalent of the
    _mm_hadd_ps the SSE3 path uses.
  * msub, nmadd and nmsub. madd already had a NEON branch; the other three
    did not, and nmadd was compiling to a separate fmul and fsub rather than
    a single fused instruction. Switched madd from vmlaq to vfmaq while here
    so all four match the fused semantics of the _mm_fmadd_ps path (clang
    was already lowering vmlaq to fmla, so codegen is unchanged).
  * vfloat4::load_pairs, and vfloat4::load(const half*), which was widening
    to 32 bits only to immediately narrow back to 16.

fast_rint in fmath.h had the same shape of gap: it was gated on SSE4 for the
single-instruction path, but aarch64 lowers std::rint to frintx just as well.

cpu_has_neon() is new, so that the hw:simd attribute reports "neon" on ARM
rather than an empty string -- cpuid() returns all zeros off x86, so every
existing query answered false there.

Two things this deliberately does not do. A NEON path for the general
shuffle<i0,i1,i2,i3> using vqtbl1q_u8 was tried and dropped: clang already
pattern-matches the scalar form into rev64/ext/zip, sometimes more cheaply
than a table lookup, and the benchmark showed no difference. And nothing was
added for the 8- and 16-wide gather/scatter fallbacks, since aarch64 has no
gather instruction.

simd_test on an M4: 428 of 649 benchmarks improved by more than 10%, none
regressed by more than 10%.

Assisted-by: Claude Code / claude-opus-5

Signed-off-by: Larry Gritz <lg@larrygritz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant