Skip to content

x86 SIMD Backends (AVX2 / AVX-512)

The x86 SIMD paths vectorise the CPU implementations of the core features. At runtime the dispatcher picks the widest instruction set the host CPU supports: AVX-512, then AVX2, then scalar. The kernels live under core/src/feature/x86/.

What each level requires

The level is decided once at start-up by vmaf_get_cpu_flags_x86() (core/src/x86/cpu.c) from CPUID and XGETBV. A processor that lacks one requirement falls to the next lower level, never to a kernel it cannot run.

Level The CPU must report
SSE2 / SSSE3 / SSE4.1 the matching CPUID leaf 1 bits
AVX2 AVX and OSXSAVE with the OS saving XMM and YMM state, FMA (leaf 1 ECX bit 12), BMI1, BMI2 and AVX2 (leaf 7 EBX)
AVX-512 everything AVX2 needs, plus the OS saving ZMM and opmask state and AVX512F, DQ, CD, BW and VL
AVX-512 ICL AVX-512 plus the Ice Lake extensions

FMA is part of the AVX2 level because every AVX2 library is built with -mfma and two kernels contain FMA instructions (ms_ssim_decimate_avx2, the SSIMULACRA 2 linear-RGB conversion). Every Intel and AMD core since Haswell and Zen has FMA; a virtual machine or emulator that masks it now runs the SSE paths, with the same scores (ADR-2055).

Which features have which kernels

AVX2 kernels exist for every feature below. AVX-512 kernels exist for all of them except the rows marked "AVX2 only".

Feature AVX2 file AVX-512 file Notes
adm adm_avx2.c adm_avx512.c integer ADM
float_adm float_adm_avx2.c float_adm_avx512.c
vif vif_avx2.c, vif_statistic_avx2.c vif_avx512.c the statistic helper is AVX2 only
motion motion_avx2.c motion_avx512.c
float_motion float_motion_avx2.c float_motion_avx512.c
psnr psnr_avx2.c psnr_avx512.c
float_psnr float_psnr_avx2.c float_psnr_avx512.c
float_ssim ssim_avx2.c ssim_avx512.c
integer_ssim integer_ssim_avx2.c none AVX2 only
float_ms_ssim ms_ssim_decimate_avx2.c ms_ssim_decimate_avx512.c decimation step
float_moment moment_avx2.c moment_avx512.c
cambi cambi_avx2.c cambi_avx512.c
ciede ciede_avx2.c ciede_avx512.c
speed_* speed_avx2.c, speed_matmul_avx2.c speed_avx512.c, speed_matmul_avx512.c
ssimulacra2 ssimulacra2_avx2.c, ssimulacra2_host_avx2.c ssimulacra2_avx512.c the host helper is AVX2 only
convolution helpers convolve_avx2.c convolve_avx512.c, ../common/convolution_avx512.c the second file is shared (see below)
psnr_hvs psnr_hvs_avx2.c none AVX2 only

Build

AVX2 kernels are compiled whenever assembly support is on. The AVX-512 option builds the AVX-512 kernels; it needs nasm 2.14 or newer and is ignored (with a warning) when enable_asm=false. It is on by default. Run from the repository root:

meson setup build core -Denable_avx512=true   # default is true on x86_64
ninja -C build

To disable the AVX-512 paths (useful when profiling scalar or AVX2 baselines or debugging a codegen regression):

meson setup build core -Denable_avx512=false

Runtime selection

CPU dispatch happens in the feature extractor once per execution; there is no per-frame overhead. The choice can be inspected through the resolved VmafFeatureExtractor::init entry for each feature.

To force a lower ISA for A/B testing, pass the --cpumask flag, which disables instruction sets bit-by-bit (see libvmaf.h):

./build/tools/vmaf --cpumask 16 ...     # disable AVX-512 (bit 4)
./build/tools/vmaf --cpumask 24 ...     # disable both AVX2 (8) and AVX-512 (16)
./build/tools/vmaf --cpumask 31 ...     # disable everything down to scalar

The full bitmask layout: SSE2 (1), SSE3/SSSE3 (2), SSE4.1 (4), AVX2 (8), AVX-512 (16), AVX-512-ICL (32).

Design notes

  • 64-byte alignment. AVX-512 loads/stores prefer 64-byte-aligned input buffers. Picture allocations go through aligned_alloc(64, …); scratch buffers allocated inside feature extractors do the same.
  • Mask registers for loop tails. The AVX-512 kernels use k1-mask stores for width tails that aren't a multiple of 16/32 elements rather than falling back to scalar cleanup loops. Lower loop overhead and no branch predictor pressure at the tail.
  • Explicit FMA only. Where a kernel wants a fused multiply-add it calls the intrinsic (_mm512_fmadd_ps, vfmadd*), one µop with better throughput and precision than mul plus add. The compiler never contracts a * b + c on its own in these files (see FP contraction).
  • Frequency downclocking is obsolete on Zen 4/5 and recent Xeons. The AVX-512 power-license throttle that affected Skylake-X is not a concern on AMD Zen 4/5 or Sapphire Rapids+. The kernels insert no "warm-up" instructions; the first-frame penalty is negligible on target hardware.
  • One kernel per (feature, ISA). Naming is <feature>_<isa>.{c,h} (e.g. adm_avx512.c, vif_avx2.c); runtime dispatch is by CPU feature flags.

Integer VIF stage checks

The integer VIF AVX-512 path retains the existing integer arithmetic, log-table normalisation, enhancement-gain limit and scale outputs. Its private vertical, horizontal and subsample helpers preserve each accumulator's tap and lane order. The two 8-bit subsample block boundaries remain noinline per ADR-0503.

Run the test

On a CPU with AVX-512, configure the core Meson project with assembly and AVX-512 enabled, then run the dedicated integer test:

python3 "$(git rev-parse --show-toplevel)/scripts/ci/run_meson_test.py" -- \
  -C build --print-errorlogs test_integer_vif_avx512_stages

Unsupported CPUs print an explicit skip; builds with AVX-512 disabled omit this target. The existing test_vif_simd exercises float VIF and does not replace this test.

What it covers

The fast test compares the actual scalar and AVX-512 implementations' exact numerator and denominator bits and all five final vertical planes, including reflected filter padding.

  • Bit depths 8 through 16 and all four scales.
  • Widths 9 to 257, across vector and scalar tails.
  • Textured and checkerboard input.

See Research-2046 for the separate original/current AVX-512 and sanitizer acceptance boundaries.

Float VIF convolution (ADR-0504)

The separable Gaussian convolution at the heart of the float VIF path (vif_filter1d_s, _sq_s, _xy_s in core/src/feature/vif_tools.c) accounts for roughly 60 % of float model wall time. As of ADR-0504 the hot inner loops are dispatched to:

CPU support Path Inner loop width
AVX-512F convolution_f32_avx512_s / _sq / _xy 16 floats per FMA
AVX2 only convolution_f32_avx_s / _sq / _xy 8 floats per FMA
Scalar vif_filter1d_s scalar fallback 1 float per iteration

The AVX-512 path lives in core/src/feature/common/convolution_avx512.c rather than under x86/ because the convolution helpers are shared by float_adm, float_motion, and several other extractors (not just VIF).

Numerical note. The AVX-512 path widens the FMA partial-sum tree from 8 to 16 lanes, which changes the rounding at the ULP level relative to the AVX2 path. This is accepted for the float VIF path per ADR-0214: the float path already diverges from the integer path. Netflix golden assertions pass at their declared places tolerance regardless of ISA.

float_moment AVX-512 (ADR-0987)

float_moment computes two per-frame reductions: the first statistical moment (mean) and the second (mean of squares) over a float-valued picture buffer. The AVX2 path processes 8 floats per inner-loop iteration; the AVX-512 path doubles this to 16 floats per iteration.

CPU support Path Inner loop width
AVX-512F compute_1st/2nd_moment_avx512 16 floats per iteration
AVX2 only compute_1st/2nd_moment_avx2 8 floats per iteration
Scalar compute_1st/2nd_moment 1 float per iteration

Reduction strategy. Both functions load a __m512, store to a 64-byte aligned temporary, then add each of the 16 lanes sequentially into a double accumulator. This matches the scalar accumulation order more closely than a horizontal _mm512_reduce_add_ps would, keeping the numerical residual within the 1e-7 relative tolerance tested by test_moment_simd.

Every x86 SIMD file is built without FP contraction

Rule: the AVX2 and AVX-512 libraries are compiled with the strict floating-point flags (vmaf_strict_fp_args, ADR-1415). The compiler may not turn a * b + c into a fused multiply-add; a kernel that wants one calls the intrinsic.

test_ssim_x86_simd compares the AVX2 and AVX-512 SSIM kernels with the scalar reference bit for bit, at element counts with and without a tail. With Intel's icx compiler, glibc's libm is linked too, so the math library matches across compilers (build flags, ADR-1495).

Adding a new SIMD path

Use the /add-simd-path skill — it scaffolds the <feature>_<isa>.{c,h} pair, the dispatch hook, and a golden-diff test against the scalar implementation.

Debugging a numerical divergence

If an AVX-512 path produces a different score than the scalar path, /cross-backend-diff narrows the delta to a specific feature and scale. Common root causes:

  • Float reduction order differs between scalar and SIMD — the fix is to accumulate in double precision inside the SIMD path. Example: commit 24c88a32 (float ADM sum_cube / csf_den_scale).
  • Unaligned tail load that wraps past the end of the allocation — usually a bug in the mask computation.
  • A scalar tail in plain C inside a SIMD file, contracted into a fused multiply-add by one compiler and not by another. Compare the object files (objdump -d <object> | grep -c vfmadd) of a GCC and an icx build; every x86 SIMD library takes vmaf_strict_fp_args in core/src/meson.build, so a difference points at a library that lost the flag or at a test that compiles its scalar reference without _simd_strict_fp_args.

History

FP contraction in icx builds (ADR-1415)

The scalar references the kernels are checked against live in baseline libraries, where no fused multiply-add instruction exists, and most kernels finish the last n % 16 elements of a row in plain C. An Intel compiler build (icx, which every SYCL build uses) used to contract those tails in the general libraries.

In ssim_avx512.c that moved a score: on an AVX-512 host the CPU float_ms_ssim of such a build differed from a GCC build and from its own scalar path by one fp32 unit in a per-scale mean on about one frame in thirty (7.7e-9 to 1.4e-8 in the score on the Netflix pair, the 1080p checkerboards and BBB 3840x2160).

adm, vif and speed files were contracted too, with no score difference measured. GCC does not contract in these files, and its objects are the same with and without the flag, so GCC builds were not affected and did not change.

Math library in icx builds (ADR-1495)

icx used to link Intel's libimf ahead of glibc's libm, and psnr, psnr_hvs, ciede and speed_chroma differed by at most 7.1e-15, 7.1e-15, 5.7e-12 and 1.2e-6 on the same fixtures. Since ADR-1495 an icx build links glibc's libm and these features are identical too.

References