Integer accumulator bounds¶

Every feature extractor adds samples, differences, squares, cubes, counts or offsets into fixed-width integers. This page records, for every such integer in every CPU extractor, SIMD path and GPU twin, the largest value it can reach on the largest pictures VMAFx accepts. It is the result of the RC3 integer-overflow audit; the per-row tables are on three appendix pages:

• CPU extractors and SIMD paths
• CUDA and HIP twins
• SYCL and Metal twins

A bound here is derived from the code, never guessed: how many terms reach one accumulator, and the largest term. Each row names the file and line it was read from (master 571565a47), so a change to that code is a change to the row.

The envelope¶

Samples go up to 16 bits (65535; \(65535^{2}\) = 4,294,836,225 is below \(2^{32}\) but above INT32_MAX), chroma at full size (4:4:4), and the content is the worst the arithmetic allows: every sample at the maximum difference, full-range noise, or a pattern built to drive one term to its maximum. A sum over the frames of a clip is given as the number of frames after which it wraps.

Each row has one verdict:

Verdicts¶

The counts are the appendix pages' as read on master 571565a47, before the fixes below.

Defects found¶

Every row that can overflow is a defect with a row in the state ledger. Fixed ones keep the score of every input that did not overflow bit for bit; the fix and the test that fails without it are named in the row.

Not defects, recorded as DEPENDS: the CUDA motion batch counter narrows the frame index to int and needs \(2^{31}\) frames (414 days at 60 fps). Samples above \(2^{\mathrm{bpc}} - 1\) also make several CPU and SIMD integers wrap (the 16-bit motion row_sad, the AVX2 and AVX-512 motion x-convolution, the integer VIF means the SIMD paths keep un-narrowed, the 16-bit ADM DWT and the psnr_hvs DCT); they are part of the out-of-range row.

Integer ADM scale-0 rows¶

The masking reduction adds, per row of a scale-0 band, the cube ((x^2 + 2^28) >> 29) * x >> shift_cub of every column, with shift_cub = ceil(log2(band width) - 4). Compared with itself, a reference with full-range detail has a zero masking threshold everywhere, so x is the CSF-weighted band value. scripts/dev/adm_cm_row_bound.py finds the largest row exactly, by a search over the column sign patterns of the four pixel rows that feed one band row, with the library's own integer arithmetic:

python3 scripts/dev/adm_cm_row_bound.py --widths 32 64 15360 32768
python3 scripts/dev/adm_cm_row_bound.py --widths 32 --weights 45200 45300 46603
python3 scripts/dev/adm_cm_row_bound.py --pattern 64

At the default Watson weights the largest row is 1.044 INT64_MAX at a band width of 16 (pictures 31 or 32 pixels wide) and 1.021 at 32 (63 or 64), 0.86 at 16K and 0.91 at the cap: below \(2^{64}\) everywhere, about half of it. core/test/adm_cm_row_overflow_frame.h is the 64-pixel picture. Under the scale-0 weight limit of 43,900 (ADR-1917) the largest row at any weight the options can choose is 0.912 of \(2^{64}\) (--weights 43899, widths 28 to 32).

How the bounds were checked¶

Reading the code gives the bound; four runs check that the code does what the reading says on the largest pictures, with worst-case content:

• 8K exactness matrix. Every exact CUDA, SYCL and HIP twin against the CPU at 8192 x 4320 in 4:4:4, 8 and 16 bits, four frames of worst-case content: 135 of 144 cells equal, 9 n/a (cambi refuses 8K at init, psnr_hvs refuses 16 bits). See the exact-twin matrix.
• 16K CPU matrix. The CPU extractor of every exact twin at 15360 x 8640 in 4:4:4, 8 and 16 bits, with the host's AVX-512 / AVX2 dispatch against --cpumask 0xffffffff: 45 of 48 cells equal, 3 n/a.
• Integer sanitizer. A clang build with -fsanitize=signed-integer-overflow,unsigned-integer-overflow,shift,pointer-overflow scored the 8K and 16K worst-case fixtures with every CPU extractor, on the host dispatch and on the scalar code: 120 runs (19 extractors at 8K, the 12 whose memory fits at 16K; psnr_hvs at 8 bits only). 112 scored and the 8 cambi runs refused the size at init. There was no signed overflow, shift or pointer-overflow report. The 78 unsigned-wrap reports (10 runs, 27 sites in three files) are modular arithmetic the code relies on: the filter border index i - fwidth / 2 formed in unsigned and read as int (integer_vif.c, x86/vif_avx512.c), the VIF variance and covariance formed as a uint32_t difference and read as int32_t (integer_vif.c:293-295; a negative covariance, and a variance that rounding takes below 0), and the unbiased shift OD_UNBIASED_RSHIFT32 of the Xiph DCT (psnr_hvs.c).
• test_accumulator_bounds_16k (fast suite) computes the size products, shifts and counts the bounds rest on with the library's own helpers at 8K, 16K and the cap, without a picture of that size: the SpEED submatrix count (below \(2^{24}\) up to 16K for every prescale), the SpEED tail block, the integer ADM row shifts, and the CAMBI window (which passes the reciprocal table above 4K, so CAMBI refuses 8K and 16K).

Device runs at 16K, device memory limits and the time these sizes take are not part of this audit; they belong to a later release candidate.

Adding or changing an accumulator¶

When you add an integer that sums, counts or indexes over a row, a plane or a clip, write its row on the appendix page of its backend: the type, the largest term, how many terms reach it, the bound at 16K and at the cap, and the verdict. A twin's integer gets the same bound as the CPU's, or the reason it differs. A sum that can pass its type on accepted input is a defect: widen it or split it, keep every existing value bit for bit, and add a test that fails without the change.