Frequently Asked Questions¶

Short answers to common questions about scoring, inputs, training and the fork's GPU backends. The page covers both the upstream Netflix/vmaf Q&A and fork-specific entries for the CUDA / SYCL / HIP / Metal backends, SIMD paths and the tiny-AI model surface. Upstream issue numbers (for example Netflix/vmaf#20) are kept for historical context; the issues themselves are long resolved.

Scoring & models¶

Q: When computing VMAF on low-resolution videos (480-pixel height, for example), why do the scores look so high, even when there are visible artifacts?¶

A: Short answer: VMAF assumes a fixed viewing distance and display size, and a native-480 pair is judged as if it were viewed from far away, which hides artifacts and inflates the score.

Why it happens¶

Any perceptual quality model has to account for viewing distance and display size (or their ratio). The same distorted video, viewed close-up, contains more visible artifacts and so has lower perceptual quality.

The classic VMAF model (model/vmaf_float_v0.6.1.pkl, also shipped as model/vmaf_v0.6.1.json) is trained to predict the quality of videos displayed on a 1080p HDTV in a living-room environment. The fork's default CLI model, vmaf_v1.0.16_3d0h, is the same 1080p, 3H operating point (see the v1 model page).

All subjective data was collected with distorted videos rescaled to 1080 and displayed from a viewing distance of three times the screen height (3H), an angular resolution of 60 pixels per degree. The implicit assumption is: a 1080 video displayed from 3H away.

The arithmetic for a 480 pair¶

When VMAF is calculated on a 480-resolution pair, it is as if the 480 video is cropped from a 1080 video. If the 480 video has height H', then H' = 480/1080 · H ≈ 0.44 · H. VMAF then effectively models a viewing distance of 3H = 6.75 · H'.

In other words, running VMAF on a native-480 pair predicts the perceptual quality of viewing from 6.75× the screen height, which hides a lot of artifacts and inflates the score.

What to do¶

Do not compare the absolute VMAF score of a 1080 video with the score of a 480 video obtained at its native resolution. It is apples vs oranges.

To predict quality at 3× height for a 480 pair:

• If the 480 distorted video has a 1080 reference, upsample the 480 distorted to 1080 and run VMAF at 1080.
• If both are 480, upsample both to 1080 and run VMAF. The default model was not trained on upsampled references, so prediction is less accurate than the first option.

Q: Will VMAF work on 4K videos?¶

A: The classic model (model/vmaf_v0.6.1.json) was trained on videos encoded at resolutions up to 1080p. It is still useful for 4K if you only need a relative score (A vs B ordering), but absolute predictions are not guaranteed.

For 4K-specific scoring, use the dedicated 4K model model/vmaf_4k_v0.6.1.json, which predicts 4KTV viewing at 1.5× display height. See the Predict Quality on a 4KTV Screen at 1.5H section of the models document.

Q: When I compare a video with itself as reference, I expect a perfect VMAF score of 100, but I see ~98.7. Is this a bug?¶

A: No. VMAF does not guarantee a perfect score on identical inputs, but should return a value close to 100. The same is true of other ML-based predictors (e.g. VQM-VFD). The gap comes from features (ADM, VIF) that do not have a closed-form identity at the fit-polynomial output stage.

Q: How is the VMAF package versioned?¶

A: Three version numbers matter, and the fork has no VERSION file:

• The fork release is one independent SemVer stream, vX.Y.Z, defined by VMAFX_VERSION in build-config.env. Upstream alignment is recorded in sync history rather than the tag. See the release guide.
• The default model is named by VMAF_DEFAULT_MODEL_VERSION in core/include/libvmaf/model.h (vmaf_v1.0.16_3d0h). It changes only through the process in default model.
• libvmaf (the C library) carries its own version in libvmaf.pc.

The upstream rule, kept for context: Netflix's VERSION file tracked the default model consumed by VmafQualityRunner and was bumped whenever the default model changed the numerical output.

Q: Why is the aggregate VMAF score sometimes biased toward "easy" content? (upstream Netflix/vmaf#20)¶

A: The default aggregate is the arithmetic mean of per-frame scores — chosen for simplicity and consistency with other metrics (e.g. mean PSNR). Psycho-visual evidence suggests humans weigh the worst-quality frames more heavily, so mean is not necessarily optimal.

Pooling can be changed on three surfaces, all of which now understand the same eight methods — mean, harmonic_mean, median, min, max, perc5, perc10, perc20:

• Python harness: the pool_method argument to run_vmaf, run_psnr, run_vmaf_training, run_testing and friends.
• C API: the VmafPoolingMethod discriminant passed to vmaf_score_pooled / vmaf_feature_score_pooled — the order-statistic methods were added in ADR-1188 and use the same linear-interpolation rule as the harness, so both surfaces report the same number. See the C API reference.
• FFmpeg: the pool option on the libvmaf* filters (see FFmpeg usage).

The vmaf CLI has no --pool flag: its XML / JSON report always carries min, max, mean and harmonic_mean side by side, and its stdout line reports the mean. Select a percentile through the C API, a binding, or the FFmpeg filter.

Inputs & formats¶

Q: Can I pass encoded H.264 / VP9 / HEVC bitstreams to VMAF as input? (upstream Netflix/vmaf#55)¶

A: Yes — use FFmpeg's libvmaf filter. Details in the FFmpeg documentation.

Q: How do I use VMAF with downscaled videos?¶

A: If a distorted video was scaled down (e.g. for adaptive streaming) and you want to evaluate its quality at the reference resolution, use FFmpeg with the libvmaf filter to rescale on-the-fly.

For example, to upscale the distorted video to 1080p:

ffmpeg -i main.mpg -i ref.mpg \
  -filter_complex "[0:v]scale=1920x1080:flags=bicubic[main];[main][1:v]libvmaf" \
  -f null -

This scales the first input video (0:v, distorted) to 1080p and forwards it to VMAF (libvmaf) as main (pad 0), to be compared against the second input 1:v (reference, pad 1). See the FFmpeg documentation for more.

Q: Why does the included SSIM tool produce different numbers than other SSIM implementations?¶

A: The SSIM implementation in VMAF includes the empirical downsampling step from the Suggested Usage section of SSIM. FFmpeg's ssim filter, for example, does not include this step.

Q: Why are PSNR values capped at 60 dB for 8-bit inputs, 72 dB for 10-bit, 84 dB for 12-bit, and 108 dB for 16-bit?¶

A: The caps follow the rule of thumb \(6N + 12\), where \(N\) is the bit depth.

This approximates the more precise \(10 \log_{10} \dfrac{(2^N - 1)^2}{1/12}\), where \(2^N - 1\) is the peak signal of an N-bit representation and 1/12 is the variance of Uniform[0, 1]-distributed quantisation noise (the noise of rounding a real-valued signal to its integer representation). For N = 8, the precise formula yields 58.92 dB; the rule-of-thumb rounds to 60.

It is true that an 8-bit distorted signal could match an 8-bit reference exactly and produce infinite PSNR, but that only happens because the reference is itself in 8-bit representation. In general, references come from higher-bit or floating-point sources, and PSNR is bounded by the quantisation-noise floor above. A bit-depth-based PSNR cap reflects that more realistic fidelity ceiling.

Training¶

Q: If I train a model with run_vmaf_training and then test with run_testing on the same dataset, why don't I get the same SRCC / PCC / RMSE? (upstream Netflix/vmaf#191)¶

A: The two scripts use slightly different pipelines.

run_vmaf_training extracts per-frame feature scores, temporally pools each feature (arithmetic mean) into a per-clip feature score, then fits the model against subjective scores. The reported metrics are the fitting result.

run_testing extracts per-frame features, applies the prediction model per-frame, and then arithmetic-means the per-frame VMAF scores into a clip score. This is a re-ordering of "temporal pooling" and "prediction".

If features are constant across a clip, the order does not matter. In practice, the difference is small but non-zero.

Q: How do I train a custom VMAF model?¶

A: See the upstream Python library guide. For the fork's tiny-AI models (ONNX-based quality predictors), see docs/ai/training.md and the ai/ package (pip install -e ai && vmaf-train --help).

Performance & backends (fork-specific)¶

Q: Does the fork support GPU acceleration?¶

A: Yes. The fork adds CUDA, SYCL, HIP and Metal backends on top of upstream (see backends/index.md). Build with the backends you want, from the repository root:

meson setup build core -Denable_cuda=true -Denable_sycl=true
ninja -C build

See backends/ for per-backend notes and development/build-flags.md for every meson_options.txt option.

At run time, a binary built with GPU backends picks one automatically. To control the choice:

• Pin one backend: --backend cuda|sycl|hip|metal|cpu (default auto). A backend that is not compiled in fails with an error instead of falling back silently.
• Opt out of one backend: --no_cuda, --no_sycl, --no_hip, --no_metal.
• Pick a device: --sycl_device N, --hip_device N, --metal_device N.

The CPU path with AVX2 / AVX-512 / NEON SIMD is the universal fallback.

Q: How do I get bit-exact round-trippable VMAF output?¶

A: Pass --precision=max to the fork-added flag on the vmaf CLI. The default format string is %.6f (alias --precision=legacy, Netflix-compatible per ADR-0119); --precision=max (alias full) selects %.17g for IEEE-754 round-trip lossless output. --precision=N (1..17) overrides with "%.<N>g".

Q: What are the "tiny-AI" models and how do I use them?¶

A: Tiny-AI is a fork-added surface for small ONNX perceptual quality models that run through ONNX Runtime inside libvmaf. See docs/ai/overview.md for the architecture and docs/ai/inference.md for the CLI.

The fork now ships registry-pinned ONNX artefacts under model/tiny/, including VMAF-tiny FR regressors, codec-aware FR regressors, LPIPS-SqueezeNet, saliency students, TransNet V2, and learned pre-filter models. Check model/tiny/registry.json for the authoritative list and each row's smoke flag; smoke entries are CI or compatibility probes, not production quality models. Invocation is:

vmaf --reference ref.y4m --distorted dis.y4m \
  --tiny-model model/tiny/vmaf_tiny_v2.onnx \
  --tiny-device auto

Production deployments should combine the model registry with --tiny-model-verify and, where appropriate, the VMAF_TINY_MODEL_DIR path jail documented in docs/ai/security.md.

Q: Does the fork preserve Netflix's golden-data numerical contract?¶

A: Yes, on the CPU path. The three canonical Netflix reference test pairs (src01/hrc00–hrc01, checkerboard 1-pixel shift, checkerboard 10-pixel shift) run in CI as a required status check, and the CPU scalar + fixed-point path must match upstream exactly.

What the GPU and SIMD paths guarantee is different:

Each exact twin is one file in scripts/ci/exact_twins.d/<feature>.<backend>; the generated table of every exact twin, with its evidence, is exact GPU twins. A twin that is not listed there differs from the CPU by small ULP-level deltas (different reduction orders and FMA contraction), typically below the six decimals of the default %.6f output; --precision=max (%.17g) exposes them.

GPU and SIMD paths are also regression-tested by fork-added snapshot JSONs under testdata/ (per-backend, ULP tolerance, regenerated via /regen-snapshots), not by the Netflix goldens. See engineering principles §3.1 for the scope of the CPU gate.

Applications¶

Q: Will VMAF work on applications other than HTTP adaptive streaming?¶

A: VMAF was designed with HTTP adaptive streaming in mind. It targets compression artifacts and scaling artifacts (see Netflix's tech blog post for context). Artifacts outside that space — transmission errors, packet loss, pre-capture noise, encoder-specific pathologies — may be predicted inaccurately.