Tiny AI¶

Tiny AI is the part of VMAFx that runs small ONNX perceptual-quality models next to the classic VMAF SVM. A tiny model can replace or augment the SVM score (full-reference), score a clip without a reference, or filter frames before encoding. libvmaf stays C-only: models run through ONNX Runtime, and training lives in the Python package ai/.

Run a shipped model¶

You need a libvmaf build with ONNX Runtime (-Denable_dnn=enabled, or auto with ONNX Runtime found by pkg-config). --tiny-model takes the path of an .onnx file, not a registry id. Models ship under model/tiny/.

vmaf -r ref.yuv -d dis.yuv -w 576 -h 324 -p 420 -b 8 \
     --tiny-model model/tiny/vmaf_tiny_v2.onnx \
     --json -o scores.json

The tiny score is an ordinary per-frame feature in the output. It is named after the name field of the model's sidecar JSON, or vmaf_tiny_model when the sidecar has no name. Look for it in frames[].metrics and pooled_metrics next to vmaf. Full details, flags and caveats are in inference.md.

Status¶

The shipped models were trained against the vmaf_v0.6.1 teacher, and model/tiny/registry.json lists 26 entries, 13 of them smoke fixtures that exercise the loader and are not quality models. The one-shot retrain against the vmaf_v1.0.16_3d0h teacher is RC9 work, see the roadmap and the retrain runbook. Treat accuracy numbers as pre-retrain.

Start here¶

Models¶

Model cards¶

One page per shipped checkpoint. Overview groups them by capability.

Quantisation¶

Data and corpora¶

Corpus ingestion¶

Evaluation¶

Retraining and operations¶