vmafx-tune-go — Go port of vmaf-tune¶

vmafx-tune-go is the Go port of the vmaf-tune rate-quality tuning CLI: it finds the encoder settings (CRF, preset, codec) that hit a VMAF target at the lowest bitrate. All fourteen vmaf-tune subcommands are ported: compare, ladder, report, recommend, predict, recommend-saliency, prefilter, tune-per-shot, fast, corpus, sidecar, benchmark, encode-profile and auto. It is the active tuning binary after the retirement of the Python CLI shadow.

This page documents the Go binary. For the overview of the tuning workflow and the Python vmaf-tune CLI, start at vmaf-tune.md. A few individual flags still require Python (see Python-only flags), and the flag spellings of the three oldest ports differ from Python (see Python vs Go differences).

Build¶

Go 1.27 or newer is required (GO_VERSION in build-config.env; go.mod declares go 1.27.2).

go build -o vmafx-tune-go ./cmd/vmafx-tune
# or with version injection:
go build -ldflags "-X main.version=$(cat VERSION)" -o vmafx-tune-go ./cmd/vmafx-tune

The binary installs as vmafx-tune-go, not vmaf-tune, so it never collides with the Python entry point. The dev container builds it with go build ./cmd/... and ships it as vmafx-tune; the package is pure Go and builds without cgo.

External tools used at run time:

Subcommands at a glance¶

A typical first run needs only a reference clip and two binaries on PATH:

vmafx-tune-go compare --reference src.mp4 --targets 85,90 --output results.json
vmafx-tune-go report results.json --format html --output report.html

compare: rate-quality sweep¶

compare runs a VMAF-target bisect for each (codec, target) pair and emits a ranked report. Stage 1 supports software encoders only: libx264 and libx265.

The vmaf binary reads only Y4M (or raw YUV with explicit geometry), so the scorer decodes the reference to Y4M once per run, unless it already is a .y4m file, and decodes each Matroska encode to Y4M as it is scored. The reference can therefore be a .y4m file or any container FFmpeg reads; a headerless .yuv reference is refused, because the scorer has no geometry for it. The decoded files are written to --work-dir and removed after use.

vmafx-tune-go compare --reference <video> [flags]

Single target, JSON output:

vmafx-tune-go compare \
  --reference src.mp4 \
  --codecs libx264,libx265 \
  --targets 85 \
  --output results.json

Multi-target sweep, Markdown output:

vmafx-tune-go compare \
  --reference src.mp4 \
  --codecs libx264 \
  --targets 80,85,90,95 \
  --format markdown

compare output schema¶

A single --targets value produces schema-v1 JSON, identical to the Python vmaf-tune compare JSON output:

{
  "src": "src.mp4",
  "target_vmaf": 85.0,
  "tool_version": "dev",
  "wall_time_ms": 4200,
  "rows": [
    {
      "codec": "libx264",
      "adapter": "...",
      "runtime_variant": "...",
      "ffmpeg_bin": "ffmpeg",
      "encoder_version": "...",
      "best_crf": 23,
      "bitrate_kbps": 1234.5,
      "encode_time_ms": 2100,
      "vmaf_score": 85.34,
      "target_vmaf": 85.0,
      "ok": true,
      "error": "",
      "bisect_samples": [...]
    }
  ]
}

Several --targets values produce schema-v2 output (adds schema_version: 2 and target_vmafs: [...]), also compatible with the Python report.py renderer. Non-finite float values (NaN, Inf) are written as null, so the output is RFC 8259 strict JSON that Go, Rust and jq --strict all parse.

ladder: per-title ABR ladder¶

ladder builds an ABR bitrate ladder for one source. For each (resolution, VMAF target) cell it bisects the highest CRF that still meets the target. The resulting (bitrate, VMAF) cloud is reduced to its upper convex hull (the Pareto frontier), and a small set of knee renditions is selected from the hull.

Every cell runs at its own resolution, as the Python vmaf-tune ladder does: each probe encode scales the source with -vf scale=W:H, and the reference is decoded to Y4M through the same filter once per cell, so a 640x360 rung is encoded at 640x360 and scored against a 640x360 reference. When no cell produces a scored encode, ladder exits 2 with the first cell's error and writes no ladder.

Each cell is also scored with the VMAF model its own height selects, the rule of resolution-aware model selection: a rung 2160 lines or taller uses vmaf_v1.0.16_1d5h_2160, every lower rung vmaf_v1.0.16_3d0h, passed to libvmaf as --model version=.... A ladder that mixes 1080p and 2160p rungs therefore scores them with different models, as the Python ladder does. The Go ladder has no --vmaf-model or --neg flag yet.

vmafx-tune-go ladder --reference <video> [flags]

vmafx-tune-go ladder \
  --reference src.mp4 \
  --codec libx264 \
  --targets 75,85,95 \
  --output ladder.json

The JSON is a superset of the Python vmaf-tune ladder schema (schema version 1: src, encoder, target_vmafs, resolutions, tool_version, wall_time_ms, cloud, hull, renditions) and works with the existing HLS/DASH manifest renderer. Failed points carry ok: false, and non-finite bitrate or VMAF values are written as 0 rather than null.

report: render Markdown or HTML¶

report reads one or more JSON files produced by compare or ladder and renders a human-readable report. It auto-detects whether each input is a compare or a ladder file (it probes the top-level rows and renditions keys) and merges them into one report, sorted by tool_version and src.

vmafx-tune-go report <input.json> [input2.json ...] [flags]

At least one input file is required.

vmafx-tune-go report results.json

Merge compare and ladder results into one HTML report:

vmafx-tune-go report compare.json ladder.json \
  --format html \
  --output report.html

The HTML output is self-contained (inlined CSS, no external dependencies) and renders without a network connection. PDF and JSON-diff modes are not implemented.

recommend: pick the CRF meeting a target¶

recommend has two modes. With --from-corpus it picks from an existing corpus JSONL and runs no new encodes. Without it, a coarse-to-fine CRF search runs the encodes first, writes every visited point to --output, and picks from those rows.

vmafx-tune-go recommend [flags]

The ML-driven group (recommend, predict, recommend-saliency, prefilter) emits JSON that is byte-identical to the Python originals: key order, the ", " and ": " separators, NaN / Infinity tokens and float rendering all match json.dumps. A downstream consumer cannot tell which binary produced a file.

recommend predicates¶

The two predicates are mutually exclusive.

recommend source and encode flags¶

--source and --preset are required unless --from-corpus is used.

recommend search flags¶

With the defaults that is 5 coarse plus up to 10 fine encodes, against 52 for a full sweep (ADR-0296). The CRF window is fixed at 10–50, matching the Python defaults, so a Go and a Python run over the same source visit the same cells and their corpora stay comparable. See vmaf-tune-coarse-to-fine.md and vmaf-tune-recommend.md.

recommend uncertainty flags¶

These flags implement ADR-0393.

A tight interval whose lower bound already clears the target short-circuits the search at that row. A wide interval refuses to short-circuit and tags the result (UNCERTAIN). This changes which encodes get probed, never which get shipped; the production-flip gate stays in the predictor's validation harness.

recommend examples¶

Pick from an existing corpus, machine-readable:

vmafx-tune-go recommend --from-corpus corpus.jsonl --target-vmaf 93 --json

Run the search, then pick:

vmafx-tune-go recommend \
  --source src.yuv --width 1920 --height 1080 --duration 10 \
  --preset medium --target-vmaf 93 --output corpus.jsonl

predict: predict per-shot VMAF, then verify¶

predict predicts VMAF per shot from cheap signals, then verifies the prediction against real libvmaf on K stratified shots and emits a verdict.

vmafx-tune-go predict --source <video> [flags]

predict verdicts¶

Shot detection degrades to a single shot spanning the clip when vmaf-perShot is unavailable, with a WARN log line.

vmafx-tune-go predict --source movie.mkv --codec libx264 \
  --target-vmaf 93 --validate-k 8 --report-out predict.json

recommend-saliency: saliency-aware ROI encode¶

recommend-saliency scores the fork's saliency_student_v1 model over sampled frames, reduces the result to a per-block QP-offset map, and hands it to the encoder through its native ROI channel.

vmafx-tune-go recommend-saliency --src <yuv> --width W --height H \
  --duration-frames N --output <path> [flags]

--src, --width, --height, --duration-frames and --output are required.

ROI channel per encoder¶

Any other encoder exits 2 unless --saliency-fallback-plain is set or VMAFTUNE_SALIENCY_FALLBACK_OK=1 is exported. See also vmaf-tune-saliency-aware.md.

prefilter: joint deband and CRF autotune¶

prefilter optimises the ten frozen Pelorus deband knobs (the ADR-0110 control-plane contract) and the CRF axis together in one TPE study, with VMAF as the oracle.

vmafx-tune-go prefilter --target-vmaf T [flags]

The objective is |achieved - target| + λ·kbps, so the search converges on the lowest-bitrate combination that hits the target. The bitrate weight is small enough that it only breaks ties between equally good quality points.

vmafx never runs the deband filter itself; it only emits the -vf string. The live loop therefore requires the pelorus_deband_vulkan filter in the ffmpeg build and refuses to start (exit 2) with an actionable message when it is absent. This is an FFmpeg filter, unrelated to the removed libvmaf Vulkan scoring backend (ADR-0726). --smoke exercises the whole search without it.

The ten swept knobs are range, thry, thrc, grainy, grainc, softness, detail, dither, dynamic and protect. The deliberately out-of-contract options (sample, blur, planes, meta) are pipeline switches set once per run and are rejected if passed to --sweep-knob.

# CI-friendly: no ffmpeg, no Vulkan filter, no GPU.
vmafx-tune-go prefilter --smoke --target-vmaf 93 --output rec.json

# Live loop.
vmafx-tune-go prefilter \
  --src ref.yuv --width 1920 --height 1080 --duration 10 \
  --target-vmaf 93 --encoder libx264 --output rec.json

Unlike the Python subcommand, this needs no optional [fast] extra: the TPE sampler is implemented natively (see TPE sampler trajectory).

fast: proxy, TPE and GPU-verify recommend¶

fast recommends a CRF for a VMAF target without running the full Phase A grid. A TPE (Tree-structured Parzen Estimator) search walks the integer CRF axis. Each trial encodes a short probe slice, extracts the canonical-6 libvmaf features, and predicts VMAF with the fr_regressor_v2 proxy. One real encode plus libvmaf score at the chosen CRF then verifies the recommendation: the proxy alone never wins (ADR-0304). The pick is the lowest predicted bitrate among the CRFs that meet the target, as in recommend. See also vmaf-tune-fast-path.md.

vmafx-tune-go fast --target-vmaf <N> [--smoke | --src <file> --width W --height H] [flags]

Smoke run (works on any host):

vmafx-tune-go fast --smoke --target-vmaf 90

fast flags¶

fast exit codes¶

These are identical to vmaf-tune fast.

Production-mode blocker: ONNX named inputs¶

The shipped proxy model/tiny/fr_regressor_v2.onnx declares two named input ports: features (shape [N, 6]) and codec (shape [N, 14]), as recorded in its sidecar's "input_names". The only ONNX inference seam in the Go tree, pkg/ai.Registry.Infer, serialises a single flat []float64 to the vmafx-ort-runner subprocess and has no wire format for a second port. Registry.InferDirect, the CGO path, is an explicit Stage-2 stub.

Flattening the two ports into one 20-D vector is not a workaround. vmaftune/proxy.py documents that exact mistake: the graph's first dense layer reads the 6-D features port only, so the 14 codec dimensions are silently interpreted as batch padding and codec receives nothing. Rather than return a quietly wrong score, pkg/fast fails with ErrProxyPortsUnsupported and a diagnostic naming both ports.

Any one of these unblocks it:

1. A vmafx-ort-runner protocol that accepts named input tensors, plus a matching pkg/ai.Registry.InferNamed. The runner is in-tree since ADR-1134 (vmafx-ort-runner.md), so this is a protocol extension rather than an external dependency.
2. Promoting pkg/ai.Registry.InferDirect onto a CGO ONNX Runtime binding (for example github.com/yalue/onnxruntime_go), which pkg/ai defers to Stage 2 because it couples the build to libonnxruntime.
3. A single-port re-export of fr_regressor_v2 that concatenates the two inputs inside the graph, shipped alongside the current model.

Parity with the Python fast implementation¶

The Go port found four defects in vmaf-tune fast; the Python path was fixed to match on 2026-09-04 (T-VMAFTUNE-FAST-PY-PROBE-BROKEN-2026-08-30 in the bug ledger). Both implementations now:

1. decode container-shaped probe and verify encodes to raw YUV before the libvmaf CLI scores them;
2. read the integer_-prefixed canonical-6 keys first, then the bare key, then a per-frame average;
3. place the codec one-hot by the model's vocabulary (libvvenc at index 3, unknown at index 11), mapping a codec outside it to unknown. Go reads the vocabulary from the model sidecar's encoder_vocab; Python keeps proxy.ENCODER_VOCAB_V2 aligned with that sidecar and names the substitution on stderr and as proxy_encoder_slot in its JSON;
4. standardise the features with the sidecar's feature_mean / feature_std before inference.

tools/vmaf-tune/tests/test_fast_parity.py::test_e2e_probe_extraction_parity runs both on the same clip and holds the raw and normalised features to 1e-6.

One behaviour is weaker in Go than in Python: TPE reproducibility. Optuna's TPESampler(seed=0) makes a run bit-reproducible. The Go port uses github.com/c-bata/goptuna, whose TPE sampler honours its seed only partially:

• tpe.SamplerOptionSeed seeds the sampler's own RNG and its startup random sampler.
• goptuna/internal/random.ArgMaxMultinomial draws from the process-global math/rand source, which Go seeds randomly at startup and which rand.Seed can no longer override.

Repeat runs therefore explore slightly different trial sequences and may return a neighbouring CRF when two candidates score within about a VMAF point of each other. Measured on the ADR-0276 smoke curve: at the shipped budgets the recommendation stays within ±1 of the brute-force optimum in about 99–100 % of runs, and at 150 trials it hit the exact optimum in 150 of 150 runs for every target tested. Closing the gap needs an upstream goptuna change threading the sampler RNG into internal/random.

auto: Phase F adaptive planner¶

auto composes the per-phase tuning stages into one deterministic decision tree and emits a JSON plan. Optionally it realises the winning cell as a real encode plus a libvmaf score.

vmafx-tune-go auto --src <video> [flags]

--src is required unless --smoke is given: the smoke planner probes nothing, and the plan then records an empty src. --execute always needs --src. A missing --src exits 2.

--model is a Go-side addition. The Python auto driver always constructs its predictor without a model path, which is the analytical fallback this flag defaults to. Supplying a model routes inference through the ONNX bridge in pkg/ai, which degrades back to the analytical curve (with a warning) when the ORT runner is not on PATH.

Plan only:

vmafx-tune-go auto \
  --src src.mp4 \
  --target-vmaf 93 \
  --allow-codecs libx264,libx265

Plan and realise the winner:

vmafx-tune-go auto \
  --src src.mp4 \
  --target-vmaf 95 \
  --max-budget-bitrate 6000 \
  --execute --runs-dir runs/

How the plan is built¶

1. Probe the source: geometry and duration via ffprobe, HDR signalling via the colour-metadata classifier. Every probe failure degrades to conservative defaults (1920x1080, duration 0, SDR) rather than aborting the run.
2. Apply the content recipe. The content class selects a small override set: a narrower or wider conformal-confidence gate, a forced single-rung ladder, a saliency intensity, and a target-VMAF offset. An HDR source carrying only a generic live_action label is promoted to the HDR recipe. The overrides load from ai/data/phase_f_recipes_calibrated.json when that file is reachable, and fall back to the documented placeholders with a one-line warning otherwise.
3. Walk the ten short-circuits, recording each one that fires under metadata.short_circuits.
4. Estimate each (rung, codec) cell: invert the predictor for a CRF, then estimate the VMAF and bitrate that CRF would produce.
5. Pick a winner against the VMAF target and the bitrate budget.

Short-circuits¶

Each predicate names a stage the tree can skip. They are evaluated in this order, and the order is part of the output contract.

Confidence-aware escalation¶

Each cell carries a conformal interval width, and that width decides whether the predictor's own verdict is overridden.

Without a calibration sidecar the interval is uncalibrated, so cells carry NaN and no override happens. The 2.0 / 5.0 defaults are an emergency floor, not a corpus fit.

Plan JSON¶

The plan is a {"cells": [...], "metadata": {...}} object with sorted keys, byte-compatible with the Python vmaf-tune auto output. Each cell carries rung, codec, verdict, crf, estimated_vmaf, estimated_bitrate_kbps, hdr_args, sample_clip_seconds, confidence_decision, interval_width, effective_predictor_target_vmaf, prediction_source, saliency_intensity and selected.

metadata.winner.status is one of:

Execute mode¶

With --execute the selected cell is encoded and scored, and one row per executed cell is appended to <runs-dir>/tune_results.jsonl. Encoded files land beside it as encode_<NNN>_<codec>_<preset>_crf<n>.mkv (NNN is the zero-padded cell index).

A failed encode is recorded in its row with a non-zero encode_exit_status, and scoring is skipped for that cell. The command exits non-zero only when cells were executed and none scored successfully.

corpus: Phase A grid sweep¶

corpus sweeps a (preset, crf) grid against one or more references, encodes each cell, scores it against the reference with the libvmaf CLI, and writes one JSONL row per (source, preset, crf) combination.

vmafx-tune-go corpus [flags]

--source, --width, --height and --preset are always required, and --crf is required unless --coarse-to-fine derives the axis.

The JSONL schema (v3) is the API contract the Phase B target-VMAF bisect and the Phase C per-title CRF predictor consume; see vmaf-tune-corpus.md for the column reference. The Go writer emits the same bytes the Python writer does, including the bare NaN tokens CPython's json module produces for columns libvmaf did not populate, so a corpus written by either binary is readable by the same trainers.

When scoring with models that omit VIF (such as the default vmaf_v1.0.16_3d0h model, ADR-1168 / ADR-1169), every Go libvmaf driver (corpus, fast, scorecli, tune/executor) automatically passes --feature vif to libvmaf and parses options-suffixed keys. This keeps the canonical-6 columns (adm2, vif_scale0..3, motion2) populated.

corpus source and encode flags¶

corpus scoring flags¶

corpus search-mode flags¶

corpus HDR flags¶

The four flags are mutually exclusive; see vmaf-tune-hdr-and-sampling.md.

Full grid:

vmafx-tune-go corpus \
  --source ref.yuv \
  --width 1920 --height 1080 \
  --framerate 24 --duration 10 \
  --preset medium \
  --crf 20 --crf 26 --crf 32 \
  --output corpus.jsonl

Coarse-to-fine against a VMAF target:

vmafx-tune-go corpus \
  --source ref.yuv \
  --width 1920 --height 1080 \
  --preset medium \
  --coarse-to-fine --target-vmaf 93 \
  --output corpus.jsonl

corpus run behaviour¶

Rows stream to the output file as each cell completes, so an interrupted sweep leaves a usable partial corpus. The selected scoring backend is echoed on stderr (vmafx-tune: scoring backend = cpu) before the first encode, and the row count is echoed when the sweep finishes.

When --keep-encodes is off, cleanup is part of a successful cell. A missing temporary encode is harmless (an injected runner may already have removed it), but any other removal failure stops the sweep instead of silently leaking data. Likewise, when source hashing is enabled, an unreadable source fails before the first encode; use --no-source-hash only when provenance is deliberately not required.

benchmark: rank encoders from a corpus¶

benchmark answers the standard post-sweep question: which encoder hit the target quality at the lowest bitrate? It reads a Phase-A corpus JSONL written by corpus and launches no ffmpeg and no libvmaf; the corpus stays the source of truth.

vmafx-tune-go benchmark --from-corpus <JSONL> [flags]

For every encoder in the corpus the report picks the lowest-bitrate row whose measured VMAF clears --target-vmaf. An encoder that never clears is reported with status unmet and its closest miss, so a missing encoder build is never mistaken for a quality result.

Markdown to stdout:

vmafx-tune-go benchmark --from-corpus corpus.jsonl

CSV at a stricter target, with a pinned baseline:

vmafx-tune-go benchmark \
  --from-corpus corpus.jsonl \
  --target-vmaf 95 \
  --baseline-encoder libx264 \
  --format csv \
  --output benchmark.csv

Rows are ranked with cleared encoders first (ascending bitrate), then the unmet ones. A row reports:

Rows are excluded from the report when exit_status is non-zero, when vmaf_score or bitrate_kbps is missing or non-finite, or when the row carries no encoder name.

The CSV output uses CRLF line endings (Python's csv "excel" dialect), and the JSON output is stable pretty RFC 8259 with sorted keys. Both are byte-identical to vmaf-tune benchmark.

encode-profile: reproduce one recommendation¶

Every vmaf-tune report embeds a machine-readable encoder_profile payload (the contract is ADR-0643). encode-profile reads that payload, selects one recommendation, and runs the matching FFmpeg encode.

vmafx-tune-go encode-profile --profile <FILE> --output <FILE> [flags]

The profile is accepted in any of the three shapes a report ships in:

Selection defaults to the first Pareto-selected row with the lowest bitrate. --codec and --target-vmaf narrow the candidate set; --recommendation-index then picks the Nth survivor (zero-based, applied after filtering).

Override flags (each falls back to the profile value when omitted):

Inspect the selection without encoding:

vmafx-tune-go encode-profile \
  --profile report.json \
  --output /dev/null \
  --dry-run

Reproduce the best x265 row at target 95:

vmafx-tune-go encode-profile \
  --profile report.html \
  --codec libx265 \
  --target-vmaf 95 \
  --output encoded.mkv

encode-profile output¶

The result is a single JSON document on stdout (sorted keys, two-space indent). A --dry-run emits ok, dry_run, profile, selected, ffmpeg_argv and output. A real run replaces dry_run with the encode outcome:

{
  "ok": true,
  "profile": "report.json",
  "selected": { "codec": "libx265", "crf": 28 },
  "ffmpeg_argv": ["ffmpeg", "-y", "-hide_banner"],
  "output": "encoded.mkv",
  "exit_status": 0,
  "encode_size_bytes": 148213,
  "encode_time_ms": 1843.2,
  "encoder_version": "libx265-3.5",
  "ffmpeg_version": "n8.1",
  "stderr_tail": "..."
}

On a real run the process exit status is FFmpeg's own, so a failed encode surfaces the encoder's code (for example 254) rather than a generic 1. The JSON payload is still written first, so a wrapper script can read exit_status and stderr_tail regardless.

Hardware-encoder caveats¶

• A QSV row's argv carries FFmpeg's QSV device chain, -init_hw_device vaapi=va:<node> -init_hw_device qsv=qsv_dev@va -filter_hw_device qsv_dev, before the first -i, and format=nv12,hwupload=extra_hw_frames=64 at the end of its -vf chain (see ADR-0601), exactly as the Python vmaf-tune builds every QSV encode. The render node is VMAFTUNE_VAAPI_DEVICE, else the first Intel render node, else /dev/dri/renderD128.
• Hardware encoders (NVENC / QSV / AMF) reject sources below roughly 320x240. A profile built from a smaller clip fails at the encoder.
• av1_videotoolbox is a placeholder: upstream FFmpeg ships no such encoder, so the adapter refuses to emit an argv shape it cannot verify (ADR-0339).

tune-per-shot: per-shot CRF tuning¶

tune-per-shot cuts the source into shots, bisects a CRF against the VMAF target inside each shot, and emits an FFmpeg encoding plan: one command per segment plus the concat-demuxer command that stitches them into the final file.

The plan is emitted, not executed. Segment encodes are independent, so you can run them sequentially or in parallel, then run the concat command.

vmafx-tune-go tune-per-shot --src <video> [flags]

Pipeline:

1. Shot detection. Shells out to the fork's vmaf-perShot binary (vmaf-perShot.md), which wraps TransNet V2. A missing or failing binary degrades to one shot spanning the clip.
2. Uniform-window splitter. Any shot longer than --max-shot-duration is sliced into equal sub-shots, so an under-cutting detector (fades, short clips) still yields a usable timeline.
3. Per-shot bisect. Each shot is extracted to raw YUV and run through the CRF bisect against --target-vmaf.
4. Plan emission. The recommendations become segment and concat commands.

tune-per-shot flags¶

--src is required. Source geometry:

Shot detection:

Tuning:

Output:

vmafx-tune-go tune-per-shot \
  --src src.mp4 \
  --target-vmaf 92 \
  --encoder libx264 \
  --plan-out plan.json \
  --segment-dir ./segments \
  --output final.mp4

Plan JSON schema¶

The plan is byte-compatible with vmaf-tune tune-per-shot: keys are sorted, floats keep Python's repr() form, and a shot whose predicate never measured a bitrate carries null rather than NaN.

{
  "concat_command": ["ffmpeg", "-y", "-hide_banner", "-f", "concat", "..."],
  "encoder": "libx264",
  "framerate": 24.0,
  "predicate": "bisect",
  "segment_commands": [["ffmpeg", "-y", "-hide_banner", "-ss", "0.000000", "..."]],
  "shots": [
    {
      "bitrate_kbps": 1234.57,
      "crf": 22,
      "end_frame": 48,
      "predicted_vmaf": 92.5,
      "start_frame": 0
    }
  ],
  "target_vmaf": 92.0
}

start_frame is inclusive and end_frame exclusive (half-open). The vmaf-perShot sidecar uses an inclusive end frame; the tuner normalises it.

The concat listing (concat.txt) is written next to --plan-out when --segment-dir is not given, matching the Python. Pass --segment-dir explicitly to pin the listing and the segment commands to the same directory; the command logs a WARN when the two diverge.

Supported codecs¶

tune-per-shot accepts the ten codecs the Go encoder registry can construct. Each emits its own quality knob in the plan:

The Python registry carries seven more adapters: av1_nvenc, av1_qsv, av1_amf, the four VideoToolbox encoders, libvvenc and libvpx-vp9. They have no Go encoder implementation yet, so --encoder rejects them by name and points at the Python binary.

sidecar: local predictor sidecar¶

sidecar trains and inspects a bias-correction term on top of the shipped predictor:

The shipped predictor is never mutated, so model upgrades stay deterministic and reproducible across hosts.

vmafx-tune-go sidecar <status|predict|record|batch-record> [flags]

Flags shared by every nested subcommand:

Nested subcommands:

Inspect, train from a capture log, then predict:

vmafx-tune-go sidecar status --json
vmafx-tune-go sidecar batch-record --captures-jsonl captures.jsonl --json
vmafx-tune-go sidecar predict --features-json shot.json --crf 26 --json

sidecar status¶

sidecar status prints the state metadata for one codec bucket: the anonymous host UUID, the on-disk state path, the predictor-version namespace, the number of folded-in captures, and the RMS of the buffered residuals (the drift signal).

vmafx-tune-go sidecar status --codec libx264 --json

{
  "codec": "libx264",
  "host_uuid": "0123456789abcdef0123456789abcdef",
  "n_updates": 0,
  "predictor_version": "predictor_v1",
  "recent_residual_rms": 0.0,
  "schema": "vmaf-tune-sidecar-status/v1",
  "schema_version": 1,
  "state_path": "/home/u/.cache/vmaf-tune/sidecar/predictor_v1/libx264/state.json"
}

Without --json the same fields print as one line:

codec=libx264 predictor_version=predictor_v1 updates=0 residual_rms=0.000000 state=/home/u/.cache/vmaf-tune/sidecar/predictor_v1/libx264/state.json

sidecar predict¶

sidecar predict predicts VMAF for one shot at one CRF with the correction applied.

vmafx-tune-go sidecar predict --features-json shot.json --crf 26 --json

The payload (schema vmaf-tune-sidecar-predict/v1) carries base_vmaf (the bare predictor), correction, and sidecar_vmaf (the sum, clamped to [0, 100]), plus codec, crf and n_updates. The text form is base=<b> correction=<c> sidecar=<s> updates=<n> with six decimals.

sidecar record¶

sidecar record folds one observed VMAF measurement into the ridge fit.

vmafx-tune-go sidecar record \
  --features-json shot.json \
  --crf 26 \
  --observed-vmaf 91.75 \
  --json

The residual is computed against the bare predictor, never against the sidecar-corrected value, so repeated captures converge rather than compounding. The payload (schema vmaf-tune-sidecar-record/v1) is the status payload plus crf, observed_vmaf, base_vmaf and residual (observed_vmaf - base_vmaf); the text form is recorded updates=<n> residual=<r> state=<path>.

sidecar batch-record¶

sidecar batch-record folds a whole JSONL capture file into the fit, one observation per line.

vmafx-tune-go sidecar batch-record --captures-jsonl captures.jsonl --json

Each line is a JSON object carrying the feature fields (either at the top level or nested under a features key) plus crf and observed_vmaf. Lines are split on \n, \r\n or a lone \r (CPython's universal-newline rule) with no length limit; blank lines are ignored but still numbered.

A malformed row is not fatal. That covers a row that is not a JSON object, a missing or non-numeric required field, and a crf string that is not an integer literal. It is reported as vmafx-tune sidecar batch-record: skip line <n>: <reason> on stderr, counted in rows_skipped, and the run still succeeds. That is deliberate: a capture log is often partially corrupt after an interrupted run, and losing the good rows to one bad line would be worse.

State is written once at the end, and only when at least one row was recorded. The payload (schema vmaf-tune-sidecar-batch-record/v1) is the status payload plus rows_recorded and rows_skipped; the text form is recorded=<rows> skipped=<rows> updates=<n> state=<path>.

Feature JSON¶

--features-json takes an object of shot features, or a {"features": {...}} wrapper so a capture row can carry crf and observed_vmaf alongside. The four probe_* fields are required (a zero probe bitrate would train the fit on a fabricated complexity barometer); everything else defaults to 0. Values may be JSON numbers or numeric strings ("2400"), as CPython's float() accepts.

{
  "probe_bitrate_kbps": 4200.5,
  "probe_i_frame_avg_bytes": 51234.0,
  "probe_p_frame_avg_bytes": 8123.25,
  "probe_b_frame_avg_bytes": 2011.75,
  "saliency_mean": 0.42,
  "saliency_var": 0.031,
  "frame_diff_mean": 7.5,
  "y_avg": 112.25,
  "y_var": 1830.5,
  "shot_length_frames": 240,
  "fps": 24.0,
  "width": 1920,
  "height": 1080
}

State and privacy¶

State lives at:

${XDG_CACHE_HOME:-~/.cache}/vmaf-tune/sidecar/
  host-uuid                                    # random 128-bit token
  <predictor-version>/<codec>/state.json       # ridge weights + inverse Gram

The host UUID is drawn from a CSPRNG on first use. It is never derived from a MAC address, hostname, /etc/machine-id, CPUID, or any other machine-identifying signal.

A predictor-version or schema mismatch on load discards the fit and resets to cold start, keeping only the host UUID. That is what makes a shipped-model upgrade safe: a stale correction can never be replayed against a refreshed predictor. At cold start the weights are zero, so the correction is exactly 0.0 and the sidecar returns the bare predictor's value untouched.

A corrupt state.json also cold-starts, including one whose weights or a_inv carry null (the residue of a NaN capture). The corrupt file is left in place so you can inspect it.

Sidecar exit codes and diagnostics¶

The four subcommands follow the Python vmaf-tune sidecar exit contract:

Diagnostics go to stderr and are not byte-identical to Python's: cobra prefixes them with Error: where Python prints vmaf-tune sidecar <cmd>:, the batch-record skip lines carry the vmafx-tune prefix, and the reason text is Go's rather than CPython's exception message. Nothing is written to stdout on failure.

Byte compatibility with vmaf-tune sidecar¶

For the same inputs, every stdout payload (JSON and text form) and every byte of state.json written by the Go binary is identical to the Python CLI's. The analytical predictor, the Sherman–Morrison update, and the json.dumps(..., indent=2, sort_keys=True) rendering are reproduced to the last bit, and the atomic write leaves the same single state.json behind (no .tmp residue).

cmd/vmafx-tune/cmd/testdata/sidecar/ holds fixtures dumped from the Python CLI by regen.sh (pinned host UUID, relative --cache-dir), and TestSidecarPythonParity replays the same 23-step operator sequence: cold status, three records, two batch-record loads, warm status and predict, a --no-persist record, a second codec bucket, and nine error paths. It requires identical stdout, identical state.json snapshots and identical exit statuses.

Known, deliberate differences:

• A capture row containing the non-standard JSON tokens NaN / Infinity is skipped by the Go binary and counted in rows_skipped. CPython's json.loads accepts the tokens: a NaN feature then silently skips the update while still counting the row as recorded, and a NaN observed_vmaf poisons the ridge weights (they persist as null and the next load cold-starts).
• Hand-edited state files outside the shape either writer produces are not guaranteed to load identically (for example a numeric string where CPython's float() would coerce and Go's decoder will not).
• --model is resolved differently; see ONNX predictor models.

ONNX predictor models¶

The two binaries interpret --model (on predict, sidecar and auto) differently, and the Python behaviour itself depends on the host.

• Python takes a filesystem path to predictor_<codec>.onnx. With onnxruntime importable, a missing file raises and the CLI exits 2; without it, the flag is silently ignored and the analytical curve is used (exit 0).
• Go resolves the value through the model registry (pkg/ai): an absolute path that exists, else <model-dir>/<name>.onnx, else <model-dir>/<name>, where the model dir is $VMAFX_MODEL_DIR or /usr/local/share/vmafx/model. An unresolvable name exits 2 on sidecar. Inference then goes through the vmafx-ort-runner subprocess (vmafx-ort-runner.md), which this repository builds from cmd/vmafx-ort-runner (ADR-1134; the dev container and the Go CI job both build it).

When the runner is absent from PATH, or present but linked against a libvmaf built without ONNX Runtime (exit 3), the predictor logs one warning carrying the runner's error and falls back to the analytical curve. The Python takes the same fallback without onnxruntime, but silently.

Omit --model to use the analytical fallback. It is the default and the only path the parity fixtures exercise, and the two binaries agree byte for byte on it.

Exit codes¶

Every subcommand exits 0 on success. A usage failure exits 2 on every subcommand, as argparse does in the Python CLI: an unknown or unparseable flag, a missing required flag (including --src of auto without --smoke, the input files of report, --target-vmaf of prefilter, and the source, size and preset flags of recommend without --from-corpus). Other failures exit 1 unless the table names another status.

TestEveryCommandRejectsUnknownFlagWithUsageStatus and TestMissingRequiredFlagExitsWithUsageStatus in cmd/vmafx-tune/cmd/required_flags_test.go hold every subcommand to the usage status.

Configuration and logging¶

vmafx-tune-go runs each subcommand inside the golusoris clikit (cobra + fx) framework. The framework injects a structured *slog.Logger and a config tree into every subcommand, so run diagnostics (sweep start and finish, ladder build summary, report rendering) are emitted as structured log lines on stderr, while subcommand output (JSON, Markdown, HTML) goes to stdout or the --output file. This keeps machine-readable output separate from logs when you pipe stdout.

Configuration is read from environment variables under the VMAFX_ prefix. golusoris maps each underscore in the variable name to a config-path delimiter (VMAFX_LOG_LEVEL → log.level).

# Quiet the per-run INFO diagnostics; keep warnings and errors.
VMAFX_LOG_LEVEL=warn vmafx-tune-go report results.json

# Force JSON logs for machine ingestion regardless of TTY.
VMAFX_LOG_FORMAT=json vmafx-tune-go compare --reference src.mp4 --targets 90

Python-only flags¶

Every vmaf-tune subcommand is ported. A few individual flags still need the Python implementation, because they depend on in-process ONNX inference or on importing a Python callable at runtime. Each is accepted by the Go parser and fails with a message naming the fallback rather than being silently ignored.

recommend-saliency --saliency-aware and predict --use-saliency are accepted by the Go binary. When the saliency session cannot be built, recommend-saliency proceeds without an ROI map (the report's saliency_aware field then reads false), and predict logs a warning and degrades saliency moments to 0.0, matching the Python behaviour. A real saliency-biased encode needs an ONNX forward pass: use vmaf-tune recommend-saliency --saliency-aware for it (see Saliency ONNX inference).

Corpus fields not carried by recommend¶

recommend's encode-driven path writes the same schema-v3 corpus JSONL the corpus subcommand does, and every key is present. Five corpus features are not carried by this group's port, and their fields hold the same zero or empty values the Python emits when the feature is unavailable, so a reader filters on them exactly as it already does:

• the content-addressed encode cache (ADR-0298),
• HDR detection (ADR-0295),
• TransNet-V2 shot metadata (shot_count stays 0),
• sample-clip windowing (clip_mode stays full),
• the encoder-internal pass-1 stats (the ten enc_internal_* columns stay 0.0, which is what the Python aggregator returns for an empty frame list).

These belong to the corpus port.

Python vs Go differences¶

The flag surface of recommend-saliency, tune-per-shot, fast, corpus, benchmark and encode-profile matches Python flag for flag. The differences below were checked against the Python argparse tree (tools/vmaf-tune/src/vmaftune/cli.py) and the Go flag registrations in cmd/vmafx-tune/cmd/.

Only the three oldest ports (compare, ladder, report) renamed their inputs; fast, auto and tune-per-shot use --src in both binaries.

Known gaps¶

Two behaviours are not at full parity with Python. Both are documented here rather than hidden behind a silent degradation.

Saliency ONNX inference¶

recommend-saliency --saliency-aware falls back to a plain encode. Everything around the model is ported and tested against the Python; only the single forward pass is missing:

• Go has no in-process ONNX Runtime in this module. The fork's bridge (pkg/ai, ADR-0713) shells out to vmafx-ort-runner and passes the input tensor as a JSON array in argv. That works for the per-shot predictor's 14 floats, but it cannot carry saliency's 3×H×W input, which is 6.2 million floats (about 75 MB of JSON) for a 1080p frame.
• The runner itself is built in this repository (cmd/vmafx-ort-runner, ADR-1134; see vmafx-ort-runner.md) and serves the predictor path. It has no transport for tensors that do not fit in argv.

Either of two changes unblocks it: a cgo ONNX Runtime binding (an ADR-level decision, since the binary currently builds without cgo), or a runner protocol that streams tensors over stdin (a protocol extension of the in-tree runner and pkg/ai, not a new dependency).

The per-shot predictor ONNX (predict --model) does route through pkg/ai, because its 14-float input fits argv comfortably. It degrades to the analytical curve when the runner is absent from PATH or linked against a libvmaf built without ONNX Runtime (exit 3), and the log says which.

TPE sampler trajectory¶

prefilter implements TPE natively (Bergstra et al. 2011 §4, the construction Optuna follows) rather than depending on a Go Optuna port that would pull gorm plus the MySQL, Postgres and cgo-SQLite drivers into a one-shot CLI. The search space, the objective, the emitted JSON and per-seed reproducibility are identical to the Python. The trial-by-trial trajectory for a given seed is not, and cannot be, because the two use different RNG streams.

fast uses goptuna instead and has the weaker reproducibility described under fast.

Architecture¶

The CLI root and every subcommand are built with the golusoris clikit (cobra + fx) framework (ADR-1119): clikit.New builds the root, clikit.Command builds each subcommand, and a thin withGolusoris adapter boots a one-shot fx graph per invocation so the command receives an injected *slog.Logger and config, runs to completion, and propagates its error as the process exit code.

The Go binary uses an adapter pattern. Each encoder shells out to ffmpeg (no libavcodec cgo dependency), and each scorer shells out to the vmaf CLI. Each package has one implementation (ADR-1137).

Core packages¶

Shared layers¶

The auto and sidecar stack¶

ADR-1125 records which consumer owns which copy of the shared layers. For the migration rationale see ADR-0705, ADR-0730 (Stage 2), ADR-0770 (Stage 4) and ADR-0702 (the Phase 4 umbrella).

Tests¶

cmd/vmafx-tune/cmd/*_test.go covers each subcommand; pkg/*/..._test.go covers the packages. Python-parity fixtures live under cmd/vmafx-tune/cmd/testdata/ and in pkg/codecadapter/testdata/. Run the Go tests with go test ./cmd/vmafx-tune/... ./pkg/....

History¶

Port stages¶

The Go port landed in stages. Every row below is merged unless noted.

tune-per-shot --fast-nr and a production-mode fast remain open and wait on an ONNX Go binding.

Roadmap table corrections¶

An earlier revision of the roadmap listed pkg/conformal as merged under Stage 3. It was not: no such package existed on master. The package landed with Stage 6, and the roadmap row was corrected. The pkg/tune/{predictor,codec, pyjson} aliases were first kept for the in-flight sidecar parity fix (#1187); that fix has landed, and the aliases are still imported by the sidecar.

See also¶

• vmaf-tune.md: overview of the tuning workflow and the Python CLI.
• vmaf-tune-corpus.md: corpus JSONL schema and flags.
• vmaf-tune-recommend.md and vmaf-tune-coarse-to-fine.md: the CRF search.
• vmaf-tune-fast-path.md: the proxy fast path.
• vmaf-tune-codec-adapters.md, vmaf-tune-hdr-and-sampling.md, vmaf-tune-score-backend.md: adapters, HDR and scoring backends.
• vmafx-ort-runner.md and vmaf-perShot.md: the two helper binaries.