`vmaf-tune` content-addressed cache¶

The vmaf-tune cache stores the encode and score result of every corpus cell under a content hash, so a repeated (source, encoder, preset, crf) cell is restored from disk instead of being encoded and scored again. It is a Python API feature: there is no command-line flag, and it is off by default. The design is in ADR-0298; the tool overview is in vmaf-tune.md.

Turn it on¶

The corpus runner enables the cache when both cache_enabled and cache_dir are set on CorpusOptions:

from pathlib import Path

from vmaftune.corpus import CorpusJob, CorpusOptions, iter_rows

opts = CorpusOptions(
    encoder="libx264",
    output=Path("corpus.jsonl"),
    cache_enabled=True,
    cache_dir=Path.home() / ".cache" / "vmaf-tune",
)
rows = list(iter_rows(job, opts))  # job: a CorpusJob for one source

Option	Default	Meaning
`cache_enabled`	`False`	Master switch for the cache.
`cache_dir`	`None`	Cache root. The cache stays off while this is `None`, even with `cache_enabled=True`.

vmaf-tune corpus has no --cache-dir, --no-cache or size flag. The only --cache-dir on the command line belongs to the sidecar subcommand and is a different cache.

default_cache_dir() in tools/vmaf-tune/src/vmaftune/cache.py returns $XDG_CACHE_HOME/vmaf-tune (or ~/.cache/vmaf-tune when the variable is unset). The corpus runner does not call it, so pass that path yourself as cache_dir if you want it.

Cache key¶

A cache entry is keyed on the SHA-256 of the canonical-JSON encoding of these fields plus the key version (CACHE_VERSION, 2):

Field	Source
`src_sha256`	Content hash of the reference file.
`encoder`	Adapter slug (`libx264`, `hevc_nvenc`, ...).
`preset`	Encoder preset string fed to the adapter.
`crf`	Quality-knob value (int).
`adapter_version`	The adapter's `adapter_version`, bumped when its argv shape, presets or range change.
`ffmpeg_version`	The version `ffmpeg -version` reports, read once per sweep.
`passes`	`2` for a 2-pass encode (`two_pass` on an adapter that supports it), else `1`.
`sample_clip_seconds`, `sample_clip_start_s`	The encoded window of sample-clip mode.
`settings`	Everything else the encode or the score depends on: width, height, source width and height, pixel format, frame rate, duration, the extra encoder argv (HDR signalling, rung scale), the VMAF model and the score backend.

cache_key() refuses an empty source hash, encoder, preset, adapter version or ffmpeg version, and tools/vmaf-tune/tests/test_cache.py checks that changing any field produces a new key. When ffmpeg -version reports no version, the corpus runner keeps the cache off for that run and logs a warning. Entries written under key version 1 hash differently, so they miss; their files stay until LRU eviction or a manual delete.

Cache layout on disk¶

<cache_dir>/
  meta/<key>.json     parsed result tuple (size, times, vmaf score, versions)
  blobs/<key>.bin     opaque encoded artifact (atomic put: tmp + rename)
  __index__.json      last-access timestamps for LRU eviction

Cache lifecycle¶

Hit: the encode and score subprocesses are skipped and the row the miss produced is replayed, column for column, with a fresh run_id and timestamp. An entry without a stored row is treated as a miss.
Miss: encode and score run as normal. A successful result (encoder exit status 0) is inserted with a fresh last_access timestamp.
Source hash off: a run with --no-source-hash has no src_sha256, so the cache is skipped for every cell of that run.
LRU eviction: each insert evicts least-recently-used entries until the total size of meta/ plus blobs/ is at or below the ceiling. The default ceiling is 10 GiB (DEFAULT_SIZE_BYTES). The corpus runner does not expose a size option; use TuneCache(path, size_bytes=...) directly to change it.
Index flush: the runner writes __index__.json once, after the sweep.

Caveats¶

The cache is not baked into the JSONL row. The row stays the canonical record and the cache is an opaque result store, so rows look identical whether a cell hit or missed (ADR-0298).
A hit does not write a synthetic encode_path; that field stays empty unless --keep-encodes is set.
TuneCache is not thread-safe. Concurrent processes (for example over NFS) can share a directory: reads work, and writes are last-writer-wins with both writers' bytes valid by content addressing.

Manual eviction¶

There is no cache prune subcommand. To start over, for example when the corpus methodology changes incompatibly, delete the directory:

rm -rf ~/.cache/vmaf-tune

vmaf-tune content-addressed cache¶