Research-2158: VMAFx API redesign, generated surfaces and the VMAFx FFmpeg filters¶

• Status: Active
• Workstream: ADR-1852, ADR-1829, ADR-1713 (draft PR #2086; RC4, #1723)
• Last updated: 2026-10-05

Design review for RC4: an inventory of every API surface the repository has today, the model of the new C API, the comparison of definition-driven generators, the VMAFx FFmpeg filters, the work plan, and the maintainer's answers to the open decisions (section 7). Read at origin/master 8eabe7c56 (function counts updated to 9bc68a108); issues #1723, #2067, #2138, #2142 and #2155; ADR-0686, ADR-0928, ADR-1199, ADR-1685, ADR-1829, ADR-1755 (PR #2108), ADR-1688, ADR-1679; HISS-14 in AGENTS.md.

0. Summary¶

1. One API definition generates every surface. Recommended source of truth: an in-house interface definition in TOML (core/api/vmafx.toml), read by small generators written in standard-library Python (scripts/codegen/vmafx_api/). It is the only candidate that describes a C ABI with opaque handles, out-parameters, fences and callbacks and the non-binding surfaces (CLI table, FFmpeg option table, MCP schemas, proto messages, docs) without fighting its own type model, needs nothing at build time beyond the Python Meson already runs, and carries no licence question. Generated files are committed; a Meson test regenerates them and fails on any difference, so a hand edit fails.
2. New C API under vmafx/*.h, prefix vmafx_, library libvmafx: contexts, devices, refcounted frames with acquire and release fences (the RC4 import API is this frame object), synchronous and asynchronous window scores with vmaf_score_pooled semantics, a provenance record behind every score, and errors that name what failed. Size-prefixed structs, opaque handles, explicit enum values, stable status codes.
3. Compatibility layer: every one of the 107 exported libvmaf functions is listed in section 2.11 with its vmafx_ target and whether its shim is generated or written by hand. Recommended packaging: a thin libvmaf.so.3 that links only the public symbols of libvmafx.so.1, which mechanically proves the new API is complete. Upstream FFmpeg's own libvmaf and libvmaf_cuda filters, the CLI, the Python harness and the golden-data gate keep working through it.
4. One vmafx scoring filter on the new API (plus vmafx_tune and vmafx_pre, D6): software frames and hardware frames for CUDA, SYCL, HIP and Metal, n_stats windows by time or frame count, provenance in the log, a retry-then-fail-named import rule and no silent host copy. Every capability of today's 20 patches survives under a VMAFx name and no filter keeps a vmaf name (section 5.9, D6); section 5.7 maps every option of every retired filter.
5. Prototype: the generator on a slice (context create and destroy, version and provenance query, one feature-score call), producing the C header, the compat shims for vmaf_init, vmaf_close, vmaf_version and vmaf_feature_score_at_index, a Python binding, the ABI layout test and a drift check, wired into Meson.

1. Inventory of today's API surfaces¶

1.1 Table¶

1.2 Observations that shape the design¶

• Six hand-maintained descriptions of one scoring request (CLI table, CLI usage, Go MCP schema, Python MCP schema, proto, OpenAPI) plus four FFmpeg option tables. The parity tests exist because the descriptions drift; generating them from one definition removes the drift instead of detecting it.
• No binding reaches the C API from Python, and Go reaches it three different ways. A generated binding per language replaces bindgen at build time (Rust) and the hand cgo (Go).
• Provenance is a CLI feature, not a library feature. It must move into the library so every consumer (filter, bindings, server) gets the same record.
• The import paths already learned the hard way that failures must name what failed (ADR-1688: -ENOTSUP naming each refused extractor; ADR-1679: a frame that cannot be imported fails the filter instead of passing through unscored) and that a hand-over without an ordering primitive races (ADR-1199). The new frame object makes both rules part of the type.

2. API model¶

2.1 Principles¶

1. Every object with a lifetime is an opaque handle with a create and a release function. No caller-visible struct holds library-owned memory.
2. Every struct that crosses the ABI starts with uint32_t struct_size and only grows at the end (HISS-14). No struct is passed by value except VmafxFence (fixed, sized). No bool, long, bit-field or C enum as a field type: enums are uint32_t fields with named constants of explicit value.
3. Every fallible function returns VmafxStatus and takes an optional VmafxError **error last.
4. One function per concept across backends: one device constructor, one frame import, one fence type, one pool. Backend specifics live in tagged descriptors, never in new function families.
5. No silent fallback (user rule): a requested backend, device or import path that is unavailable fails with a named error, and every result records which path ran.
6. Nothing in the header needs a vendor SDK: device handles are uintptr_t / int32_t file descriptors; typed helpers sit in optional vmafx/device_<backend>.h headers.

2.2 Objects¶

Scoring flow:

VmafxContextConfig cfg = VMAFX_CONTEXT_CONFIG_INIT;      /* sets struct_size */
VmafxContext *ctx; VmafxError *err = NULL;
if (vmafx_context_create(&cfg, &ctx, &err)) { log(vmafx_error_message(err)); vmafx_error_free(err); }
vmafx_model_load(&mcfg, "vmaf_v1.0.16_3d0h", &model, &err);
vmafx_context_use_model(ctx, model, &err);   /* ctx takes a reference */
vmafx_model_unref(model);                    /* caller may drop its own at once */
for (i = 0; i < n; i++) {
    vmafx_frame_import(dev, &ref_desc, &ref, &err);   /* or vmafx_frame_create_host */
    vmafx_frame_import(dev, &dist_desc, &dist, &err);
    vmafx_submit(ctx, ref, dist, i, &err);            /* consumes both references */
}
vmafx_flush(ctx, &err);
VmafxScore s = VMAFX_SCORE_INIT;
vmafx_score_pooled(ctx, &(VmafxScoreTarget){.model = model}, VMAFX_POOL_MEAN, 0, n - 1, &s, &err);
/* s.provenance -> version, build, model hash, per-feature backend */
vmafx_context_destroy(ctx, &err);

2.3 Lifetimes and ownership¶

• Context: vmafx_context_destroy() keeps today's retry contract (ADR-1336): a nonzero status leaves the context valid and every dependency (models, devices) retained so the caller can retry; success releases everything the context holds.
• Models: refcounted. vmafx_context_use_model() takes a reference that lives until the context is destroyed (ADR-1755). Models are immutable after load, so they are shared across contexts and threads without locks.
• Devices: refcounted. A context attached to a device holds a reference. External handles (a caller's CUDA context, SYCL queue, Metal command queue) stay owned by the caller and must outlive every device created from them; the library creates its own streams / queues on them.
• Frames: refcounted. vmafx_submit() consumes the caller's two references. The library keeps a frame while any extractor still reads it: temporal extractors read frames n-1 and n-2 (ADR-1478), so a frame can be retained across up to two further submissions. vmafx_context_frame_retention(ctx) returns that depth for the registered extractors so a producer can size its pool (an FFmpeg filter sets extra_hw_frames from it). An imported frame never frees producer memory.
• Fences: an acquire fence is borrowed for the duration of vmafx_frame_import() (the library enqueues a device-side wait on it, or duplicates the file descriptor). A release fence returned by vmafx_frame_release_fence() is owned by the caller (event handle or file descriptor to destroy / close).
• Windows: a window handle owns its result until vmafx_window_release(). The completion callback receives a borrowed pointer valid during the call.
• Errors: owned by the caller, one allocation per failure, never on the success path.
• Strings: every const char * the library returns lives as long as the object it came from (version strings: process lifetime; provenance strings: context lifetime).

2.4 Threading¶

2.5 Error model¶

typedef int32_t VmafxStatus;     /* 0 ok; >0 informational; <0 error */
#define VMAFX_OK            0
#define VMAFX_PENDING       1    /* window / frame not final yet (today's -EAGAIN) */
#define VMAFX_E_INVALID    -1    /* argument or state; error names the parameter */
#define VMAFX_E_NOMEM      -2
#define VMAFX_E_NOTSUP     -3    /* backend / format / option unsupported; error names it */
#define VMAFX_E_NOTFOUND   -4    /* model version, feature, device index */
#define VMAFX_E_BUSY       -5    /* already imported / attached */
#define VMAFX_E_DEVICE     -6    /* backend runtime failure; error carries the runtime's code */
#define VMAFX_E_IO         -7
#define VMAFX_E_RANGE      -8
#define VMAFX_E_NONFINITE  -9
#define VMAFX_E_TIMEOUT   -10    /* fence wait */
#define VMAFX_E_ABI       -11    /* struct_size smaller than the minimum of this ABI */

• Stable numbers on every platform; errno values differ between C runtimes (Windows' errno.h numbers several POSIX codes differently), so the compat layer maps status to errno once (vmafx_status_to_errno(), generated) and bindings map status to their idiom once (generated from the same table: Rust enum Error, Go sentinel errors wrapping os.ErrInvalid / os.ErrNotExist as pkg/libvmaf does today, Python exception classes).
• VmafxError carries: status, message (formatted at the failure site), subject_kind (feature, option, extractor, backend, device, model, frame, plane, fence, path), subject (the name), function, and cause (another VmafxError, e.g. the backend runtime's code). Accessors: vmafx_error_status/message/subject/subject_kind/cause, vmafx_error_free.
• Passing NULL for error is allowed; the status alone is returned and the message goes to the context's log callback at ERROR, so no failure is silent.
• Out-of-band "last error" (thread-local) was rejected: it breaks with asynchronous completions and across language runtimes that move work between threads.

2.6 Versioning and ABI policy¶

• vmafx/version.h (generated): VMAFX_ABI_VERSION_MAJOR/MINOR/PATCH; vmafx_abi_version() returns the runtime value; VMAFX_ABI_CHECK() in the header lets a consumer refuse a runtime with a smaller minor.
• The library's ABI number line is independent of the product version (ADR-1151 / ADR-1235 already keep the libvmaf SONAME independent). Proposed: libvmafx.so.1, ABI 1.0.0 frozen at the v1.0.0 tag; until the tag the ABI may change because no release ships it (decision D2).
• Append-only within a major (HISS-14): new functions; new fields at the end of size-prefixed structs; new enum constants with new values; new status codes. Forbidden: removing or renaming a symbol, changing a parameter, reordering or retyping a field, renumbering a constant. A break needs !, a Migration: footer and a major bump.
• Size negotiation: the caller sets struct_size = sizeof(T) (the *_INIT macros do). For inputs the library reads the fields it knows up to struct_size and treats missing tail fields as their documented defaults; a struct_size below the ABI 1.0 minimum is VMAFX_E_ABI. For outputs the library writes min(struct_size, sizeof) bytes and stores the written size back.
• Symbol versions: a generated linker version script gives every symbol the node of the ABI minor that introduced it (VMAFX_1.0, VMAFX_1.1, ...) on ELF targets, and a generated .def export list on Windows. Both come from since in the API definition, so a new symbol cannot ship unversioned.
• Deprecation: deprecated = { since, replacement, removal } in the definition generates the VMAFX_DEPRECATED("use X") attribute, the docs note and the changelog fragment.

2.7 Device frames and fences¶

typedef struct VmafxFence {
    uint32_t struct_size;
    uint32_t kind;        /* VmafxFenceKind */
    uintptr_t handle;     /* event / shared-event object, 0 when unused */
    uint64_t value;       /* timeline value for shared events, 0 otherwise */
    int32_t fd;           /* sync_file descriptor, -1 when unused */
    uint32_t reserved;    /* 0 */
} VmafxFence;

typedef struct VmafxFrameImport {
    uint32_t struct_size;
    uint32_t memory;      /* VmafxMemoryKind: DEVICE_POINTER, DEVICE_ARRAY, DMABUF, METAL_SURFACE, METAL_TEXTURE, WIN32_SHARED */
    uint32_t pix_fmt;     /* YUV420P/422P/444P/400P, NV12, P010, P016 */
    uint32_t bpc, w, h, n_planes;
    struct { uintptr_t handle; int32_t fd; uint32_t plane_index; uint64_t offset; uint64_t pitch; uint64_t modifier; } plane[3];
    VmafxFence acquire;   /* producer's write -> our read */
    uint32_t flags;       /* VMAFX_IMPORT_REQUIRE_ZERO_COPY (default on) */
} VmafxFrameImport;

Rules:

• vmafx_frame_import() never copies pixel data to the host. With VMAFX_IMPORT_REQUIRE_ZERO_COPY (default) a layout the backend cannot bind is refused with VMAFX_E_NOTSUP naming the memory kind, pixel format, modifier and plane. A caller that wants a copy says so (vmafx_frame_create_host + upload, or the flag cleared, which is logged once per context).
• NV12 / P010 / P016 are converted to planar on the device: a de-interleave and, for P010, the shift of ADR-1679; no arithmetic beyond that, so an imported frame scores bit-identically to the same frame uploaded from the host (ADR-1829 exit evidence).
• Admission: before a frame is counted the context checks every registered extractor against the frame's residency and refuses with each refusing extractor named (ADR-1688 generalised to every backend).
• sync_file export / import uses the dma-buf ioctls of the Linux uAPI (DMA_BUF_IOCTL_EXPORT_SYNC_FILE / IMPORT_SYNC_FILE, present in include/uapi/linux/dma-buf.h); the minimum kernel version is to be pinned in WP3 (not verified here).
• Fence tests are failing-first: a test build that skips the acquire wait must produce a wrong score under concurrent device load (the ADR-1199 reproduction harness, scripts/test/repro-cuda-ffmpeg-nondeterminism.sh, is the model).

2.8 Window scores (synchronous and asynchronous)¶

typedef struct VmafxWindowRequest {
    uint32_t struct_size;
    uint32_t target_kind;            /* MODEL, MODEL_SET, FEATURE */
    const VmafxModel *model; const VmafxModelSet *model_set; const char *feature;
    uint64_t first, last;            /* inclusive frame indices, as vmaf_score_pooled */
    uint32_t pool_mask;              /* bits of VmafxPool: MIN MAX MEAN HARMONIC_MEAN MEDIAN PERC5 PERC10 PERC20 */
    void (*on_complete)(VmafxWindow *, const VmafxWindowResult *, void *user);  /* optional */
    void *user;
} VmafxWindowRequest;

• vmafx_window_submit() returns at once. A window completes when every frame in [first, last] has its final scores: for motion2 / motion3 that is after frame last + 1 arrived or the context was flushed, and device results are collected one frame late (ADR-1685). Completion either calls on_complete on a library thread or is observed with vmafx_window_poll() (returns VMAFX_PENDING until done) / vmafx_window_wait(timeout).
• The result holds one value per requested pool method plus n_frames, n_scored (subsampling) and the provenance pointer. It is computed by the same pooling code as the synchronous vmafx_score_pooled(), so both are bit-identical; a test asserts it.
• vmafx_score_pooled() is the synchronous form and returns VMAFX_PENDING where vmaf_score_pooled() returns -EAGAIN today.

2.9 Provenance (#2142)¶

VmafxProvenance (context level, vmafx_context_provenance()):

Every VmafxScore and VmafxWindowResult points at the record. vmafx_report_write() embeds it in JSON ("provenance" object) and XML (<provenance>); CSV and SUB keep their columns and get a <path>.provenance.json sidecar when asked (decision in WP5). The CLI's amend_cli_backend_receipt() splice disappears because the library writes the same fields; backend_used / feature_backends stay as aliases (HISS-14). vmaf --verify-provenance <report> re-runs and compares bit for bit (#2142).

Prototype limitation: a feature is mapped to its extractor through the extractors' provided_features; option-decorated names resolve once the feature collector records the producer of each feature vector (WP5).

2.10 Header layout (all generated)¶

2.11 Compatibility layer¶

Structure:

• libvmaf/*.h stay byte-for-byte compatible: same declarations, layouts and constant values (a generated layout test pins every public struct of libvmaf.h too).
• Three shim kinds. shim: generated one-to-one call with argument conversion. glue: generated from a declared field map in the API definition (struct conversions such as VmafConfiguration to VmafxContextConfig). manual: hand-written in core/src/compat/ because the old function's semantics are stateful (picture ownership transfer, the SYCL double-buffer path); the API definition still declares it so the symbol list, docs and conformance cases are generated.
• The compat code calls only exported vmafx_ functions. With the separate-library packaging (D3) this is a link-time fact: if a libvmaf function cannot be written on the public new API, the new API is incomplete.
• VmafContext * given to compat users is the engine handle bound to a VmafxContext (the prototype keeps an owner pointer in the engine context); vmafx_context_from_libvmaf() lets a migrating caller mix both APIs on one session.
• Deprecation: VMAF_DEPRECATED on every libvmaf.h declaration behind VMAF_ENABLE_DEPRECATION_WARNINGS in 1.0 (upstream FFmpeg must build without new warnings), on by default in 1.1, removal in 2.0 (decision D7).
• Conformance: a generated case per compat function runs the old and the new call on the same input and compares results bit for bit (%a), plus the NULL / invalid-argument cases; the golden-data gate runs through compat automatically because the CLI and the Python harness use libvmaf.h.
• Upstream FFmpeg (n9.0.2, no patches) needs pkg-config libvmaf >= 2.0.0, libvmaf/libvmaf.h, libvmaf/libvmaf_cuda.h and 18 symbols (vmaf_init, vmaf_close, vmaf_use_feature, vmaf_use_features_from_model, vmaf_feature_dictionary_set, vmaf_read_pictures, vmaf_picture_alloc, vmaf_picture_unref, vmaf_score_pooled, vmaf_write_output, vmaf_model_load, vmaf_model_load_from_path, vmaf_model_feature_overload, vmaf_model_destroy, vmaf_cuda_state_init, vmaf_cuda_import_state, vmaf_cuda_preallocate_pictures, vmaf_cuda_fetch_preallocated_picture; counted in libavfilter/vf_libvmaf.c of n9.0.1). A CI job builds that FFmpeg against the installed VMAFx and scores three frames (RC4 exit evidence).

Every exported libvmaf function (107 at 9bc68a108: the 104 of 8eabe7c56 plus the three picture-conversion functions of #2140; 49 shim, 31 glue, 27 manual):

3. Generation: one definition, every surface¶

3.1 Requirements¶

3.2 Candidates (versions and licences checked 2026-10-05)¶

Sources are listed in section 9.

3.3 Recommendation: (a) in-house TOML definition + standard-library Python generators¶

• It is the only option that satisfies R1-R3 together. (b) would move the public API's implementation into Rust, which RC4 does not do (the engine stays C, Rust twins are internal and optional). (c) does not target native libraries at all. (d) fights its type model for every C construct and still leaves every non-binding emitter to us; it is the right tool for the gRPC surface, which is why the generator emits .proto messages that buf then compiles and checks. (e1) keeps C developers in C but cannot hold option groups, MCP schemas or FFmpeg tables, and annotation macros in public headers are their own maintenance surface.
• Nothing new at build time: generated files are committed, and regeneration needs only python3 >= 3.11. Rust users stop needing libclang and bindgen at build time; Go users stop hand-writing cgo.
• Risks and their answers: (1) emitter code is ours: keep each emitter small and tested, emit only what the repository uses; (2) an in-house IDL can grow into a language: the definition holds data only (no expressions) except compat glue statements, which are C fragments compiled and tested by the conformance cases; (3) buf breaking still guards the generated proto.
• TOML over JSON (no comments) and YAML (needs a third-party parser) because the standard library reads it (tomllib, Python 3.11+, already used in scripts/ci/check-workflow-versions.py).

3.4 Shape of the definition¶

[api]
name = "vmafx"
abi_version = "1.0.0"
header_dir = "vmafx"

[[status]]
name = "VMAFX_E_NOTSUP"
value = -3
errno = "ENOTSUP"
doc = "Backend, format or option not supported; the error names it."

[[handles]]
name = "VmafxContext"
release = "vmafx_context_destroy"
doc = "One scoring session."

[[structs]]
name = "VmafxProvenance"
sized = true                # first field is struct_size; append-only
since = "1.0"
fields = [
  { name = "version", type = "cstr", doc = "git describe of the build" },
  { name = "abi_major", type = "u32" },
]

[[functions]]
name = "vmafx_context_create"
since = "1.0"
returns = "status"
params = [
  { name = "config", type = "VmafxContextConfig", pass = "in_ptr", nullable = true },
  { name = "out", type = "VmafxContext", pass = "out_handle" },
  { name = "error", type = "VmafxError", pass = "out_error" },
]

[[compat]]
name = "vmaf_feature_score_at_index"
header = "libvmaf/libvmaf.h"
kind = "shim"
target = "vmafx_feature_score"

Option groups (one per concept: model, feature, backend, device, threads, subsample, pool, precision, output, window, tiny model, masks) carry type, default, range / enum, doc, and per-surface spellings: cli = ["--threads"], ffmpeg = "threads" (+ aliases = ["n_threads"]), mcp = "threads", proto = { field = 7 }. The generator refuses a group without a spelling for a surface it is declared on.

3.5 Generated artefacts¶

3.6 How a breaking change is caught¶

1. Append-only checker (python3 -m vmafx_api --abi-check --against <ref>): loads the definition at the merge base and at the last release tag and refuses removed or renamed symbols, changed parameters, reordered / retyped / removed fields, fields added to an unsized struct, renumbered constants, a lowered since. Exception: the PR title carries ! and the body a Migration: footer, and abi_version major went up.
2. Drift check (Meson test, every build): regenerating into a temporary directory must reproduce the committed files byte for byte. A hand edit of any generated file fails.
3. Layout tests: the generator computes sizeof, alignment and every offset for the LP64 and LLP64 data models from the definition; the C test asserts them with _Static_assert and each binding asserts its own view at import / compile time. The compiler, the definition and the binding must agree.
4. Symbol checks: check_exported_symbols.py reads the generated symbol list; the version script makes a symbol without a since node a link error.
5. Optional abidiff (libabigail, Apache-2.0 WITH LLVM-exception) between the last release's .so and the candidate in the release workflow, as an independent second opinion.
6. buf breaking on the generated proto.

4. What gets automated beyond bindings¶

5. FFmpeg vmafx filter¶

5.1 Shape¶

• Two video inputs like upstream libvmaf: main (distorted) and reference, synchronised with framesync (its eof_action, shortest, repeatlast, ts_sync_mode options come along). Output: the main frames, unchanged.
• Built on vmafx/*.h only; configure --enable-libvmafx, require_pkg_config libvmafx "libvmafx >= 1.0.0" vmafx/vmafx.h vmafx_context_create.
• One filter for every backend; the backend follows the input frames (backend=auto).

5.2 Options (generated from the option groups)¶

5.3 Hardware frames per backend¶

Both inputs must resolve to the same backend and device; otherwise the filter fails at config time naming both inputs' formats and devices (a host bridge is the user's explicit choice: hwdownload,format=... or import=host).

5.4 Import rule: retry, then fail named¶

Interpretation of the maintainer's "retry-then-fail-named" rule, built on ADR-1688 and ADR-1679 (to be confirmed, D8):

1. An import or admission failure with a transient status (VMAFX_E_BUSY, VMAFX_E_TIMEOUT on the acquire fence, pool exhaustion) is retried once after a host wait on the acquire fence.
2. A second failure, or any non-transient one (VMAFX_E_NOTSUP, VMAFX_E_INVALID), fails the filter graph with one error line naming: backend, device, input (main / reference), pixel format and software format, modifier, plane, and each refusing extractor with the option that makes it refuse.
3. No frame passes through unscored, no score line is printed after a failure (ADR-1688's VMAF score: 0.000000 fix), and nothing silently falls back to a host copy.

5.5 n_stats windows (#2138)¶

• Window k holds the frames whose presentation time lies in [t0 + k*n, t0 + (k+1)*n) (t0 = first frame), or frames [k*m, (k+1)*m) with n_stats_frames=m.
• When the first frame past a window arrives, the filter submits a VmafxWindowRequest for the window's indices and the requested pool methods. It completes after the frames' scores are final (one frame later for motion2 / motion3, one frame later again on device backends), so window output trails by a bounded number of frames, never by a window.
• At end of stream the last window is flushed and marked "partial": true.
• With subsample, statistics cover the scored frames; the record carries n_frames and n_scored.
• Output per window, one line (log), one NDJSON object (file) and frame metadata lavfi.vmafx.window.<metric>.<pool> on the next output frame (metadata):

{"window": 3, "start": 6.000, "end": 8.000, "n_frames": 48, "n_scored": 48, "partial": false,
 "vmaf": {"min": 91.2, "max": 99.8, "mean": 96.4, "harmonic_mean": 96.3}}

(Values illustrative.) The final report also lists the windows.

5.6 Output¶

• log_path / log_fmt: vmafx_report_write() with provenance (JSON / XML), the windows and the per-frame scores.
• Log: provenance at init (version, build_id, model hash, per-feature backend and exactness class), one line per window, the pooled scores at uninit.
• Metadata: per-frame (metadata=1) and per-window keys.

5.7 Migration from each retired filter option¶

5.8 Patch series after the switch¶

docs/usage/ffmpeg.md gains the migration guide (sections 5.7 and 5.9 as prose plus worked commands, including the #2138 encode-and-score command).

5.9 Every filter capability under a VMAFx name (D6)¶

Maintainer decision D6: every capability of today's 20 patches survives, under a VMAFx name; no filter keeps a vmaf name; the old names go in the same step.

Configure switches: --enable-libvmafx builds vmafx, vmafx_tune and vmafx_pre against libvmafx (require_pkg_config libvmafx "libvmafx >= 1.0.0" vmafx/vmafx.h vmafx_context_create); the fork's patches no longer add --enable-libvmaf-sycl, --enable-libvmaf-metal, --enable-libvmaf-vulkan or --enable-libvmaf-hip.

6. Plan for RC4¶

6.1 Work packages¶

Critical path: WP1 -> WP2 -> WP3 -> WP9 -> WP10. WP4, WP5, WP7, WP8 run beside WP3.

6.2 Test strategy¶

• Failing first, per WP: each new function lands with a test that fails without it; each gate with a planted defect it refuses (drift check: a hand-edited header; append-only checker: a reordered field, a renumbered constant, a removed function; layout test: a field type change in a copy).
• Golden-data gate through compat: unchanged test sources; after WP6 the CLI and the Python harness reach the engine through compat. Run as today, plus once with the compat library forced (LD_LIBRARY_PATH to the split build) in the release workflow.
• Bit-exact parity through the new API: the cross-backend parity gate (scripts/ci/cross_backend_parity_gate.py) gains an import mode: each twin declared exact scores an imported device frame == the host-uploaded frame and == the CPU extractor; tolerance cells keep their bounds.
• Fences: tests that fail when the acquire wait or the release signal is skipped (contention harness of ADR-1199).
• Windows: synthetic sessions with imported scores (deterministic) check boundaries, the partial last window, subsampling and sync == async.
• Compat conformance: generated old-vs-new cases, %a equality.
• Bindings: each binding's tests call the library built by Meson (Python through meson test, Rust through cargo test with the build's library path, Go through go test with CGO_LDFLAGS as pkg/libvmaf/doc.go describes).
• Upstream FFmpeg: builds against the installed tree and scores three frames.

7. Decisions (maintainer, 2026-10-05)¶

All eight were answered by popup on 2026-10-05; ADR-1852 records them and is Accepted.

8. Risks and open points¶

• Scope: RC4 already holds the Rust metric and the import API. The API, generator and filter add an estimated 12 work packages; WP3 needs macOS, SYCL and HIP devices for its exit evidence (ADR-1829 consequence).
• Engine coupling: the engine is core/src/libvmaf.c (4,923 lines) behind struct VmafContext. WP2 moves the bodies of the old functions behind vmaf_engine_* internal entry points (the prototype does this for four) rather than rewriting the engine.
• Frame retention makes pool sizing part of the contract; producers with small pools (three frames) and an extractor reading n-2 are refused today (ADR-1478); the new API reports the depth instead of failing late.
• Platform fences: Windows shared-texture fences and the dma-buf sync_file minimum kernel version need verification in WP3.
• Windows packaging: ADR-0121 builds Windows statically today; the split library (D3) applies to shared builds; static builds ship libvmafx.a and libvmaf.a.

9. Sources (checked 2026-10-05)¶

• cbindgen: https://github.com/mozilla/cbindgen (C/C++11 headers from Rust; MPL-2.0); version from https://crates.io/api/v1/crates/cbindgen (0.29.4, 2026-06-09).
• UniFFI: https://github.com/mozilla/uniffi-rs (Kotlin, Swift, Python, Ruby built in; third-party Go, C#, ...; MPL-2.0; "a long way from a 1.0 release"), https://mozilla.github.io/uniffi-rs/latest/internals/design_principles.html; crates.io 0.32.2 (2026-09-23).
• Diplomat: https://github.com/rust-diplomat/diplomat, https://rust-diplomat.github.io/diplomat/, https://rust-diplomat.github.io/diplomat/types.html ("Callbacks in parameters. Support is limited."), design doc https://github.com/rust-diplomat/diplomat/blob/main/docs/design_doc.md (Rust implementations only, opaque Box types); crates.io diplomat-tool 0.16.1 (2026-08-20, MIT OR Apache-2.0, rust-version 1.88).
• wit-bindgen: https://github.com/bytecodealliance/wit-bindgen (WebAssembly components only; "This CLI IS NOT stable"); crates.io 0.62.0 (2026-09-10).
• protobuf plugins: https://protobuf.dev/reference/other/ (CodeGeneratorRequest / CodeGeneratorResponse, any language); custom options https://protobuf.dev/programming-guides/proto3/#customoptions; release v36.2 (2026-09-17) from the project's GitHub releases.
• buf breaking: https://buf.build/docs/breaking/ (FILE / PACKAGE / WIRE_JSON / WIRE; --against '.git#branch=main'); buf v1.73.0 (2026-09-11), Apache-2.0.
• bindgen: crates.io 0.73.2 (2026-09-08, BSD-3-Clause).
• SWIG licence: https://www.swig.org/legal.html.
• libabigail: https://sourceware.org/libabigail/ (Apache-2.0 with LLVM exception).
• MCP tool schemas: https://modelcontextprotocol.io/specification/2025-11-25/server/tools (inputSchema / outputSchema default to JSON Schema 2020-12).
• FFmpeg facts from the local n9.0.1 checkout: libavutil/hwcontext.h (device types; no HIP type), libavutil/hwcontext_cuda.h (AVCUDADeviceContext { cuda_ctx; stream; }), libavfilter/vf_libvmaf.c (symbols and options), configure (require_pkg_config libvmaf "libvmaf >= 2.0.0"), fftools/ffmpeg_opt.c (loopback decoders, -dec).
• dma-buf sync_file ioctls: /usr/include/linux/dma-buf.h on the build host (DMA_BUF_IOCTL_EXPORT_SYNC_FILE, DMA_BUF_IOCTL_IMPORT_SYNC_FILE).
• Repository facts: paths cited inline, read at origin/master 8eabe7c56.

10. Prototype¶

• Draft PR #2173, label rc4, branch rc4/api-generation-prototype, head 5a90351ab, rebased on 9bc68a108; ADR-1852 Accepted 2026-10-05 (docs PR docs/adr-vmafx-api-redesign); digest Research-2158 (this document, public form).
• Definition core/api/vmafx.toml; generator scripts/codegen/vmafx-api.py + scripts/codegen/vmafx_api/ (standard-library Python, 1,861 lines, 179 of them tests); outputs: core/include/vmafx/vmafx.h, core/include/vmafx/libvmaf_bridge.h, core/src/vmafx/status_gen.{c,h}, core/src/vmafx/compat_libvmaf_gen.c, core/test/test_vmafx_abi_layout.c, bindings/python/vmafx/_api.py, docs/api/vmafx/reference.md.
• Hand-written core: core/src/vmafx/context.c, error.c, engine.h; libvmaf.c renames the bodies of vmaf_init, vmaf_close, vmaf_version, vmaf_feature_score_at_index to vmaf_engine_* and adds VmafContext.api_owner.
• Evidence (CPU build, -j4): fast suite 345 OK / 0 fail; golden-data gate through the shims 280 passed, 3 skipped; test_vmafx_api_slice 11/11, test_vmafx_python_binding 7/7, generator tests 15/15; clang-tidy cpu lane (container, 22.1.8) 0 findings on the 7 touched translation units; praetorctl audit no finding in touched files; MSVC-ism preflight pass; every new gate shown refusing a planted defect.