Scoring API contract¶

The scoring server (vmafx-server) answers over gRPC (vmafx.v1.VmafxScoring) and HTTP (POST /v1/score). This page is the contract an integrator builds on: what a request may carry, what every response reports, what the server promises about the scores, and how the API may change. The services are described on gRPC service and REST API; the server's settings on Server configuration.

The same request gives the same score everywhere¶

A request scores the same pair the same way through every VMAFx surface:

The server returns lossless scores (precision max, %.17g) unless the request asks for another precision, so the score it returns is the double the CLI and the C API compute, bit for bit. The contract test checks it on the Netflix 576x324 pair for the library default model, vmaf_v0.6.1 with four threads and vmaf_v0.6.1 on every third frame:

meson test -C build test_vmafx_score_contract   # suite: contract

It runs each request through the vmaf binary of the build, the C API (build/test/vmafx_score_contract), gRPC Score and POST /v1/score, and fails unless all four report the same score and the same library version and ABI. It skips, with the reason, when Go or the fixtures are missing; a skipped run is never reported as a pass.

Requests¶

{
  "reference": "/data/src01_hrc00_576x324.yuv",
  "distorted": "/data/src01_hrc01_576x324.yuv",
  "model": "vmaf_v0.6.1",
  "options": {"width": 576, "height": 324, "pixel_format": "420", "bitdepth": 8, "threads": 4}
}

• reference, distorted: paths the server can read; .y4m, or raw .yuv with width, height, pixel_format and bitdepth in options.
• model: a model name from the server's model directory. Omitted, the library default model (VMAF_DEFAULT_MODEL_VERSION, vmaf_v1.0.16_3d0h in this release).
• options: any field of the table below. An unset field keeps the default in its row. A value outside the field's range or list, a field the contract does not know, or a device without a backend that selects devices by index is refused (INVALID_ARGUMENT, HTTP 400), never ignored. Fields marked reserved accept only their default until the release that implements them.

Responses and provenance¶

The response to the request above (a development build; version names the build that ran):

{
  "score": 76.66783149135578,
  "features": {"vmaf": 76.66783149135578, "integer_adm2": 0.9345057762923995, "integer_motion2": 3.894360826611791},
  "provenance": {
    "library": {"abi_major": 0, "abi_minor": 1, "abi_patch": 5, "active_backend": "cpu",
                "n_extractors": 3, "version": "v1.0.0-rc.2-529-gb8b3af281",
                "build_id": "sha256:...", "models": [{"name": "vmaf", "version": "vmaf_v0.6.1",
                "sha256": "5950d61f...", "flags": "0", "overrides": ""}],
                "features": [...], "annotations": [...], "digest": "sha256:...", ...},
    "model": "vmaf_v0.6.1",
    "model_sha256": "5950d61fa1f861bd45d8149d80539ed9f3376cfc2495b8f0fa8e9f57cb131ee3",
    "backend_used": "cpu",
    "feature_backends": [{"extractor": "adm", "backend": "cpu"}, {"extractor": "motion", "backend": "cpu"},
                         {"extractor": "vif", "backend": "cpu"}],
    "precision": "max"
  }
}

Every scoring response carries provenance: the gRPC ScoreResponse, the POST /v1/score body and the final AggregateScore of ScoreStream.

A server whose vmaf binary does not report the provenance record (a binary older than this contract) fails the request instead of answering without it.

Compatibility¶

• Version. The gRPC package is vmafx.v1, the HTTP resources live under /v1. Within v1 the API only grows: new RPCs, new messages, new fields with new numbers, new values of a field's list. Nothing is renamed, removed, renumbered or retyped. buf breaking (proto) and the definition's append-only checker (python3 scripts/codegen/vmafx-api.py --abi-check) refuse such a change before it merges.
• Deprecation. A field to be removed is first marked deprecated in the definition, which puts the mark into the generated proto comment, the OpenAPI description and the tables on this page. It keeps working for at least one minor release after the mark and is removed only in a new major.
• Breaking changes. A change that cannot be additive ships as vmafx.v2 next to v1; both are served until v1 is retired with a release note.
• Defaults. An unset field means the default in its row. A default the library owns (the default model) follows the library and is named, not copied, so it moves only with the library's release notes.

Where the contract comes from¶

The options and the provenance record are defined once, in the option groups and structs of core/api/vmafx.toml, and generated into every surface (API generation):

Option reference¶