MCP servers¶

VMAFx ships three Model Context Protocol (MCP) servers that expose scoring and tooling to LLM clients such as Claude Desktop and Cursor. Pick the one that fits your deployment from the table below, then follow its install and run section.

Model Context Protocol is a JSON-RPC protocol that lets an LLM call declared tools with typed arguments. All three servers are additive; running any combination at once is fine.

Pick a server¶

The embedded server is documented in embedded.md. The rest of this page covers the Python and Go servers; the Go server section starts at Go implementation.

Use an MCP server when you want an LLM to:

• score a (reference, distorted) YUV pair and reason about the result,
• enumerate which VMAF models shipped with the build,
• probe which runtime backends (CPU / CUDA / SYCL / HIP / Metal) the local binary can dispatch to,
• run the Netflix benchmark harness and summarise the output,
• evaluate a tiny-AI ONNX regressor against a parquet feature cache on a deterministic split and report PLCC / SROCC / RMSE,
• rank several candidate tiny-AI models on the same split.

The servers exec the repo's own built vmaf binary under argv, never a shell string, and refuse any file path that is not under an allowlisted root. See security.

Tool catalogue¶

The table lists all 24 tools. "Python" is yes for the 19 tools the Python server also serves; full schemas, examples and error codes are in tools.md.

All tools return a single TextContent message whose body is a JSON document. On error the body is {"error": "<message>"} with the same shape so the client can always json.loads() the response.

Execution mechanics¶

Migrating the remaining process-based tools to direct calls is tracked in ADR-0704 and ADR-0931. The embedded C server (core/src/mcp/dispatcher.c, ADR-0209) runs entirely in-process, without child processes, and serves exactly two tools: list_features and compute_vmaf.

Install (Python server)¶

From a checkout of the repo:

1. Build vmaf from the repository root (Meson + Ninja; see building from source).

   meson setup build core -Denable_cuda=false -Denable_sycl=false
   ninja -C build
   

2. Install the MCP server package.

   cd mcp-server/vmaf-mcp
   pip install -e .
   

3. Optional: pull in the ML dependencies for eval_model_on_split and compare_models.

   pip install -e '.[eval]'
   

The server lands as vmaf-mcp on your PATH. It finds the vmaf CLI in this order: VMAF_BIN, /usr/local/bin/vmaf, <repo>/core/build/tools/vmaf, <repo>/build/tools/vmaf.

Testing¶

The test suite is in mcp-server/vmaf-mcp/tests/. The package pyproject.toml sets pythonpath = ["src"] under [tool.pytest.ini_options], so pytest run from the package directory finds the in-tree vmaf_mcp module without an editable installation:

cd mcp-server/vmaf-mcp
pytest

Run (Python server)¶

# Default stdio transport, as used by Claude Desktop and Cursor
vmaf-mcp

No network ports are opened. The server reads JSON-RPC requests from stdin and writes responses to stdout; diagnostic logs go to stderr. For the HTTP mode see http-transport.md.

Claude Desktop configuration¶

Drop this into ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%/Claude/claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "vmaf-local": {
      "command": "vmaf-mcp",
      "env": {
        "VMAF_BIN": "/home/you/dev/vmaf/build/tools/vmaf",
        "VMAF_MCP_ALLOW": "/home/you/yuv-corpus:/home/you/renders"
      }
    }
  }
}

A complete example covering the Docker image variant lives in mcp-server/vmaf-mcp/claude-desktop-config-example.json.

Environment variables¶

The tool-handler variables below are read by both servers unless the last column says otherwise. The variables only the Go server reads (its transport, the control-plane tools, the direct scoring path) are in its environment table.

The HTTP security variables (VMAFX_MCP_HTTP_TOKEN, ..._NO_AUTH, ..._BIND, ..._TLS_CERT, ..._TLS_KEY) are documented in http-transport.md.

Security model¶

The server is meant to run on the user's own machine, driven by a local LLM client. Because the LLM could craft input that tries to read arbitrary host paths, the server enforces a path allowlist.

Built-in roots, always allowed:

Extra roots can be added with VMAF_MCP_ALLOW=<abs-path>[:<abs-path>...].

Any tool argument that names a file (ref, dis, model, features, each member of models) is resolved with Path.resolve() and rejected unless it lands under one of the allowed roots and refers to an existing regular file. .. segments and symlinks that escape the allowlist are rejected by resolution.

The underlying CLI is exec'd with an argv list, never a shell string, so there is no pathway for shell-metacharacter injection.

See also ai/security.md for the tiny-AI-specific hardening (ONNX operator allowlist, model size cap).

When not to use the MCP server¶

• Bulk scoring in a pipeline — use the vmaf CLI directly. MCP is request/response; the CLI streams pictures and does not pay JSON-RPC overhead per frame.
• Integration into your own code — use the C API or the Python bindings for an in-process surface.
• CI checks — the Docker image is a better fit than stdio-attached MCP.

MCP shines when the caller is an LLM that benefits from a tool-calling interface with declared schemas and a JSON-shaped response.

Go implementation — vmafx-mcp¶

vmafx-mcp is a single static Go binary that serves the 19 tools of the Python server with byte-for-byte schema parity (ADR-0704) plus the 5 control-plane tools. It is the recommended implementation for deployments that cannot install a Python environment.

Build¶

The binary needs no runtime dependencies other than the vmaf CLI binary (resolved via VMAF_BIN or the standard search order). From the repository root, with Go 1.27 or newer:

go build -o vmafx-mcp ./cmd/vmafx-mcp/

Run¶

The Go binary runs on the golusoris fx framework (ADR-1119) and is configured entirely through environment variables; it has no CLI flags.

# Default stdio transport, a drop-in replacement for vmaf-mcp
vmafx-mcp

# Streamable-HTTP transport on the default address :3000
VMAFX_MCP_TRANSPORT=http vmafx-mcp

# Streamable-HTTP transport on a custom address
VMAFX_MCP_TRANSPORT=http VMAFX_MCP_HTTP_ADDR=:8080 vmafx-mcp

The HTTP transport serves the MCP streamable-HTTP protocol at the listen address. It does not serve the REST routes of the Python HTTP mode; see http-transport.md for the comparison and the security variables.

Environment (vmafx-mcp)¶

Every variable the Go binary reads, from the platform definition:

Claude Desktop configuration (Go binary)¶

{
  "mcpServers": {
    "vmafx-local": {
      "command": "/path/to/vmafx-mcp",
      "env": {
        "VMAF_BIN": "/home/you/dev/vmaf/build/tools/vmaf",
        "VMAF_MCP_ALLOW": "/home/you/yuv-corpus"
      }
    }
  }
}

Differences from the Python server¶

The size and startup figures are indicative, not a recorded measurement. The Go evaluation tools need a libvmaf built with -Denable_dnn; otherwise they return a clear "built without DNN support" error, mirroring Python's missing-[eval]-extra behaviour.

Go environment variables¶

Every variable vmafx-mcp reads, the framework's VMAFX_LOG_* and VMAFX_OTEL_* keys included, is in its environment table. All framework logging goes to stderr, so the stdio JSON-RPC stream on stdout stays uncorrupted.

Startup contract: all tools or none¶

vmafx-mcp serves the complete 24-tool surface or it does not start. If a tool's input schema fails to serialise at startup, the process writes the failing tool's name to stderr and exits non-zero.

The failure is a defect in the server's own schema literals, not something an operator can configure, so there is no flag to relax it. If vmafx-mcp exits at startup with registering the MCP tool surface: tool "<name>": ..., report the tool name; nothing in the environment can cause or fix it.

Tests¶

go test ./cmd/vmafx-mcp/ -v

TestToolListMatchesPython and TestToolSchemasMatchPython compare the served tools with the Python server's tool list, read from mcp-server/vmaf-mcp/tool-contract.json: every Python tool must be served with the same property names, JSON types and required arguments, and every other tool must be one of the five declared Go-only tools. They need neither Python nor the test YUVs (the package still links libvmaf, like every go test here). TestVmafScoreTool and TestGoVsPythonOutputParity require the Netflix golden YUVs and the vmaf binary; they skip automatically when these are absent.

The contract file is written from the Python server's _list_tools(), never by hand. After adding or changing a tool in server.py, regenerate it and commit it with the change:

PYTHONPATH=mcp-server/vmaf-mcp/src python3 -m vmaf_mcp.tool_contract --write

mcp-server/vmaf-mcp/tests/test_tool_contract.py fails while the file is stale (python3 -m vmaf_mcp.tool_contract --check gives the same answer), and a change to the file runs the Go checks in CI, so the Go tests then name the tool the Go server still lacks.

Related¶

• Tool reference — request/response schemas and error codes for every tool.
• Backend discovery and default allowlist — how list_backends probes compiled-in GPU runtimes and which paths the server accepts without VMAF_MCP_ALLOW (ADR-0511).
• Embedded server — the in-process server inside libvmaf.
• ADR-0100 — the per-surface doc bar this page satisfies (MCP tool: what / schema / allowed paths / example / error codes).
• ADR-0704 — decision record for the Go port.
• mcp-server/vmaf-mcp/README.md — short-form README kept alongside the Python code.