Tiny AI — run provenance sidecars¶

Every tiny-AI script that writes a durable JSON report records a run_provenance block, so a model, feature table or report can be traced back to the exact inputs, arguments and outputs that produced it. This page lists the block, the shared helpers, and which script writes which report. For the training flow itself see training.md. The convention is recorded in ADR-0661.

What the block records¶

run_provenance is intentionally compact:

Add provenance to a script¶

Use the shared helpers in aiutils.run_manifest:

The Claude workflow for adding or auditing sidecars is .claude/skills/ai-run-manifest/SKILL.md.

CLI helper layer¶

Operator-facing scripts that fit the shared shape use aiutils.cli_helpers:

Table-specific defaults, row schemas, and materializer options stay in the individual scripts; the helpers only cover boilerplate that should not drift.

Script bootstrap¶

Directly executable ai/scripts/*.py files call bootstrap_ai_script(__file__) from ai/scripts/_script_bootstrap.py before importing shared repo-local modules. It resolves the script path, repository root, ai/src, ai/scripts, and the optional tools/vmaf-tune/src root without copying ad hoc sys.path.insert(...) blocks into every script. Enable only the roots the script needs:

Which script writes which report¶

Model training and export¶

Use the saliency block when comparing v1/v2 saliency refreshes or replaying a DUTS-rooted training run from a model card.

Evaluation and validation¶

Use the evaluation block when comparing refreshed LOSO or multi-seed numbers instead of relying on shell history. The legacy reports stay comparable with the refreshed v3/v4/v5 reports because they use the same schema.

Feature tables, corpora and fetchers¶

The BVI-DVC adapter also emits the current vmaf-tune v3 additive columns with explicit unavailable defaults, so cached BVI rows stay schema-compatible. The fetcher sidecars let a later JSONL or parquet manifest be traced back to the original local download instead of an anonymous corpus directory.

Operational notes for collect_gpu_calibration_data.py:

• The default backend is CUDA and the default architecture label is cuda:default. Collect from SYCL hardware with --backends sycl --arch-id <stable-device-id>.
• Scorer output must be a JSON object with a frames array of frame objects. Malformed output is rejected before metric pairing.

Analyses, calibration and quantisation¶

Prefer the quantisation reports for model-card evidence instead of terminal logs. calibrate_nr_threshold.py quality-gates sidecar writes with --min-calibration-samples and --min-plcc. A weak fit still leaves a Markdown report for diagnosis, but does not update the tune-facing JSON unless --allow-weak-calibration is set.

Feature-correlation analysis rules¶

feature_correlation.py decides which columns it analyses, and records what it skipped, so a report can be replayed from the file alone.

1. Only columns whose parquet-backed pandas dtype is numeric count as features. Text, boolean, categorical, object and decimal columns are skipped. A numerically encoded category is included, because the physical dtype, not the column name, is authoritative.
2. The report keeps input-schema order in feature_cols and records excluded columns in skipped_non_numeric_columns, skipped_all_nan_columns and skipped_constant_columns.
3. All-null numeric features are removed before complete-case filtering, so one unavailable metric does not erase every row (common in partially populated aggregate tables).
4. Remaining per-row missing or non-finite values (NaN, positive and negative infinity) are excluded across the selected features and target before NumPy or scikit-learn sees the matrix. A table with no usable numeric feature, or no finite complete feature/target row, fails with a direct diagnostic.
5. Constant numeric columns are excluded: their Pearson correlation is undefined, NumPy warns under the warnings-as-errors policy, and a zero-importance tie must not promote them into consensus_topk. The check repeats after complete-case filtering, because removing rows for a sibling feature can turn a varying column into a constant.
6. If optional scikit-learn analysis is unavailable, its method maps and top-K lists are empty rather than carrying non-standard JSON NaN placeholders.
7. --redundancy-threshold rejects non-finite spellings at argument parsing, and the assembled report is validated with strict RFC-8259 number semantics before the atomic manifest write.