Validate data in CI

Run aontu vet in a pipeline so a document that does not hold fails the build, with the reason attached.

Rendered from docs/how-to/validate-in-ci.md in the engine repository, where a correction belongs, and where the test suite executes every example on this page.

A schema nobody runs is a comment. The vet verb turns yours into a gate: it validates data files against a schema document, prints located findings, and exits with the verdict class, which is everything a pipeline needs to go red for a reason.

Write the schema as schema.aon:

user: { id:integer name:string admin:boolean }

and the data your pipeline produced as user.json:

{"user": {"id": "seven", "name": "ada", "admin": true}}

Now vet it:

$ aontu vet schema.aon user.json
verdict: invalid

$.user.id: no_scalar_unify [conflict]
  [aontu/no_scalar_unify]: Cannot unify values at path $.user.id
  data: user.json:1:17 ("seven")
  schema: schema.aon:1:12 (integer)
$ echo $?
1

The finding names both sides, data site first, because the data is the side to edit. (To refuse extra keys too, wrap the schema in close: see Forbid unexpected keys.)

The exit code is the verdict class#

There is more than one kind of “no”, and a gate that collapses them throws information away. Drop id from the data instead of mistyping it, as user2.json does:

{"user": {"name": "ada", "admin": true}}

$ aontu vet schema.aon user2.json
verdict: incomplete

$.user.id: mapval_no_gen [incomplete]
  [aontu/mapval_no_gen]: Cannot resolve value at path $.user.id
  schema: schema.aon:1:12 (integer)
$ echo $?
3

Nothing contradicts here; the truth is simply not met yet, and the exit code says so:

(Exit 2 is usage: a bad flag or an unreadable file.) A blanket “non-zero is red” still gates correctly, but keeping 1 and 3 apart lets an emit → validate → repair loop route the failure: exit 1 means repair what was emitted, exit 3 means finish it, exit 4 means stop and page whoever owns the schema. The full classes are specified under aontu vet.

Reports for machines#

--format json emits the same findings as one object; --format sarif emits SARIF 2.1.0, the form GitHub code scanning ingests:

$ aontu vet --format sarif schema.aon user.json
{
  "$schema": "https://json.schemastore.org/sarif-2.1.0.json",
  "runs": [
    {
      "invocations": [
        {
          "executionSuccessful": true
        }
      ],
...
$ echo $?
1

The exit code still carries the verdict, so a job can upload the report and stay red.

Wire it in#

In GitHub Actions, the repository ships an action wrapping the verb, vet-action/, which fails the job by verdict class and can hand the SARIF to code scanning:

steps:
  - uses: actions/checkout@v4
  - uses: aontu-lang/aontu/vet-action@main
    with:
      schema: schema.aon
      data: user.json

As a pre-commit hook, the verb is one line, and the verdict classes mean a half-finished document blocks the commit too:

#!/bin/sh
# .git/hooks/pre-commit
exec aontu vet schema.aon user.json

While editing, --watch re-runs the whole vet whenever the schema or a data file changes, streaming one report per run:

$ aontu vet --watch schema.aon user.json

Every run is a full re-parse and re-unify, so what you see on each save is exactly what CI will say.

Make the gate prove it checked something#

A gate that passes because it examined nothing is worse than no gate: it reports success. The usual cause is one construct, and it is silent. A schema written with the wildcard other tools use is a key named * in aontu, so it meets no data key and constrains nothing. Here it is as wild.aon:

entity: { "*": { table: string } }

rows.aon is the data it was meant to check:

entity: planet: table: 42

$ aontu vet --partial --strict-coverage wild.aon rows.aon
verdict: valid
...
coverage: VACUOUS — no data leaf was constrained by the schema; this run checked nothing
coverage: 0/1 data leaves checked, 3 schema declarations
  unchecked: $.entity.planet
  unused: $.entity.*
aontu: no data leaf was constrained by the schema: this run checked nothing
...
$ echo $?
1

--strict-coverage is the CI form: it measures what the run examined and exits 1 when the answer is nothing. The verdict word stays valid, because the unification really did hold; what failed is the gate’s claim to have checked anything. Written with the template &: the same run checks the field and refuses the 42.

Use --coverage alone while writing a schema, to see which declarations no data meets and which data paths nothing constrains, and --coverage-at to measure one subtree.

Vetting gates the data; the schema itself changes too, and that gate is aontu breaking. For decoding an individual finding, see Read a conflict error.