CI Gates v1

Recommended order

eval-dashboards lint --input=.evals_output
eval-dashboards check --input=.evals_output --require-suite-pass=preflight --new-failure-key=scenario-category --max-new-failures=0 --zero-critical
eval-dashboards report --input=.evals_output --reporter=html --reporter=json-summary --report-dir=eval-dashboard

Baseline strategies

# PR lane
eval-dashboards check --input=.evals_output --baseline-strategy=rolling --allow-blocked-baseline --max-new-failures=0 --zero-critical

# Main lane
eval-dashboards check --input=.evals_output --baseline-strategy=champion --baseline-lookback=20 --max-new-failures=0 --zero-critical

Warning budgets

eval-dashboards check --input=.evals_output   --max-warnings=5   --max-warning-code=missing-kind:0   --fail-on-warning-code=missing-judge-model

CI-native machine output

eval-dashboards check --input=.evals_output --max-new-failures=0 --zero-critical --json-out=eval-report/check-result.json

# check-result.json includes newlyFailingRows[].reportAnchor for one-hop row evidence links

More: docs/gates.md.