Adopt now v1

Use your existing runner. Add eval-dashboards as a reporting and gating layer.

What to keep

  • Keep your current test or eval runner.
  • Keep your current CI workflow.
  • Keep your existing datasets and assertions.

What to add

  1. Install @icodenet/eval-dashboards.
  2. Write eval results into .evals_output/*.json as eval-report/v1.
  3. Run eval-dashboards lint, check, and report.
  4. Publish the static report directory in CI.

Copy-paste path (simple)

pnpm add -D @icodenet/eval-dashboards

# optional starter scaffolding
eval-dashboards init --preset=agent-quality --write --playbook

# keep your existing runner command here
pnpm test

# eval-dashboards layer
eval-dashboards lint --input=.evals_output
eval-dashboards check --input=.evals_output --max-new-failures=0 --zero-critical --json-out=eval-report/check-result.json
eval-dashboards report --input=.evals_output --reporter=html --reporter=json-summary --report-dir=eval-report

If your runner is Promptfoo, DeepEval, or AgentEvals

eval-dashboards import --from=promptfoo --input=./promptfoo-results.json --out=.evals_output/import-promptfoo.json
# or
eval-dashboards import --from=deepeval --input=./deepeval-results.json --out=.evals_output/import-deepeval.json
# or
eval-dashboards import --from=agentevals --input=./agentevals-results.json --out=.evals_output/import-agentevals.json

# then run the same lint/check/report flow
eval-dashboards lint --input=.evals_output
eval-dashboards check --input=.evals_output --max-new-failures=0 --zero-critical
eval-dashboards report --input=.evals_output --reporter=html --report-dir=eval-report

Definition of done for adoption

  • Your CI job fails on real regressions (not just test command failures).
  • Your team can open one static report URL and see row-level evidence.
  • You can compare current run vs baseline with stable row ids.

Adoption map (candidate repo targets)

See docs/adoption-map.md for three candidate existing-runner adoption targets and PR-style change sets. Not endorsed or accepted by those projects.