Skip to content

SuperWitness

SuperWitness docshalf-lit: Works, with known gaps the page names.

Every run, on the record.

For one run of agent work, SuperWitness joins what the agent did, how your products behaved while it did it, and whether the work was any good. A source that does not answer shows as unknown, never as a polite 0. Verdicts are append-only, and an agent may not judge a run it executed, however good it thinks the work was.

Ask Captain Jack

Ahoy. Ask me anything about SuperWitness. I answer only from these docs, and I'll tell you when they don't cover it.

An AI writes his answers from the docs; check anything you copy.

Work with an agent

I'm working with SuperWitness. Read https://docs.superwitness.dev/llms.txt to see what its docs cover, and https://docs.superwitness.dev/llms-full.txt for every page in one file.

Then explain how SuperWitness would join my agents' runs, and list what I need before I can record a verdict on one.

Name the docs page each answer comes from, and tell me when the docs do not cover something.

Quick start: join a run and judge it, with the suite faked

git clone https://github.com/SuperJackfruitLabs/superwitness && cd superwitness && go test -v ./internal/join/ ./internal/verdicts/

Needs Go (go.mod asks for 1.26) and nothing else: no Postgres, no SuperPipeline, no AgentPod. The run, its attempts, spans and logs are the repository's in-memory fakes, and verdicts go to an in-memory store. 55 tests, from a slow source showing as timeout with its counts unknown, to an agent refused when it judges its own run. To read a real run document over HTTP, the install guide's fake mode needs only a Postgres.

How a run is recorded, judged against a rubric, and read back (step through it; it waits for you)

1. Recorded. Your products send plain OTLP to the OpenTelemetry Collector, which writes it to VictoriaTraces and VictoriaLogs; SuperWitness only queries them, and nothing waits on it. A source reports its runs to POST /v1/runs as they start, change and finish, so the run registry can list them.
2. Rubric. A rubric is recorded once, with the binary: superwitness rubric-add -id press -version 1 … prints rubric:press@1 recorded. A version can never be changed or recorded again; to change a rubric, record version 2.
3. Judged. A judge posts a verdict to POST /v1/verdicts, or calls the record_verdict MCP tool, naming the standard rubric:press@1. A rubric version that does not exist is 422 unknown_rubric. A non-human judge on a run it executed is 403 self_judgement; a human may judge any run.
4. Corrected. Database triggers refuse UPDATE, DELETE and TRUNCATE on verdicts and rubrics. A change of mind is a new verdict that names the old one in supersedes, and only the old verdict's judge may write it.
5. Read back. The app, GET /v1/runs/superpipeline/{board}/{run} and the get_run MCP tool all return one run document, assembled at read time. Each source has 2 s by default (SW_SOURCE_TIMEOUT); one that does not answer shows as timeout or unavailable, and its numbers as unknown, never 0. Only the latest verdict in each chain is shown.

A run can leave evidence in several places. We bring the answers together without sanding off the gaps, so you can ask three plain questions.

What did it do? Follow the attempts that executed the run, the agent configuration behind each, and the spans they left. In the run document, start with attempts and trace.

How did the products run? Read the logs and errors that share the run’s id or its trace. In the run document, that is log_count and errors.

Was it any good? Put gate decisions from SuperPipeline beside the verdicts recorded in SuperWitness. In the run document, both arrive under verdicts.

That gives you one record to investigate instead of a small expedition through several systems.

SuperWitness is a single Go binary and a Postgres database, self-hosted and MIT licensed. Start with what it is, then install it.

For agents, the same docs are published as plain text: /llms.txt indexes them, and /llms-full.txt is every page in one file.