This is the full developer documentation for SuperWitness # SuperWitness > Observability and evaluation for agent fleets. What your agents did, how your products behaved while they did it, and whether the work was any good, joined on one run. SuperWitness docshalf-lit: Works, with known gaps the page names. ## Every run, on the record. For one run of agent work, SuperWitness joins what the agent did, how your products behaved while it did it, and whether the work was any good. A source that does not answer shows as unknown, never as a polite 0. Verdicts are append-only, and an agent may not judge a run it executed, however good it thinks the work was. [What it is →](/what-it-is/)[Install](/install/)[superwitness.dev ↗](https://superwitness.dev) ### Ask Captain Jack Ahoy. Ask me anything about SuperWitness. I answer only from these docs, and I'll tell you when they don't cover it. Ask Captain Jack about SuperWitnessAsk Captain Jack…Ask Why does my run say span_count is unknown?Can an agent grade its own work?How do I correct a verdict I already recorded?Can I try it without SuperPipeline? An AI writes his answers from the docs; check anything you copy. ### Work with an agent ``` I'm working with SuperWitness. Read https://docs.superwitness.dev/llms.txt to see what its docs cover, and https://docs.superwitness.dev/llms-full.txt for every page in one file. Then explain how SuperWitness would join my agents' runs, and list what I need before I can record a verdict on one. Name the docs page each answer comes from, and tell me when the docs do not cover something. ``` Copy prompt ### Quick start: join a run and judge it, with the suite faked ``` $ git clone https://github.com/SuperJackfruitLabs/superwitness && cd superwitness && go test -v ./internal/join/ ./internal/verdicts/ ``` Copy Needs Go (go.mod asks for 1.26) and nothing else: no Postgres, no SuperPipeline, no AgentPod. The run, its attempts, spans and logs are the repository's in-memory fakes, and verdicts go to an in-memory store. 55 tests, from a slow source showing as timeout with its counts unknown, to an agent refused when it judges its own run. To read a real run document over HTTP, the install guide's fake mode needs only a Postgres. ### How a run is recorded, judged against a rubric, and read back (step through it; it waits for you) 1Recorded2Rubric3Judged4Corrected5Read back 1\. Recordedspans and logsrun reported 2\. Rubricrubric:press\@1 3\. Judgedthe agent that ran ita reviewer 4\. Correctededit in placesupersedes vrd\_… 5\. Read backtraces: timeoutspan_count: "unknown" **1. Recorded.** Your products send plain OTLP to the OpenTelemetry Collector, which writes it to VictoriaTraces and VictoriaLogs; SuperWitness only queries them, and nothing waits on it. A source reports its runs to POST /v1/runs as they start, change and finish, so the run registry can list them. **2. Rubric.** A rubric is recorded once, with the binary: superwitness rubric-add -id press -version 1 … prints rubric:press\@1 recorded. A version can never be changed or recorded again; to change a rubric, record version 2. **3. Judged.** A judge posts a verdict to POST /v1/verdicts, or calls the record_verdict MCP tool, naming the standard rubric:press\@1. A rubric version that does not exist is 422 unknown_rubric. A non-human judge on a run it executed is 403 self_judgement; a human may judge any run. **4. Corrected.** Database triggers refuse UPDATE, DELETE and TRUNCATE on verdicts and rubrics. A change of mind is a new verdict that names the old one in supersedes, and only the old verdict's judge may write it. **5. Read back.** The app, GET /v1/runs/superpipeline/{board}/{run} and the get_run MCP tool all return one run document, assembled at read time. Each source has 2 s by default (SW_SOURCE_TIMEOUT); one that does not answer shows as timeout or unavailable, and its numbers as unknown, never 0. Only the latest verdict in each chain is shown. ← BackStep 1 of 5Next → ### Find your way Copy Markdown ## Three questions, one run [Section titled “Three questions, one run”](#three-questions-one-run) A run can leave evidence in several places. We bring the answers together without sanding off the gaps, so you can ask three plain questions. **What did it do?** Follow the attempts that executed the run, the agent configuration behind each, and the spans they left. In the run document, start with `attempts` and `trace`. **How did the products run?** Read the logs and errors that share the run’s id or its trace. In the run document, that is `log_count` and `errors`. **Was it any good?** Put gate decisions from SuperPipeline beside the verdicts recorded in SuperWitness. In the run document, both arrive under `verdicts`. That gives you one record to investigate instead of a small expedition through several systems. SuperWitness is a single Go binary and a Postgres database, self-hosted and MIT licensed. Start with [what it is](/what-it-is/), then [install it](/install/). ! Heads up v0.0.12 is the release in production. SuperWitness is meant to stand alone, and this release is not there yet: it reads runs from SuperPipeline and attempts from AgentPod, and without them it runs only in fake mode, which serves one development run on loopback ([Try it without the other products](/install/#try-it-without-the-other-products)). Pages say where a feature is on `main` and not yet released. Captain Jack says I read these docs so you don’t have to read all of them at once. Ask me in the box up top, and if the answer isn’t in here I’ll say so rather than make one up. For agents, the same docs are published as plain text: [`/llms.txt`](/llms.txt) indexes them, and [`/llms-full.txt`](/llms-full.txt) is every page in one file. # HTTP API > The HTTP routes SuperWitness serves, their parameters and their errors. SuperWitness serves a small JSON API under `/v1`. If you are building a client, we can keep the map simple: the [MCP tools](/build/mcp/) are thin layers over the same operations, and this page is the literal route, parameter and error reference. Eight routes have no tool: `GET /v1/runs/by-attempt/{attempt}`, `POST /v1/runs`, the two rubric reads, the transcript step route (`GET /v1/runs/superpipeline/{board}/{run}/transcript/items/{seq_from}`), `GET /v1/me`, `GET /v1/verdicts` and `GET /v1/scopes`. (`GET /v1/runs` is the `list_runs` tool.) ## Authentication [Section titled “Authentication”](#authentication) Every `/v1` route takes a bearer token with SuperWitness’s `SW_PUBLIC_URL` as its audience, issued by the AgentPod hub or, when `SW_ORG_PLANE_ISSUER` is set, by the organization plane (hub tokens are then refused): ```plaintext Authorization: Bearer ``` [With AgentPod and SuperPipeline](/use/with-agentpod-and-superpipeline/#tokens-callers-present) lists what a token must carry. A missing or refused token is 401 `unauthenticated`. In fake mode, tokens are `dev::`. A browser signed in to [the app](/use/the-app/) is authenticated by its session cookie instead; changes made that way must carry `Origin` equal to `SW_PUBLIC_URL`. A bearer token always wins over a cookie. `POST /v1/runs` and `/mcp` take bearer tokens only. A person’s token is resolved to their `prn_` principal id; if AgentPod cannot be asked, the answer is 503 `principal_unresolved`. ## Routes [Section titled “Routes”](#routes) | Method | Path | Answers | | ------ | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | | `GET` | `/v1/runs/superpipeline/{board}/{run}` | 200 with the [run document](/build/contracts/#the-run-document) | | `GET` | `/v1/runs/superpipeline/{board}/{run}/spans` | 200 with one page of spans | | `GET` | `/v1/runs/superpipeline/{board}/{run}/logs` | 200 with one page of log lines | | `GET` | `/v1/runs/superpipeline/{board}/{run}/transcript` | 200 with one page of an attempt’s transcript ([Transcripts](/use/transcripts/)) | | `GET` | `/v1/runs/superpipeline/{board}/{run}/transcript/items/{seq_from}` | 200 with one transcript step whole | | `GET` | `/v1/runs/by-attempt/{attempt}` | 302 to the run document | | `POST` | `/v1/runs` | 200 with one result per report ([Run registry API](/build/run-registry/)) | | `GET` | `/v1/runs` | 200 with one page of registry runs ([Run registry API](/build/run-registry/#listing-runs)) | | `GET` | `/v1/rubrics` | 200 with every rubric version, without bodies | | `GET` | `/v1/rubrics/{id}/{version}` | 200 with one rubric version and its body | | `GET` | `/v1/me` | 200 with `{"principal", "kind", "email", "via"}`: who the caller is, and whether by `session` or `bearer` | | `GET` | `/v1/verdicts` | 200 with every verdict on one subject (`subject_kind`, `subject_ref`), oldest first, each with `superseded_by`; past 500, the newest 500 | | `GET` | `/v1/scopes` | 200 with every scope runs were reported in: `{"scopes": [{"source", "id", "name", "runs"}]}` | | `POST` | `/v1/verdicts` | 201 with a new verdict, or 200 with the one already recorded under the key | Path parameters: * `{board}` and `{run}` are SuperPipeline board and run ids: letters, digits, `_` and `-`, starting with a letter or digit, up to 128 characters. Otherwise 400 `invalid_run_ref`. * `{attempt}` is `attempt_` followed by 1 to 128 letters, digits, `_` or `-`. Otherwise 400 `invalid_attempt_id`. JSON answers carry `Content-Type: application/json` and `Cache-Control: no-store`. ### GET /v1/runs/superpipeline/{board}/{run} [Section titled “GET /v1/runs/superpipeline/{board}/{run}”](#get-v1runssuperpipelineboardrun) The run document, assembled from every source at read time. It is 404 `run_not_found` only when SuperPipeline and the AgentPod hub both say they do not know the run; any other source failure is a 200 with that source’s status in `sources`. [Read a run](/use/read-a-run/#the-run-document). ### GET /v1/runs/superpipeline/{board}/{run}/spans [Section titled “GET /v1/runs/superpipeline/{board}/{run}/spans”](#get-v1runssuperpipelineboardrunspans) | Query | Meaning | | -------- | ----------------------------------------------------------------- | | `cursor` | `next_cursor` from the previous page; left out for the first page | | `limit` | page size, 1 or more; default 100, and anything above 500 is 500 | Answers `{"spans": [...], "next_cursor": "…"}`, oldest first; `next_cursor` is left out on the last page. Each span has `trace_id`, `span_id`, `parent_span_id` (left out for a root span), `name`, `service`, `start`, `duration_ms` and `attributes` (string values). ### GET /v1/runs/superpipeline/{board}/{run}/logs [Section titled “GET /v1/runs/superpipeline/{board}/{run}/logs”](#get-v1runssuperpipelineboardrunlogs) | Query | Meaning | | -------- | ----------------------------------------------------------------- | | `cursor` | `next_cursor` from the previous page; left out for the first page | | `level` | `debug`, `info`, `warn` or `error`; left out for every level | | `limit` | page size, 1 or more; default 100, and anything above 500 is 500 | Answers `{"logs": [...], "next_cursor": "…" or null, "trace_join": ""}`, oldest first; `next_cursor` is `null` on the last page. Each line has `at`, `service`, `level`, `message`, and, when the line has them, `trace_id`, `span_id` and `run_id`. `trace_join` is the traces source’s status while the run’s trace ids were looked up; when it is not `ok`, lines were matched by run id only. [Read a run](/use/read-a-run/#logs-one-page-at-a-time). ### GET /v1/runs/superpipeline/{board}/{run}/transcript [Section titled “GET /v1/runs/superpipeline/{board}/{run}/transcript”](#get-v1runssuperpipelineboardruntranscript) | Query | Meaning | | -------------------- | ----------------------------------------------------------------------------------- | | `attempt` | the attempt to read; required when the run has more than one | | `seq_from`, `seq_to` | the range, inside the attempt; default the whole attempt | | `cursor` | `next_cursor` from the previous page; send the same `seq_from` and `seq_to` with it | Needs a signed-in session or a token granted `transcripts:read`. Answers AgentPod’s transcript page unchanged, plus `attempt_id`, with `Cache-Control: no-store`; 200 steps a page. `GET …/transcript/items/{seq_from}?attempt=&full=1` answers one step whole, by its first seq; it also accepts `seq_from` and `seq_to`, the range the step was shown in, which must lie inside the attempt. [Transcripts](/use/transcripts/) has the shape, the redaction and the audit. ### GET /v1/runs/by-attempt/{attempt} [Section titled “GET /v1/runs/by-attempt/{attempt}”](#get-v1runsby-attemptattempt) 302 with `Location: /v1/runs/superpipeline/{board}/{run}` for the run the attempt belongs to, and no body. ### GET /v1/rubrics and GET /v1/rubrics/{id}/{version} [Section titled “GET /v1/rubrics and GET /v1/rubrics/{id}/{version}”](#get-v1rubrics-and-get-v1rubricsidversion) `GET /v1/rubrics` answers `{"rubrics": [...]}`, ordered by id and then newest version first. Each has `id`, `version`, `standard` (`rubric:@`, what a verdict names), `name`, `scale` as recorded, `recognised_scale`, `created_by` and `created_at`. The single-rubric route adds `body`. `recognised_scale` is the scale in one of the shapes on [Verdicts](/use/verdicts/#rubric-scales), or `null` when it is in none of them. ### POST /v1/verdicts [Section titled “POST /v1/verdicts”](#post-v1verdicts) Body: one JSON verdict request, at most 64 KiB, with no unknown fields. Answers the stored verdict. [Verdicts](/use/verdicts/) has every field and rule; [Contracts](/build/contracts/#verdicts) has the shape. ## Outside /v1 [Section titled “Outside /v1”](#outside-v1) | Path | | | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `GET /health` | No token. 200 whenever the process is alive, with each source’s status in the body ([Operations](/use/operations/#health)). | | `/mcp` | The MCP server, with the same bearer token ([MCP tools](/build/mcp/)). A request that arrives through the public edge (from a `SW_TRUSTED_PROXIES` address, carrying `CF-Connecting-IP`) is answered 404 before authentication. | | `GET /auth/login`, `GET /auth/callback`, `POST /auth/logout` | Signing in to and out of the app ([The app](/use/the-app/)). 404 “Sign-in is off” when no one is allowed to sign in. Logout answers 204, or 503 `store_unavailable` with the cookie cleared. | | `GET /` (the runs list), `GET /runs/superpipeline/{board}/{run}`, `GET /rubrics`, `GET /rubrics/{id}/{version}` | Pages of the app, built into the binary ([The app](/use/the-app/)). | Any other `GET` or `HEAD` outside `/v1` and `/mcp` is served by the app: one of its files, or the page itself for a path without a file extension. Other methods on unknown paths are 404 `not_found`. A wrong method on a route that exists, such as `POST /health`, is 405 `method_not_allowed`. ## Errors [Section titled “Errors”](#errors) Every error from `/v1` is one JSON object: ```json {"error":{"code":"logs_unavailable","message":"logs is unavailable","retryable":true}} ``` * `code` is stable; match on it. * `message` is for people and may change. * `retryable` is present, and `true`, only when the same request may succeed later. A retryable 503 also carries `Retry-After: 5`. ### Every code [Section titled “Every code”](#every-code) | Status | Code | When | | ------ | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | 400 | `invalid_run_ref` | a board or run id is malformed | | 400 | `invalid_attempt_id` | the attempt id is not `attempt_` | | 400 | `invalid_limit` | `limit` is not a whole number of 1 or more | | 400 | `invalid_cursor` | `cursor` is not one SuperWitness issued | | 400 | `invalid_seq` | `seq_from` or `seq_to` is not a whole number of 0 or more | | 400 | `attempt_required` | a transcript of a run with several attempts names none | | 400 | `range_outside_attempt` | the transcript range is not inside the attempt | | 400 | `invalid_level` | `level` is not `debug`, `info`, `warn` or `error` | | 400 | `invalid_source` | `source` is not a source name | | 400 | `invalid_status` | a `status` is not one of the six | | 400 | `invalid_time` | `since` or `until` is not RFC 3339 | | 400 | `invalid_needs_verdict` | `needs_verdict` is not `true` or `false` | | 400 | `invalid_rubric_ref` | the rubric id or version is malformed | | 400 | `invalid_json` | the body is not exactly one JSON object of known fields, or is over 64 KiB (verdicts); not one report or a batch of 1 to 100 (runs) | | 400 | `invalid_idempotency_key` | the key is missing, over 200 bytes, or holds NUL | | 400 | `invalid_kind` | `kind` is not `grader`, `review`, `eval` or `calibration` | | 400 | `invalid_subject_kind` | `subject_kind` is not `run`, `attempt` or `eval_case_run` | | 400 | `invalid_subject` | `subject_ref` does not fit its `subject_kind` | | 400 | `invalid_value` | `value` is not exactly one of decision, score (0 to 1), label or text | | 400 | `invalid_comment` | `comment` holds NUL | | 400 | `comment_too_long` | `comment` is over 10000 characters | | 400 | `invalid_evidence_refs` | `evidence_refs` is not up to 100 span ids or session ranges | | 400 | `invalid_supersedes` | `supersedes` is not a verdict id | | 400 | `judge_mismatch` | `judge` is set to someone other than the caller | | 400 | `judge_kind_not_accepted` | `judge_kind` is set; it comes from the caller’s principal record | | 401 | `unauthenticated` | the bearer token is missing or not valid for this service | | 403 | `origin_mismatch` | a change made with a session did not come from `SW_PUBLIC_URL` | | 403 | `transcripts_forbidden` | a transcript read with a token that lacks `transcripts:read` | | 403 | `not_authorised` | the session’s person is no longer on `SW_ALLOWED_PRINCIPALS` | | 403 | `bearer_required` | a run report made with a session | | 403 | `service_principal_required` | a run report from a principal that is not a service | | 403 | `insufficient_scope` | a run report whose token lacks `runs:write` | | 403 | `source_not_allowed` | a run report for a source the reporter is not bound to | | 403 | `self_judgement` | a non-human judge’s verdict on a run or attempt it executed | | 403 | `not_original_judge` | superseding someone else’s verdict | | 404 | `run_not_found` | neither SuperPipeline nor the hub knows the run | | 404 | `attempt_not_found` | the hub does not know the attempt, or the run has no such attempt | | 404 | `session_not_found` | AgentPod holds no such session or step, or the attempt has no session | | 404 | `attempt_has_no_run` | the attempt was not dispatched from a SuperPipeline run | | 404 | `subject_not_found` | the verdict’s run or attempt does not exist | | 404 | `rubric_not_found` | no such rubric version | | 404 | `not_found` | no such route | | 405 | `method_not_allowed` | the route exists, but not with this method | | 409 | `idempotency_conflict` | the key was already used for a different verdict | | 409 | `already_superseded` | the verdict named in `supersedes` already has a successor | | 413 | `item_too_large` | the step is over 1 MiB | | 413 | `body_too_large` | a run report body over 256 KiB | | 422 | `invalid_report` | a run report breaks a rule; a batch’s error has `index` | | 422 | `missing_standard` | the verdict names no standard | | 422 | `invalid_standard` | the standard is not `rubric:@`, `stage:` or `case:` | | 422 | `unknown_rubric` | the rubric version does not exist | | 422 | `supersedes_not_found` | the verdict named in `supersedes` does not exist | | 422 | `supersedes_mismatch` | the correction names a different subject or standard | | 429 | `rate_limited` | over a [rate limit](/use/the-app/#rate-limits); `Retry-After` says when to retry | | 500 | `internal` | an unexpected failure | | 502 | `hub_refused` | AgentPod refused SuperWitness’s credential for transcripts (its grant lacks `transcripts:read`) | | 502 | `_unauthorized` | a source refused SuperWitness’s credential | | 503 | `_unavailable` | a source could not be reached or failed (retryable) | | 503 | `source_unavailable` | AgentPod could not be read for a transcript (retryable) | | 503 | `store_unavailable` | the database is unavailable (retryable) | | 503 | `principal_unresolved` | AgentPod could not say which principal a person’s token names (retryable) | | 503 | `subject_unresolved` | SuperWitness could not confirm the subject or who executed it (retryable) | | 504 | `_timeout` | a source did not answer in time (retryable) | `` is the source a route depends on: `traces` for spans, `logs` for logs, and `agentpod` for `by-attempt`. So the source codes are `traces_unavailable`, `traces_timeout`, `traces_unauthorized`, `logs_unavailable`, `logs_timeout`, `logs_unauthorized`, `agentpod_unavailable`, `agentpod_timeout` and `agentpod_unauthorized`. The run document itself never fails for a source; it reports the source’s status instead. The 401 comes from the authentication layer and carries `WWW-Authenticate: Bearer realm="superwitness"`; its body has `code` and `message` only. Match on `code`, honor `retryable` and `Retry-After`, and your client can explain a failure without teaching itself to parse prose that is allowed to change. # Contracts > The run document and verdict shapes SuperWitness returns, and the recorded payloads that pin what it reads from other products. When you are implementing a client, “roughly this shape” is not much of a contract. This page lists every field SuperWitness returns in a run document and a verdict, plus the payloads it reads from other products. Use [Concepts](/concepts/) for meaning; stay here for the exact shape. Two rules hold throughout: * **Nothing a source could not supply is shown as 0 or left out.** A text field reads `"unknown"`, a count reads `"unknown"`, and a list the source would have filled comes back empty, with `sources` saying why. * **Do not rely on key order.** Objects keep the order shown here; the keys of `sources` and of span `attributes` come back sorted. ## The run document [Section titled “The run document”](#the-run-document) What `GET /v1/runs/superpipeline/{board}/{run}` and the `get_run` tool return. Every top-level key is always present: | Key | Type | | ----------- | ----------------------------- | | `run` | object, below | | `attempts` | array, possibly empty | | `trace` | object | | `errors` | array, possibly empty | | `verdicts` | array, possibly empty | | `cost` | object | | `log_count` | number, or `"unknown"` | | `sources` | object: source name to status | | `links` | object | ### `run` [Section titled “run”](#run) | Field | Value | | ------------ | ---------------------------------------------------------------------------------------------------- | | `ref` | `superpipeline:/` | | `card` | `{"id", "title"}`, each a string or `"unknown"` | | `stage` | the run’s stage key, or `"unknown"` | | `agent` | the executing agent’s principal id (`prn_…`), or `"unknown"` | | `agent_ref` | SuperPipeline’s own id for the agent, or `"unknown"` | | `state` | the run’s outcome when SuperPipeline reports one, else its status; `"unknown"` when neither is known | | `started_at` | RFC 3339 time in UTC, or `"unknown"` | | `ended_at` | RFC 3339 time; `null` while the run has not ended; `"unknown"` when no source could say | When SuperPipeline does not answer, `card.id`, `state`, `agent` and `started_at` come from the AgentPod ledger where it has them: the card id, the dispatch outcome, the first attempt’s agent principal, and the earliest attempt’s start. ### `attempts[]` [Section titled “attempts\[\]”](#attempts) | Field | Value | | ------------- | ------------------------------------------------------------------------------------------------------------------------ | | `id` | `attempt_` | | `station` | the AgentPod station, or `"unknown"` | | `state` | the attempt’s state as AgentPod reports it, or `"unknown"` | | `session_id` | the session the attempt ran in, or `"unknown"` | | `seq_from` | number: the first event of the session the attempt covers | | `seq_to` | number, or `null` while the attempt is open | | `started_at` | RFC 3339 time, or `"unknown"` | | `ended_at` | RFC 3339 time, or `null` | | `fingerprint` | `digest`, `harness`, `harness_version`, `model`, `profile`, `skill_release`, `reported_by`: each a string or `"unknown"` | | `span_count` | number of spans whose `attempt.id` is this attempt’s id, or `"unknown"` when the traces source did not answer | ### `trace` [Section titled “trace”](#trace) | Field | Value | | ----------- | ------------------------------------------------------------------------------------------------------------------------- | | `trace_ids` | array of trace ids, ordered by each trace’s earliest span; empty when none were found or the traces source did not answer | | `status` | `joined`, `partial`, `none` or `unknown` ([Concepts](/concepts/#trace-status)) | | `sampled` | `true` when spans were found, otherwise `"unknown"` | ### `errors[]` [Section titled “errors\[\]”](#errors) Up to 50 error-level log lines, oldest first: | Field | Value | | ---------- | -------------------------------- | | `service` | the service name, or `"unknown"` | | `message` | the log message | | `trace_id` | the line’s trace id, or `""` | | `at` | RFC 3339 time, or `"unknown"` | ### `verdicts[]` [Section titled “verdicts\[\]”](#verdicts) Gates from SuperPipeline first, in SuperPipeline’s order, then verdicts recorded in SuperWitness, oldest first. Of each recorded chain, only the latest verdict appears. | Field | Gate | Recorded verdict | | ------------ | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ | | `id` | SuperPipeline’s gate id | `vrd_…` | | `kind` | `gate` | `grader`, `review`, `eval` or `calibration` | | `source` | `superpipeline` | `superwitness` | | `subject` | the run’s ref | the verdict’s `subject_ref`: the run’s ref, or an attempt id | | `value` | `{"decision": ""}`, or `{"decision": null}` when not decided | as recorded | | `judge` | the decider’s principal id; SuperPipeline’s own id for a decider it has not mapped; or `"unknown"` | the principal that recorded it | | `judge_kind` | `human`, `agent`, `grader` or `unknown` | `human`, `agent` or `grader` | | `standard` | `stage:` | as recorded | | `status` | `pending`, `resolved`, `cancelled` or `unknown` | left out | | `comment` | left out | the comment; left out when empty | | `at` | when it was resolved or cancelled, else when it was created; `"unknown"` if a resolved gate has no time | when it was recorded | | `run_id` | the run the gate was raised on, or `"unknown"` | left out | | `supersedes` | left out | the verdict it corrects; left out when none | A gate SuperPipeline reports as resolved but without a decision shows as `pending`. ### `cost` [Section titled “cost”](#cost) | Field | Value | | ------------------------------- | ------------------------------------- | | `status` | `reported`, `unreported` or `unknown` | | `input_tokens`, `output_tokens` | number, or `null` | | `usd` | number, or `null` | The three numbers can be non-null only when `status` is `reported`. ### `sources` [Section titled “sources”](#sources) One key for each source, always all six: `superpipeline`, `agentpod`, `traces`, `logs`, `errors` and `verdicts`. Each value is `ok`, `unavailable`, `timeout`, `not_found` or `unauthorized`. ### `links` [Section titled “links”](#links) `self`, `spans` and `logs`: the paths of this document and of its [paged spans and logs](/use/read-a-run/#spans-one-page-at-a-time). ### Example [Section titled “Example”](#example) The development run that fake mode serves, shortened: ```json { "run": {"ref": "superpipeline:brd_01/run_01", "card": {"id": "crd_01", "title": "Write the release note"}, "stage": "draft", "agent": "prn_agent01", "agent_ref": "agt_01", "state": "completed", "started_at": "2026-10-04T10:00:00Z", "ended_at": "2026-10-04T10:05:00Z"}, "attempts": [{"id": "attempt_01", "station": "stn_01", "state": "completed", "session_id": "acps_01", "seq_from": 1, "seq_to": 9, "started_at": "2026-10-04T10:00:02Z", "ended_at": "2026-10-04T10:04:58Z", "fingerprint": {"digest": "sha256:c1a8…3426", "harness": "hermes", "harness_version": "unknown", "model": "unknown", "profile": "default", "skill_release": "none", "reported_by": "hub"}, "span_count": 2}], "trace": {"trace_ids": ["4bf92f3577b34da6a3ce929d0e0e4736", "5bf92f3577b34da6a3ce929d0e0e4737"], "status": "partial", "sampled": true}, "errors": [{"service": "agentpod-hub", "message": "bridge heartbeat failed: 502 from superpipeline", "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "at": "2026-10-04T10:02:00Z"}], "verdicts": [{"id": "gate_01", "kind": "gate", "source": "superpipeline", "subject": "superpipeline:brd_01/run_01", "value": {"decision": "rejected"}, "judge": "prn_human01", "judge_kind": "human", "standard": "stage:draft", "status": "resolved", "at": "2026-10-04T10:20:00Z", "run_id": "run_01"}], "cost": {"status": "unreported", "input_tokens": null, "output_tokens": null, "usd": null}, "log_count": 2, "sources": {"agentpod": "ok", "errors": "ok", "logs": "ok", "superpipeline": "ok", "traces": "ok", "verdicts": "ok"}, "links": {"self": "/v1/runs/superpipeline/brd_01/run_01", "spans": "/v1/runs/superpipeline/brd_01/run_01/spans", "logs": "/v1/runs/superpipeline/brd_01/run_01/logs"} } ``` ## Verdicts [Section titled “Verdicts”](#verdicts-1) What `POST /v1/verdicts` and the `record_verdict` tool return: | Field | Value | | ----------------- | -------------------------------------------------------------------------------- | | `id` | `vrd_` and 20 hex characters | | `idempotency_key` | as sent | | `kind` | `grader`, `review`, `eval` or `calibration` | | `subject_kind` | `run`, `attempt` or `eval_case_run` | | `subject_ref` | `superpipeline:/`, `attempt_` or `case:@sha256:<64 hex>` | | `judge` | the caller’s principal id | | `judge_kind` | `human`, `agent` or `grader`, from the caller’s principal kind | | `standard` | `rubric:@`, `stage:` or `case:` | | `value` | an object with exactly one of `decision`, `score`, `label` or `text` | | `comment` | a string, `""` when none | | `evidence_refs` | an array of span ids and `{session_id, seq_from, seq_to}` ranges, `[]` when none | | `supersedes` | the id of the verdict it corrects, or `null` | | `created_at` | RFC 3339 time in UTC | The database allows a fourth judge kind, `rule`, which no principal records in this release. [Verdicts](/use/verdicts/) has the rules for each field. ## Run reports [Section titled “Run reports”](#run-reports) What a source sends to `POST /v1/runs` is described by a JSON Schema at [`/schemas/run-report.schema.json`](/schemas/run-report.schema.json), also in the repository at `internal/contracts/run-report.schema.json`. A source can vendor it and validate its own output in its tests. [Run registry API](/build/run-registry/) explains each field. ## Spans [Section titled “Spans”](#spans) The span fields that matter to a run document are `run.id`, `attempt.id` and the root span name `dispatch`. [Sending telemetry](/use/telemetry/#spans) covers them and the trace shape SuperWitness expects. ## What SuperWitness reads, pinned [Section titled “What SuperWitness reads, pinned”](#what-superwitness-reads-pinned) SuperWitness keeps a recorded example of each payload it reads from another product, in `internal/contracts/` in its repository. Its adapter tests decode them, and fake mode serves the development run from them: | File | The answer from | | ----------------------------- | ---------------------------------------------------------------------------------------- | | `superpipeline_evidence.json` | SuperPipeline’s run evidence route: run, card, gates and usage | | `hub_evidence_run.json` | the AgentPod hub’s run evidence route: the ledger, attempts and fingerprints | | `hub_evidence_attempt.json` | the hub’s attempt route: which run an attempt belongs to | | `hub_principal.json` | the hub’s principal route: a principal’s id and kind | | `jaeger_trace_hub.json` | VictoriaTraces’ Jaeger API: a `dispatch`, `attempt` and `turn` trace from `agentpod-hub` | | `jaeger_trace_workers.json` | VictoriaTraces’ Jaeger API: a span from `superpipeline-api` carrying `run.id` | [With AgentPod and SuperPipeline](/use/with-agentpod-and-superpipeline/) lists the routes these come from. Together, the recorded payloads and adapter tests keep both sides of the join reviewable: you can see what SuperWitness promises out and what it expects in without reverse-engineering either from a running service. # Licences > The licences of SuperWitness and of everything it links or runs beside. SuperWitness is released under the [MIT licence](https://github.com/SuperJackfruitLabs/superwitness/blob/main/LICENSE). The short answer is permissive; the useful answer is knowing what ships in the tarball, what merely helped build it, and what runs beside it. We separate those three below. ## What ships [Section titled “What ships”](#what-ships) A release tarball holds: * the `superwitness` binary, with the app built into it; * `LICENSE` and `NOTICE`; * `licenses/fonts/`, the fonts’ licences; * `deploy/superwitness.service` and `deploy/env.example`. The binary links Go modules and embeds a web app built with Svelte and SvelteKit, with eight bundled fonts. Each keeps its own licence, and the [`NOTICE`](https://github.com/SuperJackfruitLabs/superwitness/blob/main/NOTICE) file in every release credits them. In summary: | Component | Licence | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | chi, pgx and its jackc helpers, goose and mfridman/interpolate, golang-jwt, google/jsonschema-go, segmentio/encoding, cespare/xxhash, felixge/httpsnoop, go.uber.org/multierr, cenkalti/backoff | MIT | | the MCP Go SDK | MIT for existing code, Apache-2.0 for new contributions | | segmentio/asm | MIT-0 (MIT No Attribution) | | sethvargo/go-retry, go-logr/logr and stdr, the OpenTelemetry Go API, SDK, exporters, otelhttp instrumentation and OTLP protobufs, gRPC and genproto | Apache-2.0 | | portions of the OpenTelemetry Go modules and otelhttp, from the Go Authors | BSD-3-Clause | | yosida95/uritemplate, google/uuid, grpc-gateway, Go protobuf, and golang.org/x sync, net, oauth2, sys, text and time | BSD-3-Clause | | svelte and the SvelteKit client runtime, in the embedded app | MIT | | `@superjackfruitlabs/vibekit-ui`, the suite’s shared design kit, vendored in `web/vendor/` | by the same authors; its package declares no open-source licence (`UNLICENSED`) | | Archivo, Atkinson Hyperlegible Next, Azeret Mono, Bricolage Grotesque, Figtree, Geist, Martian Mono and Newsreader, bundled in the app through vibekit (from the `@fontsource` packages) | SIL Open Font License 1.1 | The list covers what the binary links, not test-only modules. Build and test tools are not bundled: the app’s development dependencies, including caniuse-lite (CC-BY-4.0, used by the browser-list tooling), @playwright/test (Apache-2.0) and jsdom (MIT), stay out of the release. The same goes for the npm packages that build superwitness.dev and this documentation site: they are not part of any release. ## The allowlist [Section titled “The allowlist”](#the-allowlist) SuperWitness’s CI checks the licence of every Go module it builds with, using `go-licenses`, against an allowlist of permissive licences: * MIT * Apache-2.0 * BSD-2-Clause * BSD-3-Clause * ISC `go-licenses` does not classify `segmentio/asm`, which the MCP SDK pulls in. Its licence, MIT-0, was reviewed by hand: it is permissive, and the allowlist is unchanged. No AGPL code is linked or shipped, so Grafana, Loki, Tempo and Mimir are not used, and neither is anything under a source-available licence such as ELv2, BSL, FSL or SSPL. ## What runs beside it [Section titled “What runs beside it”](#what-runs-beside-it) The telemetry engines are not part of SuperWitness. They are separate upstream programs that you install and run as released, and SuperWitness only calls their published APIs over HTTP. This release queries VictoriaTraces and VictoriaLogs, fed by the OpenTelemetry Collector. Each is under its own licence: | Project | Licence | | ------------------------------------------------------------------------------------ | ---------- | | [VictoriaTraces](https://github.com/VictoriaMetrics/VictoriaTraces) | Apache-2.0 | | [VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaLogs) | Apache-2.0 | | [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) | Apache-2.0 | SuperWitness is designed to sit beside [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics) and [Perses](https://github.com/perses/perses) (both Apache-2.0) the same way, but this release does not use them. That boundary keeps the release inventory honest: a nearby service is not a linked dependency simply because the two happen to share a host. # MCP tools > The MCP server for agents, with tools for runs, spans, logs, transcripts, the run registry and verdicts; the by-attempt lookup stays HTTP-only. SuperWitness serves an MCP server at `/mcp` on its own address, over MCP’s streamable HTTP transport. Point an agent at it and the agent gets the same run, span, log, transcript, registry and verdict operations as the HTTP client, without a second model to learn. The server is stateless: every request stands alone, with no session to open or keep. It takes the same bearer token as the [HTTP API](/build/api/#authentication), and every tool acts as the principal that token names. Point an MCP client at it with the token as a header, for example: ```json { "url": "https://superwitness.example.internal/mcp", "headers": { "Authorization": "Bearer " } } ``` With that endpoint and header in place, the client can discover all six tool schemas from the server. There is no MCP session to provision, renew or wonder about later. The server names itself `superwitness`, with the running release as its version. It has six tools, thin layers over the HTTP operations for runs, spans, logs, transcripts, the run registry and verdicts. Eight routes are HTTP-only and have no tool: `GET /v1/runs/by-attempt/{attempt}`, `POST /v1/runs`, the two rubric reads, the transcript step route, `GET /v1/me`, verdict history (`GET /v1/verdicts`) and `GET /v1/scopes`. ## The tools [Section titled “The tools”](#the-tools) ### `get_run` [Section titled “get_run”](#get_run) The RunDocument for one work run: attempts with fingerprints, joined traces, errors, gate and recorded verdicts, cost, and per-source status. The same as `GET /v1/runs/superpipeline/{board}/{run}`. | Argument | Required | Description | | ---------- | -------- | ------------------------------------------------------------------- | | `source` | no | the run’s source system; only `superpipeline`, which is the default | | `board_id` | yes | SuperPipeline board id, `brd_…` | | `run_id` | yes | SuperPipeline run id, `run_…` | Result: the [run document](/build/contracts/#the-run-document). ### `list_run_spans` [Section titled “list_run_spans”](#list_run_spans) One page of a run’s spans, oldest first. The same as `GET /v1/runs/superpipeline/{board}/{run}/spans`. | Argument | Required | Description | | ---------- | -------- | ------------------------------------ | | `source` | no | `superpipeline`, the default | | `board_id` | yes | SuperPipeline board id, `brd_…` | | `run_id` | yes | SuperPipeline run id, `run_…` | | `cursor` | no | `next_cursor` from the previous page | | `limit` | no | page size from 1 to 500; default 100 | Result: `{"spans": [...], "next_cursor": "…"}`, with `next_cursor` left out on the last page. ### `list_run_logs` [Section titled “list_run_logs”](#list_run_logs) One page of a run’s log lines, matched by `run.id` and by the run’s trace ids, oldest first. The same as `GET /v1/runs/superpipeline/{board}/{run}/logs`. | Argument | Required | Description | | ---------- | -------- | ------------------------------------------------- | | `source` | no | `superpipeline`, the default | | `board_id` | yes | SuperPipeline board id, `brd_…` | | `run_id` | yes | SuperPipeline run id, `run_…` | | `cursor` | no | `next_cursor` from the previous page | | `level` | no | `debug`, `info`, `warn` or `error`; empty for all | | `limit` | no | page size from 1 to 500; default 100 | Result: `{"logs": [...], "next_cursor": "…" or null, "trace_join": ""}`. ### `list_runs` [Section titled “list_runs”](#list_runs) Runs reported to the run registry, newest first, with per-status counts and each run’s latest verdict. The same as `GET /v1/runs` ([Run registry API](/build/run-registry/#listing-runs)). | Argument | Required | Description | | ---------------- | -------- | ---------------------------------------------------------------------------------------------------------- | | `source` | no | only runs from this source | | `scope` | no | only runs in this scope id | | `status` | no | a list of statuses | | `executor` | no | only runs this principal executed | | `stage_key` | no | only runs reported with this `stage_key`. On `main`, not yet released: v0.0.12’s tool has no such argument | | `since`, `until` | no | RFC 3339 bounds on the run’s start, or first sighting | | `needs_verdict` | no | `true` for finished runs no verdict names yet | | `cursor` | no | `next_cursor` from the previous page | | `limit` | no | page size from 1 to 200; default 50 | ### `record_verdict` [Section titled “record_verdict”](#record_verdict) Record a verdict as the calling principal. Append-only; to correct one, record a new verdict that supersedes it. The same as `POST /v1/verdicts`; [Verdicts](/use/verdicts/) has the rules. | Argument | Required | Description | | ----------------- | -------- | --------------------------------------------------------------------------------------------------- | | `idempotency_key` | yes | a key you choose; a retry with the same key returns the original verdict | | `kind` | no | `grader`, `review`, `eval` or `calibration`; defaults to `review` for humans and `grader` otherwise | | `subject_kind` | yes | `run`, `attempt` or `eval_case_run` | | `subject_ref` | yes | `superpipeline:/`, `attempt_`, or `case:@sha256:` | | `standard` | yes | `rubric:@`, `stage:` or `case:` | | `value` | yes | exactly one of `{decision}`, `{score 0..1}`, `{label}`, `{text}` | | `comment` | no | free text, at most 10000 characters | | `evidence_refs` | no | span ids or `{session_id, seq_from, seq_to}` | | `supersedes` | no | the id of your earlier verdict this one corrects | Result: the [verdict](/build/contracts/#verdicts), whether it was just recorded or returned for a retry. Unlike the HTTP body, the tool has no `judge` or `judge_kind` argument: the judge is always the caller. ### `get_transcript` [Section titled “get_transcript”](#get_transcript) One page (up to 200 steps) of an attempt’s transcript, redacted by AgentPod. The same as `GET /v1/runs/superpipeline/{board}/{run}/transcript`. The token must be granted `transcripts:read`; an MCP call is never a browser session. | Argument | Required | Description | | ------------ | --------------------------------- | ----------------------------------------------------------------------------------- | | `board_id` | yes | SuperPipeline board id, `brd_…` | | `run_id` | yes | SuperPipeline run id, `run_…` | | `attempt_id` | when the run has several attempts | the attempt to read, `attempt_…` | | `seq_from` | no | first session seq; default the attempt’s first | | `seq_to` | no | last session seq; default the attempt’s last | | `cursor` | no | `next_cursor` from the previous page; send the same `seq_from` and `seq_to` with it | Result: the transcript page ([Transcripts](/use/transcripts/#the-api)). ## Results and errors [Section titled “Results and errors”](#results-and-errors) A tool’s result is the same JSON the HTTP route returns, twice over: as text content, and as structured content. A refusal is a tool result marked as an error (`isError`), whose text content is the same error object the HTTP API returns, with the same [codes](/build/api/#every-code): ```json {"error":{"code":"run_not_found","message":"neither superpipeline nor the hub ledger knows superpipeline:brd_01/run_02"}} ``` There is one code the HTTP API has no use for: `invalid_source`, when `source` is anything but `superpipeline`. Each tool’s input schema is published with it, with the descriptions above. Arguments are checked against it before the tool runs: a missing required argument, or one the schema does not list, is also an error result, whose text is a plain validation message rather than an error object. Your agent therefore gets the same data contract and stable error codes as an HTTP client, with input mistakes stopped before they reach the operation. # Run registry API > How a source reports its runs to SuperWitness, and how runs are listed back. SuperWitness keeps its own list of runs so you can browse work without asking every source for a fresh inventory. Each product that runs work, a **source**, reports its runs as they start, change and finish, much as it sends telemetry to an OTLP endpoint. SuperWitness never asks a product to list its runs. If a source has not reported a run, the list cannot invent it—but the run’s own page still works when you have its link. ## Reporting runs [Section titled “Reporting runs”](#reporting-runs) `POST /v1/runs` takes one report, or a batch of 1 to 100: ```json { "source": "superpipeline", "external_ref": "brd_01/run_01", "scope": {"id": "brd_01", "name": "Press"}, "title": "Draft the release note", "stage_key": "draft", "card_id": "crd_01", "executor": {"id": "prn_…", "name": "drafter"}, "status": "running", "source_status": "in_progress", "started_at": "2026-10-06T09:00:00Z", "ended_at": null, "reported_at": "2026-10-06T09:00:01Z" } ``` ```json { "runs": [ { … }, { … } ] } ``` | Field | Required | Rule | | ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `source` | yes | lowercase letters, digits and `-`, starting with a letter, at most 32 | | `external_ref` | yes | the source’s own reference for the run, 1 to 256 characters | | `scope` | no | `{"id", "name"}`: where the run belongs, such as a board. `id` is required, 1 to 128 characters; `name`, 1 to 200, may be left out. Leave `scope` out rather than send `null` | | `title` | no | 1 to 200 characters. Leave it out rather than send `null` or `""` | | `stage_key` | no | the source’s stage the run belongs to, such as a board stage’s key, 1 to 128 characters. Leave it out rather than send `null` or `""` | | `card_id` | no | the source’s work item the run is for, such as a board card’s id, 1 to 128 characters. Leave it out rather than send `null` or `""` | | `executor` | no | `{"id", "name"}`: who ran it. `name` is required, 1 to 200 characters; `id`, when known, is a hub principal id (`prn_` and 20 lowercase hex digits). Leave `executor` out rather than send `null` | | `status` | yes | `queued`, `running`, `waiting`, `succeeded`, `failed` or `cancelled` | | `source_status` | yes | the source’s own word for the state, 1 to 64 characters | | `started_at`, `ended_at` | no | RFC 3339 timestamps, or `null` when not yet known | | `reported_at` | yes | RFC 3339: when the source produced this report | On main, not yet released `stage_key` and `card_id`, and the `stage_key` filter on [listing runs](#listing-runs), are on `main` and not in v0.0.12, the latest release. A v0.0.12 server refuses a report that carries either field (422 `invalid_report`, unknown field) and ignores a `stage_key` query. Leave both fields out when reporting to a release older than the one that ships them. SuperPipeline sends them only when its `SUPERWITNESS_REPORT_STAGE` is set. Unknown fields are refused, and field names are matched exactly. Timestamps are RFC 3339 `date-time` strings of at most 64 characters, kept to the microsecond. The format is strict: the fraction separator is a dot, not a comma, and an offset hour is at most 23. The body may be at most 256 KiB. ### Who may report [Section titled “Who may report”](#who-may-report) A reporter is an AgentPod **service** principal whose token carries the `runs:write` scope and SuperWitness’s audience (`SW_PUBLIC_URL`). SuperWitness also binds each reporter to one source with `SW_RUN_SOURCES`, for example `SW_RUN_SOURCES=prn_reporter01=superpipeline`; a report for any other source is refused. Session cookies are never accepted here. ### Newer reports win [Section titled “Newer reports win”](#newer-reports-win) A report updates its run only when its `reported_at` is **strictly newer** than the one stored. An equal or older report changes nothing and answers `"applied": false`, so retries and out-of-order deliveries cannot move a run backwards. A `reported_at` more than 5 minutes ahead of SuperWitness’s clock is refused, so that one report from a fast clock cannot freeze a run. ### The answer [Section titled “The answer”](#the-answer) ```json {"results": [{"external_ref": "brd_01/run_01", "applied": true}]} ``` One result per report, in the order sent. A batch is checked as a whole: if one report is invalid, nothing is written, and the error names its position with `index`. A single report’s errors carry no `index`; a batch from a reporter that is not bound to the source gets `index` 0: ```json {"error": {"code": "invalid_report", "message": "runs[3].status: one of queued, …", "index": 3}} ``` | Status | Code | When | Retry? | | ------ | ---------------------------- | ------------------------------------------------------------ | ------ | | 400 | `invalid_json` | the body is not one report or `{"runs": [...]}` of 1 to 100 | no | | 401 | `unauthenticated` | no token, or one not valid for this service | no | | 403 | `service_principal_required` | the token’s principal is not a service | no | | 403 | `insufficient_scope` | the token lacks `runs:write` | no | | 403 | `source_not_allowed` | the reporter is not bound to this report’s source | no | | 413 | `body_too_large` | the body is over 256 KiB | no | | 422 | `invalid_report` | a report breaks a rule above | no | | 503 | `store_unavailable` | SuperWitness’s database is unavailable; `Retry-After` is set | yes | A reporter should retry 5xx answers and network errors with backoff, and set any other 4xx aside: sending it again will get the same answer. Each report refused after its token is accepted is logged as `runs.report_rejected`, with the caller’s `principal`, the error `code` and, for a batch, the `index`. An `applied: true` result means the registry now reflects that report. An older retry returns `applied: false`, which is still a successful receipt: the registry kept the newer truth. ### The schema [Section titled “The schema”](#the-schema) The body is described by a JSON Schema (draft 2020-12), published at [`/schemas/run-report.schema.json`](/schemas/run-report.schema.json) and kept in the repository at `internal/contracts/run-report.schema.json`. SuperWitness’s own tests run the same bodies through the schema and through its handler and require the same answer from both. SuperWitness keeps four rules beyond the schema: `reported_at` may be at most 5 minutes ahead of its clock, no string may contain NUL (`\u0000`), a timestamp may not be a leap second, and a timestamp’s year in UTC must be 0 to 9999. ## Listing runs [Section titled “Listing runs”](#listing-runs) `GET /v1/runs` lists runs newest first: by `started_at`, or by when SuperWitness first heard of the run when it has not started, then by `source` and `external_ref`. | Query | Meaning | | ---------------- | --------------------------------------------------------------------------------------------------------------------- | | `source` | only runs from this source | | `scope` | only runs in this scope id | | `status` | only runs with this status; repeat it for several | | `executor` | only runs this principal executed | | `stage_key` | only runs reported with this `stage_key`; with `scope`, one stage’s history on one board. On `main`, not yet released | | `since`, `until` | RFC 3339; runs whose start (or first sighting) is at or after `since` and before `until` | | `needs_verdict` | `true`: finished runs (`succeeded`, `failed`, `cancelled`) that no verdict names yet | | `cursor` | `next_cursor` from the previous page | | `limit` | page size, 1 or more (0 or less is 400 `invalid_limit`); default 50, and anything above 200 is 200 | ```json { "runs": [ { "source": "superpipeline", "external_ref": "brd_01/run_01", "ref": "superpipeline:brd_01/run_01", "scope": {"id": "brd_01", "name": "Press"}, "title": "Draft the release note", "stage_key": "draft", "card_id": "crd_01", "executor": {"id": "prn_…", "name": "drafter"}, "status": "succeeded", "source_status": "done", "started_at": "2026-10-06T09:00:00Z", "ended_at": "2026-10-06T09:05:00Z", "reported_at": "2026-10-06T09:05:01Z", "first_seen_at": "2026-10-06T09:00:02Z", "updated_at": "2026-10-06T09:05:02Z", "latest_verdict": { "id": "vrd_01", "judge": "prn_human01", "judge_kind": "human", "standard": "rubric:press@1", "value": {"decision": "pass"}, "created_at": "2026-10-06T09:30:00Z" } } ], "next_cursor": null, "counts": {"queued": 0, "running": 2, "waiting": 1, "succeeded": 14, "failed": 3, "cancelled": 0} } ``` * `ref` is the run as `:`. A verdict’s run subject is accepted only as `superpipeline:/`, so for a SuperPipeline run record a verdict with `"subject_kind": "run"` and this as `subject_ref`. A run from any other source, or with an `external_ref` that is not `/`, cannot be a verdict’s subject yet: it is refused 400 `invalid_subject`. * `stage_key` and `card_id` are as last reported, or `null` when the newest report left them out. On `main`, not yet released: v0.0.12 answers neither. * `latest_verdict` is the newest verdict on the run that no other verdict supersedes, or `null`. Verdicts on the run’s attempts do not count. * `counts` totals every status for the same filter without `status`, so a view can show its tabs. * `next_cursor` is `null` on the last page. A cursor stays valid while new runs arrive: the next page continues where the last one ended. Errors: 400 `invalid_source`, `invalid_status`, `invalid_time`, `invalid_cursor`, `invalid_limit` or `invalid_needs_verdict`, and 503 `store_unavailable`. Any principal whose token carries SuperWitness’s audience may list runs, as with every other read. The MCP tool `list_runs` takes the same filters as arguments (`status` as a list) and answers the same body ([MCP tools](/build/mcp/)). Follow `next_cursor` and you can walk the matching history in its documented order, while `counts` keeps the status tabs honest across every page rather than just the slice in front of you. # Concepts > Runs, attempts, configuration fingerprints, verdicts, sources and their statuses, and why SuperWitness says unknown rather than 0. The run document has a small vocabulary, but each word carries a boundary. We will walk through the field names and values SuperWitness returns, then finish with the distinction that matters most: `unknown` is not 0. You will see all of them in `GET /v1/runs/superpipeline/{board}/{run}`. ## Run [Section titled “Run”](#run) A **run** is one piece of work on a SuperPipeline board. It is addressed by its board and run ids, and the document names it in `run.ref` as `superpipeline:/`, for example `superpipeline:brd_01/run_01`. Board and run ids are letters, digits, `_` and `-`, starting with a letter or digit, up to 128 characters. The `run` section carries the card it belongs to (`card.id`, `card.title`), its `stage`, its `state`, `started_at` and `ended_at`, and two agent fields: `agent` is the agent’s principal id and `agent_ref` is SuperPipeline’s own id for it. `ended_at` is `null` while the run has not ended, and `"unknown"` when no source could say. SuperWitness owns none of this. It reads the run from SuperPipeline every time the document is asked for. When SuperPipeline cannot answer, the document falls back to what AgentPod’s ledger knows about the run: the card id, the state, the agent and the start time. When neither knows the run, the request answers 404 `run_not_found`. ## Attempt [Section titled “Attempt”](#attempt) An **attempt** is one execution of the run by an agent on AgentPod. A run can have several. Each entry in `attempts` has: | Field | Meaning | | ---------------------------------- | ----------------------------------------------------------------------------------------------- | | `id` | `attempt_` | | `station` | the AgentPod station it ran on | | `state` | the attempt’s state as AgentPod reports it | | `session_id`, `seq_from`, `seq_to` | the session and the range of its events the attempt covers; `seq_to` is `null` while it is open | | `started_at`, `ended_at` | when it ran | | `fingerprint` | the configuration that ran it (below) | | `span_count` | spans carrying this attempt’s id as `attempt.id`, or `"unknown"` | `GET /v1/runs/by-attempt/{attempt}` redirects (302) from an attempt to its run’s document. ## Configuration fingerprint [Section titled “Configuration fingerprint”](#configuration-fingerprint) Every attempt carries the **fingerprint** of the configuration that ran it: | Field | Meaning | | ---------------------------- | ---------------------------------------------------------------------------------------------------------- | | `digest` | one `sha256:` digest of the configuration, so two attempts with the same digest ran the same configuration | | `harness`, `harness_version` | the agent harness and its version | | `model` | the model | | `profile` | the harness profile | | `skill_release` | the skill release installed | | `reported_by` | which component reported the fingerprint | The fingerprint is reported with the attempt by AgentPod; SuperWitness shows it as reported. A field nobody reported reads `"unknown"`. It is what lets you ask whether a change of configuration changed the results. ## Verdict [Section titled “Verdict”](#verdict) A **verdict** is a judgement about a subject, against a named standard. SuperWitness’s own database holds verdicts, append-only. The run document shows two kinds side by side in `verdicts`: * **Gates** from SuperPipeline: `kind` is `gate`, `source` is `superpipeline` and `standard` is `stage:`. A gate’s `status` is `pending`, `resolved` or `cancelled`, or `unknown` when SuperPipeline reports a status this release does not know; a decided gate’s value is `{"decision": …}`. SuperWitness shows gates and never stores them. * **Recorded verdicts**, posted to `POST /v1/verdicts` or the `record_verdict` MCP tool. Their `kind` is `grader`, `review`, `eval` or `calibration`. Left out, it is `review` for a human judge and `grader` for anyone else. Every verdict has: * **A subject.** Its `subject_kind` is one of: * `run`, with a reference `superpipeline:/`; * `attempt`, with a reference `attempt_`; * `eval_case_run`, with a reference `case:@sha256:`. * **A value**: an object with exactly one of `decision` (text), `score` (a number from 0 to 1), `label` (text) or `text`. * **A judge and a judge kind.** The judge is the principal that recorded it. Its `judge_kind` is taken from that principal’s record, never from the request: a human principal judges as `human`, an agent as `agent`, a service as `grader`. The model has a fourth kind, `rule`; no principal records one in this release. Only a human may judge a run or attempt they executed; any other judge that executed it is refused. * **A standard**, always versioned or keyed: `rubric:@` for a rubric that exists in SuperWitness, `stage:` for a pipeline stage, or `case:` for an eval case. A verdict is never edited. To correct one, its judge records a new verdict on the same subject and standard that names the old one in `supersedes`. The run document shows only the latest verdict in each chain. ## Sources and their statuses [Section titled “Sources and their statuses”](#sources-and-their-statuses) The run document is assembled at read time from **sources**, and `sources` says how each one answered: | Source | What it supplies | | --------------- | --------------------------------------------------- | | `superpipeline` | the run, its card, stage, state, cost and gates | | `agentpod` | the attempts and their fingerprints | | `traces` | the run’s spans and trace ids, from VictoriaTraces | | `logs` | the run’s log count, from VictoriaLogs | | `errors` | the run’s error-level log lines, from VictoriaLogs | | `verdicts` | recorded verdicts, from SuperWitness’s own database | Each source’s status is one of: | Status | Meaning | | -------------- | --------------------------------------------------------------- | | `ok` | it answered | | `unavailable` | it could not be reached or failed | | `timeout` | it did not answer in time (`SW_SOURCE_TIMEOUT`, 2 s by default) | | `not_found` | it answered and does not know the run | | `unauthorized` | it refused SuperWitness’s credentials | A source that fails never fails the document. The rest of it still renders. The counts and text fields that source would have supplied read `"unknown"`, and the lists it would have filled (`attempts`, `errors`, `verdicts`, `trace.trace_ids`) come back empty; `sources` says why. ## Unknown is not 0 [Section titled “Unknown is not 0”](#unknown-is-not-0) A count in the run document, such as `log_count` or an attempt’s `span_count`, is either a number a source supplied or the string `"unknown"` when it could not. It is never a silent 0: a 0 means a source answered and found none. The same holds for `trace.sampled`, which is `true` when spans were found and `"unknown"` otherwise, and for text fields, which read `"unknown"` rather than being empty. Cost follows the same rule. `cost.status` is `reported` when SuperPipeline reported token counts and spend, `unreported` when SuperPipeline answered without them, and `unknown` when it did not answer. ## Trace status [Section titled “Trace status”](#trace-status) `trace.status` says how well the run’s spans join up: | Status | Meaning | | --------- | --------------------------------------------------------------------------- | | `joined` | the spans found share one trace id, and it has a root span named `dispatch` | | `partial` | spans were found, but not in that shape | | `none` | the traces source answered and found no spans for the run | | `unknown` | the traces source did not answer | `none` is not an error. Telemetry is sampled and short-lived, so missing spans are not missing evidence: attempts, gates and verdicts still stand. Once you keep that boundary in view, a sparse run document stays honest instead of merely looking tidy. # Install > Install SuperWitness on one Linux host with systemd, next to Postgres, VictoriaTraces, VictoriaLogs and the OpenTelemetry Collector. We are going to put the whole stack on one Linux host: the SuperWitness binary as a systemd service, its Postgres database, the two telemetry engines it queries, and the collector that feeds them. Every network address we set below is on loopback. You will finish with a working local service, not an accidental public one; nothing configured here is reachable from outside the host until you [choose to expose it](#configure). To look around first without the other products, skip to [Try it without the other products](#try-it-without-the-other-products). ## Requirements [Section titled “Requirements”](#requirements) * **Linux on amd64 or arm64, with systemd.** Releases ship a tarball for each. * **Postgres.** SuperWitness’s own tests run against Postgres 17. Postgres 15 and later need no schema statements beyond the two blocks below; 14 or older needs [two more](#postgres-14-or-older). * **VictoriaTraces** and **VictoriaLogs**, which hold the telemetry SuperWitness reads. * **The OpenTelemetry Collector**, which receives OTLP from your products and writes it to both engines. The long-running server needs no data directory: it reads over HTTP and writes only to Postgres. The optional canary is separate and writes its tracker to the path you pass with `--results`. ## Quick install of the binary [Section titled “Quick install of the binary”](#quick-install-of-the-binary) ```sh curl -fsSL https://github.com/SuperJackfruitLabs/superwitness/releases/latest/download/install.sh | bash ``` This verifies the download against `SHA256SUMS` and installs `superwitness` and the alias `suwi` into `~/.local/bin` (`BIN_DIR=` changes it, `VERSION=` pins a release). It installs no service and starts nothing; the steps below still set up the server. `superwitness skill` prints an operations skill for agents (what each command does and which ones change data); `suwi skill install` writes it to `~/.claude/skills`, and `SKILL_DIR=` on the installer does the same. Needs v0.0.7 or later. You now have the binary and its checksum verified. The server is still off, which is exactly where we want it while we prepare its database and sources. The script and the macOS builds are published from v0.0.7 on; v0.0.6 and earlier have neither. ## Download and verify [Section titled “Download and verify”](#download-and-verify) Each release has a tarball per architecture and a `SHA256SUMS` file covering them: ```sh curl -LO https://github.com/SuperJackfruitLabs/superwitness/releases/download/v0.0.12/superwitness_0.0.12_linux_amd64.tar.gz curl -LO https://github.com/SuperJackfruitLabs/superwitness/releases/download/v0.0.12/SHA256SUMS sha256sum --ignore-missing -c SHA256SUMS tar xzf superwitness_0.0.12_linux_amd64.tar.gz ``` On arm64, replace `amd64` with `arm64`. The tarball holds the binary, `LICENSE`, `NOTICE`, `licenses/` (the licences of the eight bundled fonts), `deploy/superwitness.service` and `deploy/env.example`. Install the binary: ```sh sudo install -m 0755 superwitness_0.0.12_linux_amd64/superwitness /usr/local/bin/ superwitness version ``` The version output tells you what your shell will run, so a `PATH` that still points at another installation cannot slip past unnoticed. ## Postgres roles [Section titled “Postgres roles”](#postgres-roles) Verdicts are append-only: triggers refuse `UPDATE`, `DELETE` and `TRUNCATE` on the `verdicts` and `rubrics` tables. A table’s owner can disable or drop those triggers, so the running service must not own the tables. SuperWitness therefore uses two roles: * **`superwitness_owner`** owns the database and the tables. SuperWitness connects as it, through `SW_MIGRATE_DATABASE_URL`, only to run migrations. * **`superwitness_app`** is the runtime role, through `SW_DATABASE_URL`. It can insert and read verdicts and rubrics, insert, read and update registry runs, and keep app sign-in sessions: the [runtime grants](#migrate) below, and nothing else. Run this as the Postgres superuser (for example `sudo -u postgres psql`) before the first start. Passwords, or peer or certificate authentication, are yours to choose; set them now, for example with `\password superwitness_owner` and `\password superwitness_app` in `psql`. ```sql CREATE ROLE superwitness_owner LOGIN; CREATE ROLE superwitness_app LOGIN; CREATE DATABASE superwitness OWNER superwitness_owner; REVOKE ALL ON DATABASE superwitness FROM PUBLIC; GRANT CONNECT ON DATABASE superwitness TO superwitness_app; ``` The database now has separate owner and runtime identities. The runtime role still has only `CONNECT`; it receives the table and schema grants after the owner runs the migrations. ### Postgres 14 or older [Section titled “Postgres 14 or older”](#postgres-14-or-older) On Postgres 14 or older, schema `public` belongs to the bootstrap superuser and every role may create tables in it. There, after the block above and before you [migrate](#migrate), run this as the superuser, connected to the `superwitness` database, so the owner role owns the schema and the runtime role cannot create tables: ```sql ALTER SCHEMA public OWNER TO superwitness_owner; REVOKE CREATE ON SCHEMA public FROM PUBLIC; ``` Postgres 15 and later need neither statement: `public` is owned by the database owner and `PUBLIC` has no `CREATE` on it. ## The telemetry engines [Section titled “The telemetry engines”](#the-telemetry-engines) Install VictoriaTraces and VictoriaLogs from their upstream releases; SuperWitness uses them as released and only through their published query APIs. Their own documentation covers installing them as services: [VictoriaTraces](https://docs.victoriametrics.com/victoriatraces/) and [VictoriaLogs](https://docs.victoriametrics.com/victorialogs/). For this guide, each needs a data directory and a loopback address: ```sh victoria-traces-prod -storageDataPath=/var/lib/victoria-traces -httpListenAddr=127.0.0.1:10428 victoria-logs-prod -storageDataPath=/var/lib/victoria-logs -httpListenAddr=127.0.0.1:9428 ``` Then run the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) with an OTLP receiver and one exporter per engine. The core distribution has both. A minimal configuration: ```yaml receivers: otlp: protocols: http: { endpoint: 127.0.0.1:4318 } exporters: otlphttp/traces: traces_endpoint: http://127.0.0.1:10428/insert/opentelemetry/v1/traces otlphttp/logs: logs_endpoint: http://127.0.0.1:9428/insert/opentelemetry/v1/logs service: pipelines: traces: { receivers: [otlp], exporters: [otlphttp/traces] } logs: { receivers: [otlp], exporters: [otlphttp/logs] } ``` Your products send OTLP over HTTP to `127.0.0.1:4318`. Which spans and log lines SuperWitness finds for a run is covered in [Sending telemetry](/use/telemetry/). At this point the collector has one local door in and two local paths out: traces to VictoriaTraces, logs to VictoriaLogs. SuperWitness will read both without becoming the warehouse. ## Configure [Section titled “Configure”](#configure) Create the service user and its settings file. The file is readable by root and the `superwitness` group only: ```sh sudo useradd -r -s /usr/sbin/nologin superwitness sudo install -d -m 0750 -o root -g superwitness /etc/superwitness sudo install -m 0640 -o root -g superwitness superwitness_0.0.12_linux_amd64/deploy/env.example /etc/superwitness/env ``` Edit `/etc/superwitness/env`. These are the settings `deploy/env.example` holds, with this guide’s addresses filled in: ```sh SW_LISTEN=127.0.0.1:8790 SW_PUBLIC_URL=https://superwitness.example.internal # Runtime role: the runtime grants below (verdicts, rubrics, runs, sessions); never the table owner. SW_DATABASE_URL=postgres://superwitness_app:@127.0.0.1:5432/superwitness?sslmode=disable # Owner role, used only for migrations (serve's background migrations and `superwitness migrate`). SW_MIGRATE_DATABASE_URL=postgres://superwitness_owner:@127.0.0.1:5432/superwitness?sslmode=disable SW_SUPERPIPELINE_URL=https://app.superpipeline.dev SW_HUB_URL=https://hub.agentpod.dev SW_HUB_CLIENT_ID=svc_REPLACE_ME SW_HUB_CLIENT_SECRET_FILE=/etc/superwitness/hub-client-secret SW_TRACES_URL=http://127.0.0.1:10428 SW_LOGS_URL=http://127.0.0.1:9428 # SW_TRACES_TOKEN_FILE=/etc/superwitness/traces-token # SW_LOGS_TOKEN_FILE=/etc/superwitness/logs-token SW_SOURCE_TIMEOUT=2s # SW_OTLP_ENDPOINT=http://127.0.0.1:4318 ``` * `SW_LISTEN` is the address SuperWitness binds. This guide keeps it on loopback; left unset it is `:8790`, every interface. To let other machines reach it, either bind it to a private or tailnet address of this host (`SW_LISTEN=:8790`) or keep it on loopback and put a reverse proxy in front. * `SW_PUBLIC_URL` is the address callers reach SuperWitness at. Tokens sent to it must name it as their audience. * `SW_SUPERPIPELINE_URL`, `SW_HUB_URL`, `SW_HUB_CLIENT_ID` and `SW_HUB_CLIENT_SECRET_FILE` are where SuperWitness reads runs and attempts, and the AgentPod service credential it signs in with. * Outside fake mode, `serve` refuses to start unless every one of these is set: `SW_DATABASE_URL`, `SW_PUBLIC_URL`, `SW_SUPERPIPELINE_URL`, `SW_HUB_URL`, `SW_HUB_CLIENT_ID`, `SW_HUB_CLIENT_SECRET_FILE`, `SW_TRACES_URL` and `SW_LOGS_URL`. Its error names each one missing. * `SW_OTLP_ENDPOINT` is optional: where SuperWitness sends its own traces and metrics. The collector configuration above has no metrics pipeline, so add one before setting it. The hub client secret goes in a file of its own, holding the secret alone. SuperWitness refuses to read it if group or others can: ```sh sudo install -m 0600 -o superwitness -g superwitness /dev/null /etc/superwitness/hub-client-secret sudoedit /etc/superwitness/hub-client-secret ``` The secret now sits in its own owner-readable file, ready for the service without becoming part of the settings file or a shell command. ## Migrate [Section titled “Migrate”](#migrate) Create the schema as the owner role. `superwitness migrate` reads only `SW_MIGRATE_DATABASE_URL` (else `SW_DATABASE_URL`), runs every pending migration once, and exits 0, or non-zero with the error. It gives up after 60 s if Postgres does not answer: ```sh sudo sh -c 'set -a; . /etc/superwitness/env; set +a; superwitness migrate' ``` Then grant the runtime role, connected to the `superwitness` database as `superwitness_owner` (for example `psql "postgres://superwitness_owner@127.0.0.1:5432/superwitness"`): ```sql GRANT USAGE ON SCHEMA public TO superwitness_app; GRANT SELECT, INSERT ON TABLE verdicts, rubrics TO superwitness_app; GRANT SELECT, INSERT, UPDATE ON TABLE runs TO superwitness_app; GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE sessions TO superwitness_app; ``` The schema is current and the runtime role has exactly the listed table and schema grants the service needs. That is the database boundary in place; later migrations that add a table must earn a new grant. That is the whole runtime grant set. The runtime role cannot update, delete, truncate, alter or drop `verdicts` and `rubrics` or their triggers, can update registry rows in `runs` but never delete them, and cannot create tables. Nothing is granted through `ALTER DEFAULT PRIVILEGES`, so a later migration that adds a table needs a grant of its own. When upgrading, run `superwitness migrate`, then, as the owner, only the grants your starting version lacks: * From 0.0.2: `GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE sessions TO superwitness_app;`. * From 0.0.1: both `GRANT SELECT, INSERT, UPDATE ON TABLE runs TO superwitness_app;` (added in 0.0.2) and the `sessions` grant above (added in 0.0.3). Until the `runs` grant is in place the run registry answers 503 `store_unavailable`; until the `sessions` grant is, sign-in shows the “Can’t reach the database” page. The settings file above keeps the owner’s DSN in the service’s environment, so the running process holds the owner password. To keep it out, leave `SW_MIGRATE_DATABASE_URL` only in your own environment for `superwitness migrate`, run that before each upgrade, and also run `GRANT SELECT ON goose_db_version TO superwitness_app;` as the owner. `serve` then runs its background migration pass as the runtime role, which succeeds when nothing is pending. ## Start [Section titled “Start”](#start) ```sh sudo install -m 0644 superwitness_0.0.12_linux_amd64/deploy/superwitness.service /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl enable --now superwitness curl -s localhost:8790/health ``` The unit runs `superwitness serve` as the `superwitness` user with its settings from `/etc/superwitness/env`, and restarts it if it exits. `/health` needs no token and answers 200 whenever the process is alive; each source’s state is in the body: ```json {"sources":{"agentpod":"ok","logs":"ok","superpipeline":"ok","traces":"ok","verdicts":"ok"},"status":"ok","version":"v0.0.12"} ``` A source that cannot be reached shows `unavailable` or `timeout` there, and `/health` stays 200. `serve` starts answering at once and runs its migrations in the background, retrying while Postgres is unreachable; until they succeed, `verdicts` reads `unavailable`. You now have a service that starts with the host and a health document that distinguishes the process from each dependency. A green process can still tell you, plainly, which source needs attention. ## Try it without the other products [Section titled “Try it without the other products”](#try-it-without-the-other-products) With `SW_FAKE_SOURCES=1`, SuperWitness serves one development run, `brd_01/run_01`, from built-in fixtures, and accepts development tokens of the form `dev::`. Because it accepts those, it refuses to listen on anything but loopback, and it needs only a database: ```sh SW_FAKE_SOURCES=1 SW_DATABASE_URL='postgres://:@127.0.0.1:5432/?sslmode=disable' superwitness serve curl -H 'Authorization: Bearer dev:prn_human01:human' localhost:8790/v1/runs/superpipeline/brd_01/run_01 ``` Without `SW_MIGRATE_DATABASE_URL`, migrations run over `SW_DATABASE_URL`, which must then own the database. That suits a scratch database for development, not the install above. The response is a complete development run document, so you can learn the shape before asking any other product to cooperate. Next, [Concepts](/concepts/) explains what every part of it means. # The canary > superwitness canary, the live acceptance test that proves nine flows on real runs and the alert path with a forced failure. `superwitness canary` is the live acceptance test we run when “the services are up” is not a convincing ending. It creates one card on a SuperPipeline board, waits for an agent to run it, and then checks, as a black-box client of every product involved, that you can observe the run end to end. The canary is accepted once the first nine **flows** have been proven on real runs and flow 10 has been proven by a forced failure that reaches the alert channel. It is meant to run on the SuperWitness host, nightly from a timer and by hand when needed. It keeps its history in a JSON-lines file, the **tracker**, which it appends one result to per run. ## The ten flows [Section titled “The ten flows”](#the-ten-flows) | Flow | Proves | | ---- | -------------------------------------------------------------------------------------------------------------------- | | 1 | The run document has the run, its card, the work stage and its gates, from SuperPipeline’s run evidence route | | 2 | The AgentPod ledger has the run’s attempts, each with a fingerprint whose harness is `SWC_EXPECT_HARNESS` | | 3 | The run’s trace is joined, with the hub’s dispatch, attempt and turn spans in it | | 4 | The agent’s node continued the trace the hub started | | 5 | A SuperPipeline API span carries the run’s `run.id` | | 6 | The gate on `SWC_GATE_STAGE` is shown pending, and a later run sees it decided by a human who is not the run’s agent | | 7 | `record_verdict` over MCP records a verdict, and a retry with the same idempotency key returns the same one | | 8 | A synthetic error line, sent to `SWC_OTLP_LOGS_URL` with the run’s id, shows up in the run’s logs and errors | | 9 | Two markers planted in the card’s prompt reach neither VictoriaLogs nor VictoriaTraces | | 10 | A failure reaches your alert channel (see [`record-alert`](#canary-record-alert)) | Flow 6 needs a person to decide the gate, so it is usually proven by a later run than the one that saw the gate pending. The canary looks back at up to seven earlier runs for it. ## The commands [Section titled “The commands”](#the-commands) ### `canary run` [Section titled “canary run”](#canary-run) ```sh superwitness canary run --trigger nightly --results /var/lib/superwitness/canary.jsonl ``` | Flag | Meaning | | -------------- | -------------------------------------------------------------------------------------------- | | `--trigger` | `nightly` or `manual` (the default) | | `--force-fail` | record a deliberate failure without creating a card or calling any product, to prove flow 10 | | `--results` | the tracker to append the result to; required | | `--result-out` | also write this run’s result, as indented JSON, to this file | It prints one line, `canary PASS …` or `canary FAIL …`, and exits 0 on a pass, 1 on a failure, and 2 when its flags are wrong or the tracker cannot be read. A result that cannot be appended to the tracker counts as a failure. The whole run is bounded at 70 minutes. The line tells you whether this run passed; the tracker preserves which flows it proved. One run need not perform a tiny miracle and prove all ten if earlier runs already supplied valid evidence. ### `canary record-alert` [Section titled “canary record-alert”](#canary-record-alert) ```sh superwitness canary record-alert --results /var/lib/superwitness/canary.jsonl --ntfy-id --query '' ``` Your alerting wrapper calls this after the alert channel (ntfy) has accepted and stored the message for a forced failure. It records flow 10’s proof. All three flags are required, and the last result in the tracker must be a forced failure; otherwise it records nothing and exits 2. Once recorded, the next status read can distinguish “we emitted an alert” from “the alert channel actually kept it.” That distinction is the entire reason flow 10 exists. ### `canary status` [Section titled “canary status”](#canary-status) ```sh superwitness canary status --results /var/lib/superwitness/canary.jsonl ``` Prints which flows have been proven, when, and on which run, with the last run’s result. With `--json` it prints the same summary as one JSON object. You now have a compact acceptance ledger: the latest result, each proven flow, and the run that earned it, rather than ten green boxes with no memory. Flags may be written with one dash or two, as with every Go flag set. ## Settings [Section titled “Settings”](#settings) The canary reads its own environment. Names it shares with the service keep the `SW_` prefix and mean the same thing ([Configuration](/use/configuration/)); canary-only names start with `SWC_`. `deploy/canary/canary.env.example` in the repository is a starting file. | Variable | Required | Default | Meaning | | -------------------------------------------- | ----------------- | ------------------------------- | --------------------------------------------------------------------------------------- | | `SW_SUPERPIPELINE_URL` | yes | none | SuperPipeline, where the canary creates its card | | `SWC_SUPERWITNESS_URL` | yes | none | SuperWitness itself, for `/v1` and `/mcp`; usually loopback | | `SW_TRACES_URL`, `SW_LOGS_URL` | yes | none | VictoriaTraces and VictoriaLogs, for flows 5 and 9 | | `SW_TRACES_TOKEN_FILE`, `SW_LOGS_TOKEN_FILE` | no | none | bearer tokens for the engines | | `SWC_BOARD_ID` | yes | none | the SuperPipeline board the canary’s cards run on | | `SW_HUB_URL` | without the plane | none | the AgentPod hub | | `SWC_HUB_CLIENT_ID` | without the plane | none | the canary’s own hub service credential id, `svc_…` | | `SWC_HUB_CLIENT_SECRET_FILE` | without the plane | none | a file holding that credential’s secret | | `SW_ORG_PLANE_URL` | no | none | the organization plane; setting it replaces the three hub settings with the two below | | `SWC_ORG_PLANE_SERVICE_CREDENTIAL_FILE` | with the plane | none | a file holding the canary’s plane service credential, `svc_:` | | `SWC_SUPERWITNESS_AUDIENCE` | with the plane | none | SuperWitness’s `SW_PUBLIC_URL`, the audience of its tokens (not `SWC_SUPERWITNESS_URL`) | | `SWC_OTLP_LOGS_URL` | no | `http://127.0.0.1:4318/v1/logs` | the collector’s OTLP/HTTP logs endpoint, for flow 8’s synthetic error line | | `SWC_WORK_STAGE` | no | `work` | the board stage the card is created on and an agent claims | | `SWC_GATE_STAGE` | no | `review` | the board stage whose gate flow 6 watches | | `SWC_EXPECT_HARNESS` | no | `hermes` | the harness flow 2 expects in each attempt’s fingerprint | | `SWC_STANDARD` | no | `rubric:superwitness-canary@1` | the standard flow 7’s verdict names; must be `rubric:@` | | `SWC_CLAIM_TIMEOUT` | no | `10m` | how long an agent has to claim the card | | `SWC_RUN_TIMEOUT` | no | `30m` | how long the run has to end completed | | `SWC_SETTLE_DELAY` | no | `90s` | how long to wait after the run ends, for telemetry to arrive | | `SWC_DOC_TIMEOUT` | no | `5m` | how long to keep fetching the run document until it is complete | | `SWC_POLL_INTERVAL` | no | `15s` | the wait between polls | The five durations are Go durations and must be positive. A missing required setting, or a duration or `SWC_STANDARD` that does not parse, is recorded as a failed run whose error starts `setup:`, and `canary run` exits 1. Like the service, the canary refuses a secret file that is empty or readable by group or others. Every call it makes has a timeout, so one hung product cannot keep it running past its bound. # Configuration > Every environment variable SuperWitness reads, with its default. SuperWitness reads its settings from environment variables; five of them point to secret files instead of putting secrets in the environment itself. Under systemd they come from `/etc/superwitness/env` ([Install](/install/#configure)). We keep the full table here so you can answer “what does this process actually depend on?” in one place. Leading and trailing spaces are trimmed from every value. ## Settings [Section titled “Settings”](#settings) “Normal” is `serve` reading real sources; “fake” is `serve` with `SW_FAKE_SOURCES=1`. | Variable | Required (normal / fake) | Default | Meaning | | -------------------------------------- | --------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SW_LISTEN` | no / no | `:8790`; in fake mode `127.0.0.1:8790` | the address `serve` binds. In fake mode it must be a loopback address. | | `SW_PUBLIC_URL` | yes / no | in fake mode `http://` + `SW_LISTEN` | the address callers reach SuperWitness at. Caller tokens must carry it as their audience. | | `SW_DATABASE_URL` | yes / yes | none | the Postgres DSN for the runtime role. | | `SW_MIGRATE_DATABASE_URL` | no / no | `SW_DATABASE_URL` | the Postgres DSN migrations run over: the owner role. | | `SW_SUPERPIPELINE_URL` | yes / no | none | SuperPipeline, where runs, cards, gates and usage are read. | | `SW_HUB_URL` | yes / no | none | the AgentPod hub: attempts, principals, the service token, and, without the plane, the issuer of caller tokens. | | `SW_SUPERPIPELINE_AUDIENCE` | no / no | `SW_SUPERPIPELINE_URL` | exact registered HTTPS resource for the plane service token; preserves trailing slashes. | | `SW_HUB_AUDIENCE` | no / no | `SW_HUB_URL` | exact registered HTTPS resource for the plane service token; preserves trailing slashes. | | `SW_HUB_CLIENT_ID` | yes / no | none | SuperWitness’s AgentPod service credential id, `svc_…`. | | `SW_HUB_CLIENT_SECRET_FILE` | yes / no | none | a file holding that credential’s secret. | | `SW_ORG_PLANE_ISSUER` | no / no | none: the hub issues every token | the organization plane’s issuer, compared exactly with each token’s `iss` (no trailing slash). Setting it switches caller tokens, browser sign-in and SuperWitness’s own service tokens to the plane; nothing accepts both. | | `SW_ORG_PLANE_JWKS_URL` | with the plane | none | the plane’s key set. Keys are cached for at most 10 minutes, refetched on an unknown `kid`, and the last good set is used while the plane is unreachable. | | `SW_ORG_PLANE_URL` | with the plane | none | the plane’s base URL, for sign-in (`/api/auth/oauth2/*`) and service tokens (`/api/token/service`). | | `SW_ORG_PLANE_SERVICE_CREDENTIAL_FILE` | with the plane (not in fake mode) | none | a file holding SuperWitness’s plane service credential, `svc_:`, on one line; not readable by group or others. Under the plane, `SW_HUB_CLIENT_ID` and `SW_HUB_CLIENT_SECRET_FILE` are not used. | | `SW_TRACES_URL` | yes / no | none | VictoriaTraces. | | `SW_TRACES_TOKEN_FILE` | no / no | none | a file holding a bearer token to send to VictoriaTraces. | | `SW_LOGS_URL` | yes / no | none | VictoriaLogs. | | `SW_LOGS_TOKEN_FILE` | no / no | none | a file holding a bearer token to send to VictoriaLogs. | | `SW_RUN_SOURCES` | no / no | none: nobody may report runs | the run reporters, as `prn_=,…`. Each service principal named may `POST /v1/runs` for its one source ([Run registry API](/build/run-registry/#who-may-report)). | | `SW_ALLOWED_PRINCIPALS` | no / no | none: the app is off | comma-separated `prn_` ids who may sign in to the app ([The app](/use/the-app/)). | | `SW_APP_CLIENT_ID` | when the app is on | none | the AgentPod OAuth client browsers sign in through. | | `SW_SESSION_SECRET_FILE` | when the app is on | none | a file of at least 32 random bytes that keys the sign-in cookie; it must not be readable by group or others. | | `SW_TRUSTED_PROXIES` | no / no | this host’s own addresses | IP addresses or CIDR prefixes whose `CF-Connecting-IP` header is believed. | | `SW_SOURCE_TIMEOUT` | no / no | `2s` | how long each source has to answer, as a Go duration such as `2s` or `1500ms`; it must be positive. | | `SW_OTLP_ENDPOINT` | no / no | none: SuperWitness sends no telemetry of its own | the OTLP/HTTP endpoint for SuperWitness’s own traces and metrics ([Sending telemetry](/use/telemetry/#superwitnesss-own-telemetry)). | | `SW_FAKE_SOURCES` | no / – | `false` | `1` or `true` serves the built-in development run and accepts development tokens ([Install](/install/#try-it-without-the-other-products)). Accepts `true`/`false`/`1`/`0`; empty is false. | With the plane, SuperWitness asks for tokens using `SW_HUB_AUDIENCE` and `SW_SUPERPIPELINE_AUDIENCE`, falling back to each product’s URL when omitted. Set the audience to the already registered resource when the API endpoint differs from that resource, for example `https://hub.agentpod.dev` or `https://app.superpipeline.dev`. The URL settings remain the fetch endpoints. These trusted operator settings do not register resources or change caller-token acceptance. A refused exchange fails without trying the endpoint as another audience. Legacy hub service-token exchange does not use these audience settings. Explicit audiences must be exact HTTPS URLs without credentials, query or fragment; whitespace is rejected and trailing slashes are preserved. In fake mode, the product, engine and credential settings are not used: the sources are built in. A URL among them that is set is still validated. `SW_OTLP_ENDPOINT` and `SW_SOURCE_TIMEOUT` still apply. ## Validation [Section titled “Validation”](#validation) `serve` checks its settings before it starts and exits 2 with a message on the first problem it finds: * **Missing settings.** Every required setting that is unset is named in one message, for example `missing required settings: SW_PUBLIC_URL, SW_HUB_URL`. * **URLs.** `SW_PUBLIC_URL`, `SW_SUPERPIPELINE_URL`, `SW_HUB_URL`, `SW_TRACES_URL`, `SW_LOGS_URL` and `SW_OTLP_ENDPOINT`, when set, must be absolute `http` or `https` URLs with a host. A trailing `/` is removed. * **`SW_SOURCE_TIMEOUT`** must parse as a positive duration. * **`SW_RUN_SOURCES`** must be comma-separated `prn_=` pairs, a source being lowercase letters, digits and `-`, and no principal bound twice. * **`SW_FAKE_SOURCES`** must parse as a boolean. * **The app.** With `SW_ALLOWED_PRINCIPALS` set, `SW_APP_CLIENT_ID` and `SW_SESSION_SECRET_FILE` are required, `SW_PUBLIC_URL` must be `https` (fake mode excepted), and in fake mode `SW_HUB_URL` is required. Each listed id must be `prn_` and letters, digits, `_` or `-`. * **`SW_TRUSTED_PROXIES`** must be IP addresses or CIDR prefixes. * **Fake mode** refuses a `SW_LISTEN` that is not loopback. Secret files are checked as `serve` starts. `SW_HUB_CLIENT_SECRET_FILE` (or, under the plane, `SW_ORG_PLANE_SERVICE_CREDENTIAL_FILE`), `SW_TRACES_TOKEN_FILE`, `SW_LOGS_TOKEN_FILE` and `SW_SESSION_SECRET_FILE` must each hold a non-empty value (surrounding whitespace is trimmed; the session secret must hold at least 32 bytes, and the plane credential must be one line of the form `svc_:`) and must not be readable by group or others; otherwise `serve` exits 1 naming the file. ## Other commands [Section titled “Other commands”](#other-commands) * `superwitness migrate` reads only `SW_MIGRATE_DATABASE_URL`, else `SW_DATABASE_URL`, and exits 2 if neither is set. * `superwitness rubric-add` needs `SW_DATABASE_URL`, and migrates first over `SW_MIGRATE_DATABASE_URL` when that is set. * `superwitness canary` reads its own settings: the canary-only `SWC_*` names and a few `SW_*` ones it shares with the service ([The canary](/use/canary/#settings)). That separation lets you audit the long-running service, one-off migrations, rubric changes and the live canary without assuming that one command quietly reads another command’s credentials. # Operations > Health, backups, upgrades, the database roles, the listen address and the commands, for running SuperWitness day to day. Day to day, the server has only two things to operate: one binary and one Postgres database. The server keeps no state on disk; telemetry lives in the engines, and everything it reads from AgentPod and SuperPipeline stays there. The optional canary does write a tracker to the path passed with `--results`. That small footprint is useful, but we will still be precise about what to health-check, migrate, back up and restart. ## The commands [Section titled “The commands”](#the-commands) | Command | Does | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `superwitness serve` | runs the service; also what runs with no command | | `superwitness migrate` | runs pending migrations once and exits | | `superwitness rubric-add` | records a rubric version ([Verdicts](/use/verdicts/#rubrics)) | | `superwitness canary` | the live acceptance test: `canary run`, `canary record-alert` and `canary status` ([The canary](/use/canary/)) | | `superwitness skill` | prints the operations skill for agents; `superwitness skill install` writes it to `~/.claude/skills/superwitness-ops/SKILL.md`, or under `--dir ` instead | | `superwitness version` | prints the version; `--version` and `-version` do the same | `suwi` is the same binary under a short name, so `suwi skill install` is `superwitness skill install`. `serve` exits 2 when its settings are missing or invalid: missing settings are all named in one message, and any other invalid setting stops it at the first one found ([Configuration](/use/configuration/#validation)). It exits 1 when it cannot start for another reason, such as an unreadable secret file. It stops cleanly on `SIGINT` or `SIGTERM`, giving requests in flight up to 10 seconds. It logs JSON to standard output. ## Health [Section titled “Health”](#health) `GET /health` needs no token. It answers 200 whenever the process can answer at all, so a monitor can tell “SuperWitness is down” from “a source is down”. Each source’s state is in the body: ```json {"sources":{"agentpod":"ok","logs":"ok","superpipeline":"ok","traces":"ok","verdicts":"ok"},"status":"ok","version":"v0.0.12"} ``` * `status` is always `ok`, and `version` is the running release. * Each of `agentpod`, `logs`, `superpipeline` and `traces` is a `GET /health` against that product or engine: any 2xx is `ok`, 401 or 403 `unauthorized`, anything else `unavailable`. Each check gets 1 second, after which it reads `timeout`. The engine checks carry the `SW_TRACES_TOKEN_FILE` or `SW_LOGS_TOKEN_FILE` token when one is set. * `verdicts` is a ping of SuperWitness’s own Postgres. It reads `unavailable` until migrations have succeeded. To alert on a source that keeps failing during real reads, use the `superwitness.source.fetches` metric ([Sending telemetry](/use/telemetry/#superwitnesss-own-telemetry)). ## Migrations at start [Section titled “Migrations at start”](#migrations-at-start) `serve` answers at once and runs its migrations in the background, over `SW_MIGRATE_DATABASE_URL`, else `SW_DATABASE_URL`. While they fail it retries, waiting 1 second and doubling to at most 30, and logs each failure. Until they succeed: * `/health` shows `verdicts` as `unavailable`; * run documents show `sources.verdicts` as `unavailable`, and everything else in them as usual; * `POST /v1/verdicts` answers 503 `store_unavailable`, which is retryable, and the `record_verdict` tool returns the same error. A Postgres that is down never takes run documents with it. ## The two database roles [Section titled “The two database roles”](#the-two-database-roles) SuperWitness uses two Postgres roles, set up in [Install](/install/#postgres-roles): * **`superwitness_owner`** owns the database and the tables, and is used only for migrations, through `SW_MIGRATE_DATABASE_URL`. * **`superwitness_app`** is the runtime role, through `SW_DATABASE_URL`, with `SELECT` and `INSERT` on `verdicts` and `rubrics`, `SELECT`, `INSERT` and `UPDATE` on `runs` (the run registry), and `SELECT`, `INSERT`, `UPDATE` and `DELETE` on `sessions` (app sign-ins), and nothing else ([the grants](/install/#migrate)). The split keeps the running service from rewriting verdicts through SQL: it does not own the tables, so it cannot disable or drop the triggers that refuse `UPDATE`, `DELETE` and `TRUNCATE`. It does not protect against a compromised process that holds the owner’s password, so if `SW_MIGRATE_DATABASE_URL` is in the service’s settings file, the service holds that password. To keep it out: 1. leave `SW_MIGRATE_DATABASE_URL` out of `/etc/superwitness/env`, and set it only in your own environment when you run `superwitness migrate`; 2. as the owner, grant the runtime role read access to the migration bookkeeping table: ```sql GRANT SELECT ON goose_db_version TO superwitness_app; ``` `serve` then runs its background migration pass as the runtime role. With that grant, the pass succeeds when nothing is pending; without it, the pass always fails. When a migration is pending it fails either way, and verdicts stay unavailable until you run `superwitness migrate` as the owner. The owner password is now outside the long-running process, while startup can still prove that the schema is already current. A pending migration fails closed for verdicts rather than quietly running with the wrong role. ## Upgrades [Section titled “Upgrades”](#upgrades) 1. Download and verify the new release as in [Install](/install/#download-and-verify). 2. Run the new binary’s migrations as the owner, before restarting: ```sh SW_MIGRATE_DATABASE_URL='postgres://superwitness_owner:@127.0.0.1:5432/superwitness?sslmode=disable' \ superwitness__linux_amd64/superwitness migrate ``` It exits 0 when the schema is up to date, and non-zero with the error otherwise. It gives up after 60 seconds if Postgres does not answer. 3. Install the binary and restart: `sudo systemctl restart superwitness`. Once the restart succeeds, the new binary is running against its matching schema. Check [`/health`](#health) before you call the upgrade finished; it keeps a source problem separate from a process that did not come back. A migration that adds a table needs a grant of its own for the runtime role: nothing is granted through `ALTER DEFAULT PRIVILEGES`. ## Backups [Section titled “Backups”](#backups) SuperWitness’s only state is plain Postgres: the `superwitness` database, with its `verdicts` and `rubrics` tables, the run registry in `runs`, and app sign-ins in `sessions`. Back it up with the usual tools, for example as the owner role or the Postgres superuser: ```sh pg_dump -Fc -f superwitness.dump 'postgres://superwitness_owner@127.0.0.1:5432/superwitness' ``` That dump contains the server’s database state. If you run the canary, back up its `--results` tracker separately. The short-lived telemetry engines do not belong in either backup. Use the owner or the superuser: the runtime role cannot read the migration bookkeeping table unless you granted it above. The telemetry engines are not SuperWitness’s to back up. Telemetry is lossy by design, sampled and kept for a short time, and SuperWitness never treats a missing span as missing evidence. Evidence that belongs to AgentPod or SuperPipeline stays in those products, not in SuperWitness’s database. ## The listen address [Section titled “The listen address”](#the-listen-address) `SW_LISTEN` is the address `serve` binds. Left unset it is `:8790`, every interface. The [Install](/install/#configure) guide sets `127.0.0.1:8790`; to reach it from other machines, bind a private address of the host instead, or keep loopback and put a reverse proxy in front. In fake mode (`SW_FAKE_SOURCES=1`) SuperWitness accepts development tokens, so it refuses to start unless `SW_LISTEN` is a loopback address: `localhost`, a `127.x.x.x` address or `[::1]`, with a port. Left unset in fake mode, it is `127.0.0.1:8790`. A client has 5 seconds to send a request’s headers. # Read a run > The app's run view, the run document, and the paged spans and logs behind them. You can read a run three ways: in the app, over HTTP, or through the [MCP tools](/build/mcp/). We will take the same document through each useful view, then page through the heavier spans and logs separately. All three entry points assemble the run at read time from every source. [Concepts](/concepts/) explains its fields; [Contracts](/build/contracts/) lists every one. ## In the app [Section titled “In the app”](#in-the-app) Signed in to [the app](/use/the-app/), open a run from the list, or go to `/runs/superpipeline/{board}/{run}`. The run view shows the run’s title, status and a strip of facts (agent, attempts, errors, duration and cost), and tabs for the trace, logs, errors, verdicts and attempts. A source that did not answer shows as unavailable in its tab, and a value no source supplied reads `unknown`, never 0. ## The run document [Section titled “The run document”](#the-run-document) ```sh curl -H 'Authorization: Bearer ' \ https://superwitness.example.internal/v1/runs/superpipeline/brd_01/run_01 ``` `GET /v1/runs/superpipeline/{board}/{run}` returns the run document. Board and run ids are letters, digits, `_` and `-`, starting with a letter or digit, up to 128 characters; anything else is 400 `invalid_run_ref`. The answer is 404 `run_not_found` only when SuperPipeline and the AgentPod hub both answer that they do not know the run. Any other failure of a source is a 200 with that source’s status in `sources`. The response gives you the joined view even when one source is having a bad day. Read `sources` first and you know whether a 0 is a real 0, an `unknown` is missing evidence, or a dependency needs attention. The document’s `links` hold the paths for this run: ```json "links": { "self": "/v1/runs/superpipeline/brd_01/run_01", "spans": "/v1/runs/superpipeline/brd_01/run_01/spans", "logs": "/v1/runs/superpipeline/brd_01/run_01/logs" } ``` ## From an attempt to its run [Section titled “From an attempt to its run”](#from-an-attempt-to-its-run) `GET /v1/runs/by-attempt/{attempt}` asks the AgentPod hub which run an attempt belongs to and answers 302, with the run document’s path in `Location`: ```plaintext HTTP/1.1 302 Found Location: /v1/runs/superpipeline/brd_01/run_01 ``` An attempt id is `attempt_`; anything else is 400 `invalid_attempt_id`. The hub not knowing the attempt is 404 `attempt_not_found`, and an attempt that was not dispatched from a SuperPipeline run is 404 `attempt_has_no_run`. The redirect gives you the canonical run address, so an attempt link becomes a run link without a second lookup in your client. ## Spans, one page at a time [Section titled “Spans, one page at a time”](#spans-one-page-at-a-time) `GET /v1/runs/superpipeline/{board}/{run}/spans?cursor=&limit=` lists the run’s spans, oldest first (by start time, then trace id, then span id): ```json { "spans": [ { "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "span_id": "00f067aa0ba902b7", "name": "dispatch", "service": "agentpod-hub", "start": "2026-10-04T10:00:00Z", "duration_ms": 300000, "attributes": {"board.id": "brd_01", "card.id": "crd_01", "external.source": "superpipeline", "run.id": "run_01"} } ], "next_cursor": "eyJvIjoxfQ" } ``` Each span has `trace_id`, `span_id`, `parent_span_id` (left out for a span with no parent), `name`, `service`, `start`, `duration_ms` and `attributes`, whose values are all strings. `next_cursor` is left out on the last page. Which spans count as the run’s is covered in [Sending telemetry](/use/telemetry/#spans). ## Logs, one page at a time [Section titled “Logs, one page at a time”](#logs-one-page-at-a-time) `GET /v1/runs/superpipeline/{board}/{run}/logs?cursor=&level=&limit=` lists the run’s log lines, oldest first: ```json { "logs": [ { "at": "2026-10-04T10:00:02Z", "service": "agentpod-hub", "level": "info", "message": "attempt opened", "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "span_id": "00f067aa0ba902b8", "run_id": "run_01" } ], "next_cursor": "eyJvIjoxfQ", "trace_join": "ok" } ``` * `level` filters by level: `debug`, `info`, `warn` or `error`. Left out, every level is listed. Any other value is 400 `invalid_level`. * Each line’s `level` is one of `debug`, `info`, `warn`, `error` or `unknown`, and `service` is `unknown` when the line has no service name. `trace_id`, `span_id` and `run_id` are left out when the line has none. * `next_cursor` is `null` on the last page. * A line belongs to the run when it carries the run’s id, or one of the run’s trace ids. To know the trace ids, SuperWitness first reads the run’s spans; `trace_join` is the traces source’s status for that read. When it is not `ok`, lines were matched by run id alone. ## Paging [Section titled “Paging”](#paging) Both lists page the same way: * `limit` is the page size. Left out, it is 100; above 500, it is 500. Anything that is not a whole number of at least 1 is 400 `invalid_limit`. * `cursor` is the `next_cursor` from the previous page. Treat it as opaque. A cursor SuperWitness did not issue is 400 `invalid_cursor`. If the engine behind a list cannot answer, the request fails rather than returning an empty page: `traces_unavailable` or `logs_unavailable` (503), `traces_timeout` or `logs_timeout` (504), or `traces_unauthorized` or `logs_unauthorized` (502). The [HTTP API](/build/api/#errors) lists every error. Follow `next_cursor` until it is absent or null and you have the ordered evidence; if a source fails, you get an explicit failure instead of an empty page wearing a convincing hat. # Sending telemetry > Which spans and log lines SuperWitness finds for a run, the trace conventions it relies on, and the telemetry it sends about itself. SuperWitness does not receive your telemetry, and that boundary is worth keeping visible. Your products send OTLP to the OpenTelemetry Collector, which writes spans to VictoriaTraces and logs to VictoriaLogs ([Install](/install/#the-telemetry-engines) has a minimal configuration). When you read a run, SuperWitness queries both engines through their published APIs and keeps nothing. We will trace exactly what it looks for, so you can send enough context without sending content. Both engines are searched over the last 7 days, and older telemetry is not found. A run whose spans are not found has trace status `none`, which is not an error. ## Spans [Section titled “Spans”](#spans) SuperWitness reads spans from VictoriaTraces’ Jaeger-compatible API, with one request for each of these services: * `agentpod-hub` * `agentpod-node-agent` * `superpipeline-api` ```plaintext GET /select/jaeger/api/traces?service=&tags={"run.id":""}&start=<µs>&end=<µs>&limit=100 ``` So spans are found through the **`run.id` attribute**, which holds the SuperPipeline run id (`run_01`, not the full ref), on a span from one of those three services. Each request returns up to 100 traces, and every span of those traces is listed, including spans without `run.id` of their own. A span that appears in more than one answer is listed once. A trace is found only through a span from one of these three services; spans from any other service appear only when they share a trace with one. From the spans it finds, SuperWitness uses: | What | Used for | | -------------------------------------- | ----------------------------------------------------------------------------------- | | `run.id` attribute | finding the run’s traces | | trace ids | `trace.trace_ids`, ordered by each trace’s earliest span, and matching log lines | | `attempt.id` attribute | each attempt’s `span_count`: spans whose `attempt.id` is that attempt’s id | | a span named `dispatch` with no parent | `trace.status` is `joined` when every span found is in one trace that has this root | If any of the three requests fails, the traces source reports the worst status among them, and SuperWitness uses none of the answers rather than some of them. ### The trace shape SuperWitness expects [Section titled “The trace shape SuperWitness expects”](#the-trace-shape-superwitness-expects) SuperWitness’s contract fixtures pin the trace it is built to read, with these spans from `agentpod-hub`: * a root span **`dispatch`**, carrying `run.id`, `board.id`, `card.id` and `external.source`; * under it, an **`attempt`** span per attempt, carrying `attempt.id`, `station.id`, `fingerprint.digest`, `harness.name`, `acp.session_id`, `acp.seq_from` and `run.id`; * under that, **`turn`** spans carrying `attempt.id`, `acp.seq_from` and `acp.seq_to`; * spans from `superpipeline-api` carrying `run.id`. ### Trace context between products [Section titled “Trace context between products”](#trace-context-between-products) For a run to land in one trace, each hop must continue the caller’s trace context. This is the convention AgentPod and SuperPipeline follow. They carry the context; SuperWitness only reads the trace that results: | Hop | Where the context travels | | ------------------------------------------------ | -------------------------------------------------------------- | | HTTP between products | the W3C `traceparent` header | | AgentPod’s hub to its node-agent, over WebSocket | `_meta.traceparent` on each request message | | node-agent to the agent harness, over ACP | `traceparent` in `_meta` on `session/new` and `session/prompt` | SuperWitness does not send or read `_meta` itself. When a hop does not continue the trace, the run’s spans arrive in more than one trace. SuperWitness still finds them through `run.id`, lists every trace id, and reports `trace.status` as `partial` instead of `joined`. ## Log lines [Section titled “Log lines”](#log-lines) SuperWitness reads logs from VictoriaLogs with LogsQL (`POST /select/logsql/query`). A line belongs to a run when either: * its `run.id` field equals the run id; or * its `trace_id` field is one of the run’s trace ids, found as above. Lines from any service match; there is no service list for logs. The fields SuperWitness reads from each line: | Field | Becomes | | ---------------------------------- | --------------------------------- | | `_time` | `at` | | `_msg` | `message` | | `service.name` | `service` (`unknown` when absent) | | `severity_text`, `severity_number` | `level` | | `trace_id`, `span_id` | `trace_id`, `span_id` | | `run.id` | `run_id` | A line’s level comes from `severity_text` when it names one, in any case: text starting with `err`, or equal to `fatal` or `critical`, reads as `error`; text starting with `warn` as `warn`; `info` and `debug` as themselves. Otherwise it comes from `severity_number`: 5–8 `debug`, 9–12 `info`, 13–16 `warn`, 17–24 `error`. A line with neither reads `unknown`. Filtering by level matches `severity_number` in that range, or `severity_text` matching the level’s name, case-insensitively. From these lines the run document takes: * `log_count`: the number of matching lines, at every level; * `errors`: the first 50 error-level lines, oldest first, each with `service`, `message`, `trace_id` and `at`. The full list is paged through [the logs route](/use/read-a-run/#logs-one-page-at-a-time). ## Content stays out of telemetry [Section titled “Content stays out of telemetry”](#content-stays-out-of-telemetry) No prompt, message or tool content belongs in any span or log. Content stays in the product that holds it, and telemetry points at it instead: the `attempt` and `turn` spans above carry `acp.session_id` and a range of event sequence numbers, not the events. Verdicts follow the same rule: their `evidence_refs` are span ids or `{session_id, seq_from, seq_to}` ranges, never the evidence. ## SuperWitness’s own telemetry [Section titled “SuperWitness’s own telemetry”](#superwitnesss-own-telemetry) Set `SW_OTLP_ENDPOINT` to the collector’s OTLP/HTTP address, for example `http://127.0.0.1:4318`, and SuperWitness sends its own: * **traces**, on the standard OTLP/HTTP traces path under that endpoint: spans for the HTTP requests it serves and the HTTP requests it makes to the products and engines; * **metrics**, on the standard OTLP/HTTP metrics path, every 30 seconds, including the `superwitness.source.fetches` counter. Its resource is `service.name` `superwitness`, with `service.version` set to the release. `superwitness.source.fetches` counts each source read made while building a run document, with exactly two attributes: `source` (`superpipeline`, `agentpod`, `traces`, `logs` or `errors`) and `status` (`ok`, `unavailable`, `timeout`, `not_found` or `unauthorized`). No run, attempt, station or fingerprint id is ever a label. Alert on `unavailable`, `timeout` and `unauthorized` to see a source failing; `not_found` only means a source did not know a run. SuperWitness propagates W3C trace context, so its spans for a request join the caller’s trace when the request carries `traceparent`. Its exporters drop data rather than block: the span queue holds 2048 spans. Left unset, `SW_OTLP_ENDPOINT` sends nothing. SuperWitness also writes JSON logs to standard output: startup, migration retries, and a warning naming any source that refused its credential. With those signals in place, you can observe the observer without turning run ids, attempts or fingerprints into metric labels. That is a sentence we only need because telemetry systems have a talent for becoming their own weather. # The app > Signing in with AgentPod, browsing runs, reading a run and recording verdicts in the browser. The API is not the only way in. SuperWitness serves a web app from the same binary and address, so you can browse reported runs, open one, record a verdict against a rubric and revise it without building a client first. People on an allowlist sign in with their AgentPod account; agents and scripts keep using the [HTTP API](/build/api/) and the [MCP tools](/build/mcp/) with bearer tokens. ## Turning it on [Section titled “Turning it on”](#turning-it-on) The app is off until someone is allowed to sign in. Four settings turn it on ([Configuration](/use/configuration/)): | Setting | Meaning | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `SW_ALLOWED_PRINCIPALS` | comma-separated AgentPod principal ids (`prn_…`) who may sign in. Empty, nobody may, and the app is off. | | `SW_APP_CLIENT_ID` | the AgentPod OAuth client browsers sign in through. | | `SW_SESSION_SECRET_FILE` | a file holding at least 32 random bytes, readable by SuperWitness alone. It keys the short-lived cookie that carries a sign-in to AgentPod and back. | | `SW_TRUSTED_PROXIES` | addresses or CIDR prefixes allowed to say who a client is with `CF-Connecting-IP`. Unset, it is this host’s own addresses. | `SW_PUBLIC_URL` must be `https` when sign-in is on: the app’s cookies are `Secure`, and a browser drops them over plain http. (Fake mode, which listens on loopback only, is the exception.) Make the secret with, for example: ```sh head -c 48 /dev/urandom | base64 | sudo tee /etc/superwitness/session-secret >/dev/null sudo chmod 0600 /etc/superwitness/session-secret && sudo chown superwitness /etc/superwitness/session-secret ``` The cookie key is now private to the service. Once the OAuth client has the callback and audience below, an allowlisted person can complete sign-in instead of meeting a very secure dead end. At AgentPod, the OAuth client named by `SW_APP_CLIENT_ID` must list `{SW_PUBLIC_URL}/auth/callback` as a redirect URI and `SW_PUBLIC_URL` as an audience. SuperWitness also needs its own service credential to hold `evidence:read`, as it already does to read attempts: it uses it to look up who signed in. ## Signing in [Section titled “Signing in”](#signing-in) **Sign in** sends the browser to AgentPod, which signs the person in if they are not already and sends them back to `/auth/callback` with a one-time code. SuperWitness trades the code for a token server to server, with PKCE, checks the token, and looks up the AgentPod principal behind it. It then checks that the principal is a person, is not suspended, and is on `SW_ALLOWED_PRINCIPALS`. Anyone else, including other valid AgentPod accounts, agents and services, sees **Not authorised**. Each refusal is logged as `auth.signin_refused` with a reason, for example `not_allowlisted`, `login_missing` (the sign-in cookie was absent) or `no_code` (the callback carried no code). A sign-in that succeeds is logged as `auth.signin` with the principal. The log never holds a token or a cookie. With `SW_ORG_PLANE_ISSUER` set, the app signs people in through the organization plane as the OAuth client named by `SW_APP_CLIENT_ID` (the plane registers it with the redirect `SW_PUBLIC_URL/auth/callback`). Both the authorize and the token request carry `resource=SW_PUBLIC_URL`, and the authorize asks for `scope=openid offline_access`. The token’s `sub` is the person’s `prn_` id, checked against `SW_ALLOWED_PRINCIPALS` with no lookup. Three more refusal reasons apply: `product_not_enabled` (the person’s workspace has not enabled SuperWitness), `issuer_unavailable` (the plane did not answer) and `no_refresh_token` (the plane issued no refresh token). The pages then name the plane’s host instead of AgentPod. Under the plane the session holds the plane’s refresh token, server side: it is encrypted in the database with a key derived from the session cookie, which the database never holds, and it never reaches the browser. That makes SuperWitness one of the person’s connected apps at the plane, and the session lives only as long as that grant: * when the plane’s access token is past its five-minute life, the next request refreshes it with the stored refresh token and keeps the rotated one. Requests on one session share a single refresh. If the new refresh token cannot be saved, the request answers 503 and a retry within the plane’s 30-second replay window gets the same answer back; * if the plane refuses the refresh (the person signed SuperWitness out at the plane, left the workspace, or the workspace turned SuperWitness off), the session ends at that request and the app shows the sign-in screen (`auth.session_ended`, reason `grant_refused`); * if the plane cannot be reached, the session carries on, retrying every 30 seconds and logging `auth.refresh_unavailable`. It ends if the plane is still unreachable 10 minutes after the first failure (reason `issuer_unreachable`); * a session signed in before sessions held a grant ends at its next request (reason `no_grant`); signing in again is silent while the plane’s own session lasts. The sign-in itself rides on a signed `__Host-sw_login` cookie that lives ten minutes and is used once. A signed-in browser then holds a `__Host-sw_session` cookie and a SuperWitness session, never an AgentPod token. Over plain http (fake mode only) the cookies are `sw_login` and `sw_session`, without the `__Host-` prefix. * the session lasts at most 12 hours, and ends after 2 hours unused; * every request checks the allowlist again, so taking someone off it (and restarting SuperWitness) ends their access at their next click; * **Sign out** (`POST /auth/logout`) ends the session at once and clears the cookie. Under the plane it first revokes the session’s refresh token at the plane’s revocation endpoint (found through the plane’s discovery document, and used only on the plane’s own origin); a revocation that fails is logged as `auth.revoke_failed` and the session ends all the same. If the database cannot be reached the answer is 503 `store_unavailable`, with the cookie cleared all the same, and the session expires on its own. A sign-out is logged as `auth.signout` with the principal, and one the database refused as `auth.signout_failed`. Signing in again from a browser that still holds a session ends the old one first, logged the same way with reason `signed_in_again`; * sessions do not need AgentPod: if AgentPod is down, open sessions keep working and only new sign-ins wait. Changes made with a session must come from the app’s own page (their `Origin` must equal `SW_PUBLIC_URL`); anything else is refused with 403 `origin_mismatch`. Requests with a bearer token are not affected, and a request that carries a bearer token is judged by the token alone. Run reports (`POST /v1/runs`) and `/mcp` never accept a session. ## Pages [Section titled “Pages”](#pages) * **Runs**: every run sources have reported, newest first, grouped under Today, Yesterday and earlier dates in your own time zone. The sidebar has All, Needs verdict (finished runs no verdict names yet), Failed and Waiting, and a view per board. Each card shows the title, the status, the board, who ran it, how long it took or when it started, and its latest verdict. The filters are in the address, so a view can be bookmarked. With no runs yet, the page links to the [Run registry API](/build/run-registry/). * **A run**: its source and reference, title, status and **Record verdict**; a strip with the agent, attempts, errors, duration and cost; and tabs for the trace (a span waterfall, 500 spans at a time), logs (with a level filter), errors, verdicts (with revisions shown as history: a superseded verdict is struck through and links to the one that replaced it) and attempts (each with its configuration fingerprint). A source that did not answer shows as unavailable, and a value no source supplied reads `unknown`, never 0. A run missing from the list still opens by its address, `/runs/superpipeline/{board}/{run}`. * **Rubrics**: every rubric version, and each one’s scale and text. Rubrics are added with `superwitness rubric-add`. The sidebar’s dot is green when every source answers `/health`, amber otherwise; hover for which. The app has four vibes (Daylight, Paper, Studio and Quiet), each in light and dark. Open the account chip, then Appearance, to choose the vibe, to follow the sun, stay light or stay dark, and to pick an accent; the choice is kept for this browser. On a phone the sidebar folds into a menu. ## What an agent did [Section titled “What an agent did”](#what-an-agent-did) Click a span’s name in the Trace tab for its details, including the request and response where the span maps to the agent’s session; the Transcript tab reads the attempt as a conversation. Either can be cited in a verdict. [Transcripts](/use/transcripts/) has the details. ## Recording a verdict [Section titled “Recording a verdict”](#recording-a-verdict) **Record verdict** opens a drawer. Choose the subject (the run, or one of its attempts) and a rubric, then give the value the rubric’s scale asks for ([rubric scales](/use/verdicts/#rubric-scales)): a decision, a score from 0 to 1, a label, or text. Add a comment of up to 10,000 characters if you like. Spans ticked in the Trace tab, up to 100, go with it as evidence. A rubric whose scale the app cannot offer an input for is listed but cannot be chosen. The drawer makes its idempotency key when it opens, so a double click or a retry records one verdict. **Revise** appears on your own latest verdicts: it opens the drawer with the same subject and rubric and records a verdict that supersedes the old one. The app explains refusals in plain words, for example “You’ve already revised this verdict” when someone revised it first. After you save, the verdict appears on the run and in the registry’s latest-verdict view. A revision keeps the old record visible in history, so correcting a judgement does not airbrush the first one. ## Rate limits [Section titled “Rate limits”](#rate-limits) SuperWitness limits requests in its own process, by client address or by principal: | What | Limit | | --------------------------- | -------------------------------------------- | | `/auth/*` | 10 a minute per client address, bursts of 20 | | changes made with a session | 30 a minute per person, bursts of 10 | | `POST /v1/runs` | 600 a minute per reporter, bursts of 100 | Over a limit the answer is 429 `rate_limited` with `Retry-After`. The client address is the connection’s, unless the connection comes from `SW_TRUSTED_PROXIES`, in which case it is the `CF-Connecting-IP` header. ## Putting it on the internet [Section titled “Putting it on the internet”](#putting-it-on-the-internet) SuperWitness serves the app, the API and `/mcp` on one address. To publish the app through a tunnel or a reverse proxy, point the proxy at SuperWitness from an address in `SW_TRUSTED_PROXIES` and have it set `CF-Connecting-IP`. A request that arrives that way, through the public edge, is answered 404 on `/mcp` before any authentication, so the MCP tools stay on your private network. Callers on the private network, which carry no `CF-Connecting-IP`, are unaffected. Set `SW_PUBLIC_URL` to the public https address, and add that address as an audience to every AgentPod client whose tokens call SuperWitness, before you change it, so that tokens carry both the old and the new audience during the switch. ## When something is down [Section titled “When something is down”](#when-something-is-down) | What | What you see | | ------------------------------- | ------------------------------------------------------------------ | | AgentPod, during sign-in | “AgentPod sign-in is unavailable”; open sessions keep working | | a source | that run’s tab says it is unavailable; the sidebar dot turns amber | | SuperWitness’s database | “Can’t reach the database” | | a link to a run no source knows | “Run not found” | # Transcripts > What an agent was asked, what it sent to each tool and what came back, read live from AgentPod, redacted, and never kept by SuperWitness. A run’s spans tell you where and when something happened. When you need the “what,” open the transcript: the prompt an attempt received, what the agent sent to each tool, what came back, which permissions it requested and how they were answered. SuperWitness reads that content live from AgentPod’s session events, the source of truth, one attempt at a time. **Content is evidence, never telemetry.** Traces and logs stay content-free. SuperWitness never stores a transcript, never logs one and never lets one be cached: every answer carries `Cache-Control: no-store`. ## In the app [Section titled “In the app”](#in-the-app) * **Span details.** In the Trace tab, click a span’s name. A pane opens on the right (the whole screen on a phone) with the span’s timing, status, attributes and log lines. For an `attempt`, `turn`, `tool_call` or `permission` span it also shows the **Request** and **Response**: a tool call’s input and output, a permission’s question, options and outcome, or the prompt and the agent’s reply. Content is read only when a span is opened. * **The Transcript tab** shows the attempt as a conversation: prompt, agent message, reasoning (folded), tool calls (a failed call has a red edge and opens by itself), permissions (question, options, outcome), state changes, errors, and any other event by its type. 200 steps a page, with **Load more**. A run with several attempts has a picker. * **Links both ways.** **Open in transcript ↗** in the pane goes to the same step; **span ↗** on a step goes back to the Trace tab with its span open. The address keeps the tab, the span, the attempt and the `seq`, so a link opens the exact moment. * **Cite.** A span or a step can be cited as a verdict’s evidence (a span id, or `{session_id, seq_from, seq_to}`), at most 100 per verdict. A verdict that cites a step links back to it. * **When content is not there.** “Transcripts aren’t available to this account” (the rest of the page still works), “Transcript unavailable, try again” (AgentPod could not be read), and “Attempt still running; refresh for more”. ## Who may read content [Section titled “Who may read content”](#who-may-read-content) * A person signed in to the app. Only principals on `SW_ALLOWED_PRINCIPALS` can sign in. * A bearer token whose AgentPod-signed `scope` includes `transcripts:read`. Anyone else gets 403 `transcripts_forbidden`, over HTTP and over MCP alike. `evidence:read` alone is not enough. SuperWitness itself reads AgentPod with its own service credential, whose grant must hold `transcripts:read` beside `evidence:read`. Without it, transcript reads answer 502 `hub_refused` and everything else works as before. ## Redaction [Section titled “Redaction”](#redaction) AgentPod redacts every string in a transcript before it leaves the hub; SuperWitness shows what it receives. A redacted value reads `[redacted:]`, and each step and page says how many values were redacted. The rules are AgentPod’s: the hub’s own secrets, common credential formats (API keys, tokens, private keys, `Authorization` values, credentials in URLs), JSON keys whose names say they hold a secret, and any rules its operator adds. The stored events are never changed. A field longer than 16 KiB arrives cut, as `{"truncated": true, "bytes": N, "head": "…"}`. **Show full** reads that one step whole (up to 1 MiB). ## Audit [Section titled “Audit”](#audit) Every read that reaches AgentPod leaves two records, neither holding content: * at AgentPod, an audit row naming SuperWitness, with the person or agent SuperWitness read for in `on_behalf_of` (SuperWitness sends their `prn_` id as `X-On-Behalf-Of`); * in SuperWitness’s log, one `transcript.read` line with `principal`, `via` (`session` or `bearer`), `run`, `attempt`, `seq_from`, `seq_to`, `item`, `full`, and on success `items` and `redactions`; a refused read logs at WARN with its `code`. A read SuperWitness itself refuses never reaches AgentPod, so it leaves only that WARN line. ## The API [Section titled “The API”](#the-api) ```plaintext GET /v1/runs/superpipeline/{board}/{run}/transcript?attempt=&seq_from=&seq_to=&cursor= GET /v1/runs/superpipeline/{board}/{run}/transcript/items/{seq_from}?attempt=&seq_from=&seq_to=&full=1 ``` * `attempt` is required when the run has more than one attempt (400 `attempt_required`). * The range must lie inside the attempt (400 `range_outside_attempt`); without one, the whole attempt is read. A running attempt has no end yet: the read goes to the session’s current end. * On the item route, `seq_from` and `seq_to` are optional: the range the step was shown in, which the app forwards on **Show full**. They must lie inside the attempt too. * The answer is AgentPod’s, unchanged, plus `"attempt_id"`: `{"attempt_id", "session_id", "seq_from", "seq_to", "items": […], "next_cursor", "redactions", "truncated_fields"}`. Pass `next_cursor` back as `cursor`, with the same `seq_from` and `seq_to`, for the next page. * Each item has a `kind`: `prompt`, `message`, `reasoning`, `tool_call`, `permission`, `state`, `error` or `other`, with its seqs and fields; a tool call or permission that started before the range has `"partial": true`. Agents use the MCP tool [`get_transcript`](/build/mcp/#get_transcript), with the same rules. Every code is on the [HTTP API](/build/api/#every-code) page. Whichever path you use, the useful result is the same: a redacted, uncached view of one attempt, with enough coordinates to cite the moment without copying its content into a verdict. # Verdicts > Recording a verdict, the standards it names, and why verdicts are append-only. A verdict is a judgement about a run, an attempt or an eval case run, against a named standard. We will record one, name the rubric or stage it belongs to, and correct it without changing the original. SuperWitness stores verdicts in its own Postgres and never edits one once it is written; the paper trail is the feature, not paperwork escaping from a drawer. [Concepts](/concepts/#verdict) introduces the model; this page puts it to work. ## Record a verdict [Section titled “Record a verdict”](#record-a-verdict) Post one JSON object to `POST /v1/verdicts`, or call the `record_verdict` MCP tool with the same fields: ```sh curl -X POST -H 'Authorization: Bearer ' -H 'Content-Type: application/json' \ https://superwitness.example.internal/v1/verdicts -d '{ "idempotency_key": "review-brd_01-run_01-1", "subject_kind": "run", "subject_ref": "superpipeline:brd_01/run_01", "standard": "stage:draft", "value": {"decision": "approved"}, "comment": "Reads well." }' ``` | Field | Required | Meaning | | ----------------- | -------- | -------------------------------------------------------------------------------------------------------------- | | `idempotency_key` | yes | a key you choose, 1 to 200 bytes; see [Retries](#retries-and-idempotency) | | `kind` | no | `grader`, `review`, `eval` or `calibration`; left out, `review` for a human judge and `grader` for anyone else | | `subject_kind` | yes | `run`, `attempt` or `eval_case_run` | | `subject_ref` | yes | `superpipeline:/`, `attempt_`, or `case:@sha256:<64 hex>` to match `subject_kind` | | `standard` | yes | what the verdict judges against; see [Standards](#standards) | | `value` | yes | the judgement; see [Values](#values) | | `comment` | no | free text, at most 10000 characters | | `evidence_refs` | no | up to 100 pointers to evidence; see [Evidence references](#evidence-references) | | `supersedes` | no | the id of your earlier verdict that this one corrects | The HTTP body also accepts `judge`, which must equal the caller’s own principal id if it is present, and refuses `judge_kind` outright (400 `judge_kind_not_accepted`): the judge is always the caller, and its kind comes from the caller’s principal record. Any other field is refused as 400 `invalid_json`, as is a body over 64 KiB or one holding more than one JSON value. A `kind` outside the four is 400 `invalid_kind`; a `subject_kind` outside the three is 400 `invalid_subject_kind`; a `subject_ref` that does not fit its kind is 400 `invalid_subject`. A new verdict answers 201 with the stored verdict: ```json { "id": "vrd_…", "idempotency_key": "review-brd_01-run_01-1", "kind": "review", "subject_kind": "run", "subject_ref": "superpipeline:brd_01/run_01", "judge": "prn_human01", "judge_kind": "human", "standard": "stage:draft", "value": {"decision": "approved"}, "comment": "Reads well.", "evidence_refs": [], "supersedes": null, "created_at": "…" } ``` `comment` is `""` and `evidence_refs` is `[]` when none were given. Recorded verdicts on the run and its attempts appear in the run document’s `verdicts`, after the run’s gates. That response is your receipt: the judgement has a stable id, the caller is the judge, and the run document can now show it beside the gate evidence it was meant to complement. ## Values [Section titled “Values”](#values) `value` is an object with exactly one key: | Value | Rule | | ------------------- | -------------------- | | `{"decision": "…"}` | a non-empty string | | `{"score": 0.7}` | a number from 0 to 1 | | `{"label": "…"}` | a non-empty string | | `{"text": "…"}` | a non-empty string | Anything else is 400 `invalid_value`, as is a string holding NUL or an unpaired UTF-16 surrogate escape. ## Standards [Section titled “Standards”](#standards) Every verdict names a standard. Leaving it out is 422 `missing_standard`; a standard in none of these forms is 422 `invalid_standard`: | Standard | Form | | ----------------------- | ------------------------------------------------------------------------------------------------------------------- | | `rubric:@` | a rubric recorded in SuperWitness: id of letters, digits, `_`, `.` and `-`, up to 64; version a whole number from 1 | | `stage:` | a pipeline stage: key of letters, digits, `_`, `.` and `-`, up to 64 | | `case:` | an eval case: id of letters, digits, `_`, `.` and `-`, up to 128 | A rubric standard must name a rubric version that exists, or the verdict is 422 `unknown_rubric`. ### Rubrics [Section titled “Rubrics”](#rubrics) Rubrics are recorded with the binary, against the same database: ```sh superwitness rubric-add -id press -version 1 -name Press \ -scale '{"kind":"decision","options":["pass","fail"]}' -body-file press.md -created-by prn_human01 ``` All six flags are required: `-id`, `-version` (1 or more), `-name`, `-scale` (valid JSON), `-body-file` (a file holding the rubric text) and `-created-by` (the author’s principal id). `rubric-add` needs `SW_DATABASE_URL`. It runs any pending migrations first, over `SW_MIGRATE_DATABASE_URL` when that is set, then inserts the rubric over `SW_DATABASE_URL` and prints `rubric:press@1 recorded`. Give rubrics ids that a standard can name: letters, digits, `_`, `.` and `-`, up to 64. The printed standard is the exact value you can use in the next verdict. Version 1 is now fixed; when the rubric changes, version 2 gets its own record rather than borrowing version 1’s name. Rubrics are append-only, like verdicts. A version, once recorded, cannot be changed or recorded again; `rubric-add` fails with `rubric version already exists`. To change a rubric, record the next version and judge against `rubric:press@2`. ### Rubric scales [Section titled “Rubric scales”](#rubric-scales) A rubric’s scale is JSON. These shapes are recognised, and the SuperWitness app will offer an input for each: | Scale | Input | Value recorded | | ----------------------------------------------------------------- | ------------------------------------- | ---------------------------------- | | `{"kind":"decision","options":["pass","fail"]}` (2 to 20 options) | one button per option | `{"decision": "pass"}` | | `{"kind":"score"}` | a slider from 0 to 1 in steps of 0.05 | `{"score": 0.75}` | | `{"kind":"label","labels":["clear","muddled"]}` (1 to 50 labels) | a pick list | `{"label": "clear"}` | | `{"kind":"text"}` | a text area | `{"text": "…"}` | | `{"min":0,"max":1}` (any range, the older form) | a slider over the range | `{"score": …}`, rescaled to 0 to 1 | Each shape allows no other key. Options and labels must be unique, non-blank and at most 64 characters each. `rubric-add` records a rubric in any other shape but prints a warning, and `GET /v1/rubrics` shows its `recognised_scale` as `null`. `GET /v1/rubrics` and `GET /v1/rubrics/{id}/{version}` read rubrics back ([HTTP API](/build/api/#get-v1rubrics-and-get-v1rubricsidversion)). ## Subjects and who may judge them [Section titled “Subjects and who may judge them”](#subjects-and-who-may-judge-them) * A `run` or `attempt` subject must exist. SuperWitness checks with SuperPipeline and the AgentPod hub; a subject neither knows is 404 `subject_not_found`, and one it cannot check is 503 `subject_unresolved`. * An `eval_case_run` subject is not checked against any product. * **A non-human judge may not judge a run or attempt it executed**: 403 `self_judgement`. The executors are the run’s agent principal from SuperPipeline and each attempt’s agent principal from AgentPod. If SuperWitness cannot establish every one of them as a hub principal, because a source did not answer or an executor is not mapped to a principal yet, a non-human verdict on that subject is 503 `subject_unresolved` rather than accepted on incomplete information. * A human judge may judge any subject, including one they executed. The judge’s kind is never taken from the request. A human principal records as `human`, an agent as `agent`, and a service as `grader`. ## Corrections: append-only [Section titled “Corrections: append-only”](#corrections-append-only) Nothing about a verdict can change once it is written. In the database, triggers refuse `UPDATE` and `DELETE` on each row of `verdicts` and `rubrics`, and `TRUNCATE` on either table. The runtime role has only `SELECT` and `INSERT` on the two tables, and is not their owner, so it cannot drop the triggers either ([Install](/install/#postgres-roles)). A change of mind is a new verdict that names the old one in `supersedes`: ```json { "idempotency_key": "review-brd_01-run_01-2", "subject_kind": "run", "subject_ref": "superpipeline:brd_01/run_01", "standard": "stage:draft", "value": {"decision": "changes_requested"}, "supersedes": "vrd_…" } ``` * Only the judge of the old verdict may supersede it: anyone else gets 403 `not_original_judge`. * The correction must name the same subject and standard: 422 `supersedes_mismatch`. * The old verdict must exist: 422 `supersedes_not_found`. `supersedes` must look like a verdict id (`vrd_…`): 400 `invalid_supersedes`. * Each verdict is superseded at most once, so a chain stays a line. Superseding a verdict that already has a successor is 409 `already_superseded`; supersede the latest in the chain instead. The run document shows only the latest verdict in each chain. The correction is now the visible judgement, while the superseded record remains available in history. You can change your mind without changing what happened. ## Retries and idempotency [Section titled “Retries and idempotency”](#retries-and-idempotency) Every verdict carries an `idempotency_key`, and a key is used once across the whole store. Send the same request again with the same key, and SuperWitness returns the verdict it already stored, with 200 instead of 201, and writes nothing. Send a different verdict with a key that is already used, by you or anyone else, and it is 409 `idempotency_conflict`. So on a timeout or a retryable error, retry with the same key. The retryable errors are 503 `store_unavailable` and 503 `subject_unresolved`; both carry `"retryable": true` and a `Retry-After: 5` header. ## Evidence references [Section titled “Evidence references”](#evidence-references) `evidence_refs` points at evidence; it never holds it. It is an array of at most 100 entries, each either: * a span id: 16 lowercase hex characters, such as `"00f067aa0ba902b8"`; or * a range of session events: `{"session_id": "…", "seq_from": 1, "seq_to": 9}`, with `seq_from` 0 or more and `seq_to` not below it, and no other keys. Anything else is 400 `invalid_evidence_refs`. ## When the database is down [Section titled “When the database is down”](#when-the-database-is-down) `serve` runs its migrations in the background at start. Until they succeed, and whenever Postgres cannot answer, `POST /v1/verdicts` is 503 `store_unavailable`, run documents show `sources.verdicts` as `unavailable` (or `timeout`), and the rest of the document still renders. # With AgentPod and SuperPipeline > What SuperWitness reads from AgentPod and SuperPipeline, how it authenticates to them, and the tokens its own callers present. In this release, we build a run document from two product views: the run comes from SuperPipeline and its attempts come from the AgentPod hub. SuperWitness reads both over their APIs, as a service principal of the AgentPod hub, and owns none of what it reads. This page follows the credential, the routes and the status mapping so you can see every trust boundary rather than taking “the integration” as a mysterious third product. Neither product requires SuperWitness. They send plain OTLP to the collector and never wait on SuperWitness, and SuperWitness only ever reads from them. ## The service credential [Section titled “The service credential”](#the-service-credential) SuperWitness authenticates to both products with one AgentPod service credential: | Setting | Holds | | --------------------------- | ------------------------------------------------------------------------ | | `SW_HUB_URL` | the AgentPod hub, for example `https://hub.agentpod.dev` | | `SW_HUB_CLIENT_ID` | the credential’s id, `svc_…` | | `SW_HUB_CLIENT_SECRET_FILE` | a file holding the credential’s secret alone, readable by its owner only | Create the credential at the hub as a service principal for SuperWitness with the `evidence:read` and `transcripts:read` scopes. With `evidence:read` alone everything works except [transcripts](/use/transcripts/#who-may-read-content), which answer 502 `hub_refused`. SuperWitness refuses to start if the secret file is empty, or readable by group or others. Under the organization plane (`SW_ORG_PLANE_ISSUER` set), SuperWitness signs in with the plane service credential in `SW_ORG_PLANE_SERVICE_CREDENTIAL_FILE` instead, and `SW_HUB_CLIENT_ID` and `SW_HUB_CLIENT_SECRET_FILE` are not used ([Configuration](/use/configuration/#settings)). To read, SuperWitness exchanges the credential for a short-lived token: ```plaintext POST {SW_HUB_URL}/api/auth/service-token Authorization: Bearer : ``` The hub answers `{"token": …, "expiresIn": }`. SuperWitness keeps the token until 30 seconds before it expires, and uses the same token for every request to the hub and to SuperPipeline. If the hub refuses the credential (400, 401 or 403), the sources that need it report `unauthorized`. If a product answers 401 or 403, SuperWitness drops the token and exchanges again on the next read. ## What SuperWitness reads from AgentPod [Section titled “What SuperWitness reads from AgentPod”](#what-superwitness-reads-from-agentpod) From the hub at `SW_HUB_URL`, with the service token: | Request | For | | -------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | `GET /api/evidence/runs/superpipeline/{run}` | the run’s ledger: its attempts, each attempt’s fingerprint and agent principal, and the dispatch outcome | | `GET /api/evidence/attempts/{attempt}` | which run an attempt belongs to, for `GET /v1/runs/by-attempt/{attempt}` and attempt verdicts | | `GET /api/evidence/principals/{id}` | a principal’s kind, to show a gate’s judge and judge kind; `{id}` is a `prn_…` id or a hub user id | | `GET /health` | `/health`’s `agentpod` status (no token) | And, without a token, the hub’s published keys at `GET /api/auth/jwks`, to verify the tokens callers present (below). A run the ledger knows on a different board from the one asked for counts as not found. ## What SuperWitness reads from SuperPipeline [Section titled “What SuperWitness reads from SuperPipeline”](#what-superwitness-reads-from-superpipeline) From SuperPipeline at `SW_SUPERPIPELINE_URL`, for example `https://app.superpipeline.dev`, with the same service token: | Request | For | | ------------------------------------------------------------------ | ----------------------------------------------------------------------- | | `GET {SW_SUPERPIPELINE_URL}/v1/boards/{board}/runs/{run}/evidence` | the run, its card, stage and state, its agent, its gates, and its usage | | `GET /health` | `/health`’s `superpipeline` status (no token) | When SuperPipeline cannot answer, the run document falls back to what the AgentPod ledger knows about the run ([Concepts](/concepts/#run)). ## How answers become statuses [Section titled “How answers become statuses”](#how-answers-become-statuses) For both products: | Answer | Source status | | ------------------------------------------ | -------------- | | 200 with a body that parses | `ok` | | 401 or 403 | `unauthorized` | | 404 with the product’s own not-found error | `not_found` | | any other 404, or any other status | `unavailable` | | no answer within `SW_SOURCE_TIMEOUT` | `timeout` | | an answer about a different run | `unavailable` | The product’s not-found error is `{"error": "not_found"}` from the hub, and an error whose `code` is `RUN_NOT_FOUND` or `BOARD_NOT_FOUND` from SuperPipeline. A bare 404, such as a route that is not deployed, is `unavailable`, so a missing route never reads as a missing run. ## Tokens callers present [Section titled “Tokens callers present”](#tokens-callers-present) Every `/v1` route and `/mcp` take a bearer token from one issuer: the AgentPod hub, or, when `SW_ORG_PLANE_ISSUER` is set, the organization plane, and then hub tokens are refused. With the hub as issuer, SuperWitness verifies a token offline, against the keys from `{SW_HUB_URL}/api/auth/jwks`, and accepts it only when: * it is an EdDSA (Ed25519) JWT whose key id is among the hub’s published keys; * its issuer (`iss`) is `SW_HUB_URL`; * its audience (`aud`) includes `SW_PUBLIC_URL`; * it has an expiry (`exp`) that has not passed and an issued-at time (`iat`), with 30 seconds of clock leeway; * it names a subject (`sub`) and a `tenant`, and its `principalKind` is `human`, `agent` or `service`. Both URLs are compared as configured, less any trailing `/`. If the token has a `scope` claim, SuperWitness reads it as a space-separated list. Two things need a scope today: `POST /v1/runs` needs `runs:write`, from a service principal ([Run registry API](/build/run-registry/#who-may-report)), and reading transcripts with a token needs `transcripts:read` ([Transcripts](/use/transcripts/#who-may-read-content)). A `scope` claim that is not a string makes the token invalid. Under the organization plane (`SW_ORG_PLANE_ISSUER` set), SuperWitness follows the suite’s access-token rules, the same in every product’s verifier and tested against treaty’s signed token vectors. The issuer is that exact value and the keys come from `SW_ORG_PLANE_JWKS_URL`; only Ed25519 keys are used, and a token must name one by `kid`. It accepts a token only when: * it is an EdDSA JWT whose audience (`aud`) is, or includes, `SW_PUBLIC_URL`; * `exp` and `iat` are present, `exp` has not passed, and neither `iat` nor a `nbf` is in the future, each with 30 seconds of clock leeway; and it lives at most 300 seconds (`exp - iat`); * it carries `jti`, `principalKind` (`human`, `agent` or `service`), `org` and `ent` in place of `tenant`, and the control pair: `mayDispatch`, a list of `prn_` ids (any other value refuses the token), and `mayGrantReach`, a boolean; * `sub` and `org` follow the id grammar exactly: `prn_` or `org_` and 20 lowercase hex characters; * every optional claim it carries (`act`, `email`, `email_verified`, `scope`, `amr`, `orgRole`, `client_id`, `azp`, `sid`) has its type. SuperWitness reads `scope` only from agent and service tokens. A token whose `ent` claim lacks `superwitness` is answered `403 {"error":"product_not_enabled","org":"org_…"}`. Every such principal is admitted; which principals can get a token for SuperWitness’s audience is decided at the hub. The principal kind decides how its verdicts are recorded: their `judge_kind`, their default `kind`, and whether it may judge work it executed ([Verdicts](/use/verdicts/#subjects-and-who-may-judge-them)). SuperWitness keeps the hub’s keys for 10 minutes. A token with an unknown key id makes it fetch the keys again, at most once every 10 seconds. If the hub cannot be reached, keys it already holds keep working. A missing or refused token is 401 `unauthenticated`, with `WWW-Authenticate: Bearer realm="superwitness"`: ```json {"error":{"code":"unauthenticated","message":"the bearer token is not valid for this service"}} ``` With no token at all, the message is `a bearer token is required`. In fake mode (`SW_FAKE_SOURCES=1`) none of this applies: SuperWitness reads no product and accepts development tokens of the form `dev::` instead ([Install](/install/#try-it-without-the-other-products)). That gives you a clean local path for learning the document; switching to real sources then means replacing the fixtures with the exact reads and token checks above, not changing what a run means. # What SuperWitness is > Observability and evaluation for agent fleets. Three questions answered on one run, and why telemetry and evidence are kept apart. SuperWitness is observability and evaluation for agent fleets. Give us one run of work and we bring three views together for you: what the agent did, how the products around it behaved, and whether the result was any good. Generic observability tools stop at a trace. We follow that trace to the run it belongs to, the attempts that executed it, the agent configuration behind them, and the verdicts that judged it. That is how “did this configuration get better?” becomes a question you can answer instead of a hunch with a dashboard. ## Three questions, one run [Section titled “Three questions, one run”](#three-questions-one-run) | Question | Where the answer comes from | In the run document | | ----------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------- | | **What did it do?** | The attempts that executed the run, each with its configuration fingerprint, and the spans they left | `attempts`, `trace` | | **How did the products run?** | Log lines and errors that carry the run’s id or one of its trace ids | `log_count`, `errors` | | **Was it any good?** | Gate decisions read from SuperPipeline, and verdicts recorded in SuperWitness | `verdicts` | The run document is what `GET /v1/runs/superpipeline/{board}/{run}` returns. It is assembled at read time from every source, and it says which sources answered: a source that is down shows as `unavailable` or `timeout`, and the numbers it would have supplied show as `unknown`, never as 0. [Concepts](/concepts/) has the vocabulary. ## Telemetry and evidence are different things [Section titled “Telemetry and evidence are different things”](#telemetry-and-evidence-are-different-things) SuperWitness handles two kinds of data and keeps them apart: | | Telemetry | Verdicts | | ----- | ---------------------------------- | -------------------------------------------------------------------------------- | | What | Traces, metrics, logs | Grader scores, offline eval results, calibration runs, configuration comparisons | | Loss | Expected: sampled, short retention | Never | | Where | Upstream engines | SuperWitness’s own database | Telemetry lives in VictoriaTraces and VictoriaLogs, sent there through the OpenTelemetry Collector. SuperWitness only queries it. Verdicts live in SuperWitness’s own Postgres, where they are append-only: database triggers refuse to update, delete or truncate them, and a change of mind is a new verdict that supersedes the old one. A missing span is not missing evidence. When a run has no spans, its trace status is `none` and the rest of the document still stands. Evidence that belongs to another product, such as a work run, a gate decision or a transcript, stays in that product. SuperWitness reads it through that product’s API and never becomes its owner. ## The rules the project keeps [Section titled “The rules the project keeps”](#the-rules-the-project-keeps) 1. **It is meant to stand alone.** The design rule is that no other SuperJackfruit product should be required, so that anyone’s agents can send OTLP and record verdicts. This release is not there yet: it reads runs from SuperPipeline and attempts from AgentPod, and without them it runs only in fake mode (below). 2. **Nothing requires it.** Products send plain OTLP and never wait on SuperWitness. 3. **Upstream engines, unforked.** SuperWitness queries VictoriaTraces and VictoriaLogs, fed by the OpenTelemetry Collector, through their published query APIs only. It is designed to sit beside VictoriaMetrics and Perses the same way: upstream releases, run as released, never forked. 4. **Permissive licences only** in anything linked or shipped: MIT, Apache-2.0, BSD. No AGPL, so no Grafana, Loki, Tempo or Mimir, and no source-available licences (ELv2, BSL, FSL, SSPL). Bundled components are credited in the `NOTICE` file in every release. 5. **Agents first.** It is evaluation and observability for agent fleets, with product health underneath. It is not a general monitoring platform that also handles agents. ## What this release reads [Section titled “What this release reads”](#what-this-release-reads) Run documents in this release are for SuperPipeline runs, addressed as `superpipeline:/`, with attempts read from the AgentPod hub. Start SuperWitness against real sources and it asks you for both products’ URLs and an AgentPod service credential. If you want to explore without either one, use `SW_FAKE_SOURCES=1`; you will get one development run on loopback and can inspect the shape before wiring the real sources. The [install guide](/install/#try-it-without-the-other-products) shows both paths.