The app
The API is not the only way in. SuperWitness serves a web app from the same binary and address, so you can browse reported runs, open one, record a verdict against a rubric and revise it without building a client first. People on an allowlist sign in with their AgentPod account; agents and scripts keep using the HTTP API and the MCP tools with bearer tokens.
Turning it on
Section titled “Turning it on”The app is off until someone is allowed to sign in. Four settings turn it on (Configuration):
| Setting | Meaning |
|---|---|
SW_ALLOWED_PRINCIPALS |
comma-separated AgentPod principal ids (prn_…) who may sign in. Empty, nobody may, and the app is off. |
SW_APP_CLIENT_ID |
the AgentPod OAuth client browsers sign in through. |
SW_SESSION_SECRET_FILE |
a file holding at least 32 random bytes, readable by SuperWitness alone. It keys the short-lived cookie that carries a sign-in to AgentPod and back. |
SW_TRUSTED_PROXIES |
addresses or CIDR prefixes allowed to say who a client is with CF-Connecting-IP. Unset, it is this host’s own addresses. |
SW_PUBLIC_URL must be https when sign-in is on: the app’s cookies are Secure, and a browser
drops them over plain http. (Fake mode, which listens on loopback only, is the exception.) Make
the secret with, for example:
head -c 48 /dev/urandom | base64 | sudo tee /etc/superwitness/session-secret >/dev/nullsudo chmod 0600 /etc/superwitness/session-secret && sudo chown superwitness /etc/superwitness/session-secretThe cookie key is now private to the service. Once the OAuth client has the callback and audience below, an allowlisted person can complete sign-in instead of meeting a very secure dead end.
At AgentPod, the OAuth client named by SW_APP_CLIENT_ID must list {SW_PUBLIC_URL}/auth/callback
as a redirect URI and SW_PUBLIC_URL as an audience. SuperWitness also needs its own service
credential to hold evidence:read, as it already does to read attempts: it uses it to look up
who signed in.
Signing in
Section titled “Signing in”Sign in sends the browser to AgentPod, which signs the person in if they are
not already and sends them back to /auth/callback with a one-time code. SuperWitness trades
the code for a token server to server, with PKCE, checks the token, and looks up the AgentPod
principal behind it. It then checks that the principal is a person, is not suspended, and is on
SW_ALLOWED_PRINCIPALS. Anyone else, including other valid AgentPod accounts, agents and
services, sees Not authorised. Each refusal is logged as auth.signin_refused with a reason,
for example not_allowlisted, login_missing (the sign-in cookie was absent) or no_code (the
callback carried no code). A sign-in that succeeds is logged as auth.signin with the
principal. The log never holds a token or a cookie.
With SW_ORG_PLANE_ISSUER set, the app signs people in through the organization plane as the
OAuth client named by SW_APP_CLIENT_ID (the plane registers it with the redirect
SW_PUBLIC_URL/auth/callback). Both the authorize and the token request carry
resource=SW_PUBLIC_URL, and the authorize asks for scope=openid offline_access. The token’s
sub is the person’s prn_ id, checked against SW_ALLOWED_PRINCIPALS with no lookup. Three
more refusal reasons apply: product_not_enabled (the person’s workspace has not enabled
SuperWitness), issuer_unavailable (the plane did not answer) and no_refresh_token (the plane
issued no refresh token). The pages then name the plane’s host instead of AgentPod.
Under the plane the session holds the plane’s refresh token, server side: it is encrypted in the database with a key derived from the session cookie, which the database never holds, and it never reaches the browser. That makes SuperWitness one of the person’s connected apps at the plane, and the session lives only as long as that grant:
- when the plane’s access token is past its five-minute life, the next request refreshes it with the stored refresh token and keeps the rotated one. Requests on one session share a single refresh. If the new refresh token cannot be saved, the request answers 503 and a retry within the plane’s 30-second replay window gets the same answer back;
- if the plane refuses the refresh (the person signed SuperWitness out at the plane, left the
workspace, or the workspace turned SuperWitness off), the session ends at that request and the
app shows the sign-in screen (
auth.session_ended, reasongrant_refused); - if the plane cannot be reached, the session carries on, retrying every 30 seconds and logging
auth.refresh_unavailable. It ends if the plane is still unreachable 10 minutes after the first failure (reasonissuer_unreachable); - a session signed in before sessions held a grant ends at its next request (reason
no_grant); signing in again is silent while the plane’s own session lasts.
The sign-in itself rides on a signed __Host-sw_login cookie that lives ten minutes and is used
once. A signed-in browser then holds a __Host-sw_session cookie and a SuperWitness session,
never an AgentPod token. Over plain http (fake mode only) the cookies are sw_login and
sw_session, without the __Host- prefix.
- the session lasts at most 12 hours, and ends after 2 hours unused;
- every request checks the allowlist again, so taking someone off it (and restarting SuperWitness) ends their access at their next click;
- Sign out (
POST /auth/logout) ends the session at once and clears the cookie. Under the plane it first revokes the session’s refresh token at the plane’s revocation endpoint (found through the plane’s discovery document, and used only on the plane’s own origin); a revocation that fails is logged asauth.revoke_failedand the session ends all the same. If the database cannot be reached the answer is 503store_unavailable, with the cookie cleared all the same, and the session expires on its own. A sign-out is logged asauth.signoutwith the principal, and one the database refused asauth.signout_failed. Signing in again from a browser that still holds a session ends the old one first, logged the same way with reasonsigned_in_again; - sessions do not need AgentPod: if AgentPod is down, open sessions keep working and only new sign-ins wait.
Changes made with a session must come from the app’s own page (their Origin must equal
SW_PUBLIC_URL); anything else is refused with 403 origin_mismatch. Requests with a bearer token
are not affected, and a request that carries a bearer token is judged by the token alone. Run
reports (POST /v1/runs) and /mcp never accept a session.
- Runs: every run sources have reported, newest first, grouped under Today, Yesterday and earlier dates in your own time zone. The sidebar has All, Needs verdict (finished runs no verdict names yet), Failed and Waiting, and a view per board. Each card shows the title, the status, the board, who ran it, how long it took or when it started, and its latest verdict. The filters are in the address, so a view can be bookmarked. With no runs yet, the page links to the Run registry API.
- A run: its source and reference, title, status and Record verdict; a strip with the
agent, attempts, errors, duration and cost; and tabs for the trace (a span waterfall, 500 spans
at a time), logs (with a level filter), errors, verdicts (with revisions shown as history: a
superseded verdict is struck through and links to the one that replaced it) and attempts (each
with its configuration fingerprint). A source that did not answer shows as unavailable, and a
value no source supplied reads
unknown, never 0. A run missing from the list still opens by its address,/runs/superpipeline/{board}/{run}. - Rubrics: every rubric version, and each one’s scale and text. Rubrics are added with
superwitness rubric-add.
The sidebar’s dot is green when every source answers /health, amber otherwise; hover for which.
The app has four vibes (Daylight, Paper, Studio and Quiet), each in light and dark. Open the account
chip, then Appearance, to choose the vibe, to follow the sun, stay light or stay dark, and to pick an
accent; the choice is kept for this browser. On a phone the sidebar folds into a menu.
What an agent did
Section titled “What an agent did”Click a span’s name in the Trace tab for its details, including the request and response where the span maps to the agent’s session; the Transcript tab reads the attempt as a conversation. Either can be cited in a verdict. Transcripts has the details.
Recording a verdict
Section titled “Recording a verdict”Record verdict opens a drawer. Choose the subject (the run, or one of its attempts) and a rubric, then give the value the rubric’s scale asks for (rubric scales): a decision, a score from 0 to 1, a label, or text. Add a comment of up to 10,000 characters if you like. Spans ticked in the Trace tab, up to 100, go with it as evidence. A rubric whose scale the app cannot offer an input for is listed but cannot be chosen.
The drawer makes its idempotency key when it opens, so a double click or a retry records one verdict. Revise appears on your own latest verdicts: it opens the drawer with the same subject and rubric and records a verdict that supersedes the old one. The app explains refusals in plain words, for example “You’ve already revised this verdict” when someone revised it first.
After you save, the verdict appears on the run and in the registry’s latest-verdict view. A revision keeps the old record visible in history, so correcting a judgement does not airbrush the first one.
Rate limits
Section titled “Rate limits”SuperWitness limits requests in its own process, by client address or by principal:
| What | Limit |
|---|---|
/auth/* |
10 a minute per client address, bursts of 20 |
| changes made with a session | 30 a minute per person, bursts of 10 |
POST /v1/runs |
600 a minute per reporter, bursts of 100 |
Over a limit the answer is 429 rate_limited with Retry-After. The client address is the
connection’s, unless the connection comes from SW_TRUSTED_PROXIES, in which case it is the
CF-Connecting-IP header.
Putting it on the internet
Section titled “Putting it on the internet”SuperWitness serves the app, the API and /mcp on one address. To publish the app through a
tunnel or a reverse proxy, point the proxy at SuperWitness from an address in
SW_TRUSTED_PROXIES and have it set CF-Connecting-IP. A request that arrives that way, through
the public edge, is answered 404 on /mcp before any authentication, so the MCP tools stay on
your private network. Callers on the private network, which carry no CF-Connecting-IP, are
unaffected. Set SW_PUBLIC_URL to the public https address, and
add that address as an audience to every AgentPod client whose tokens call SuperWitness, before
you change it, so that tokens carry both the old and the new audience during the switch.
When something is down
Section titled “When something is down”| What | What you see |
|---|---|
| AgentPod, during sign-in | “AgentPod sign-in is unavailable”; open sessions keep working |
| a source | that run’s tab says it is unavailable; the sidebar dot turns amber |
| SuperWitness’s database | “Can’t reach the database” |
| a link to a run no source knows | “Run not found” |