Skip to content

Agent-readable PR checks: a lab-computed aggregate over both forge backends, polled client-side

An agent that pushes a PR and opens it has no way to find out whether CI passed — labctl pr stops at create | view | list | merge, so the fix-the-red loop (push, wait for CI, read what failed, fix, repeat) currently has no agent-side leg to stand on. The agent either merges blind or asks the human to paste in a CI status by hand. This ADR closes that: the Tracker seam grows Checks(ctx, number) ([]Check, error), reachable as labctl pr checks <n> over the run-token agent surface (ADR-0014), so a session can wait out its own PR's CI and diagnose a red gate without leaving the tool-call loop.

Each backend resolves the PR's HEAD SHA at query time — never a cached sha from create — so a checks call always reads the commit currently on the branch; an unknown PR number wraps tracker.ErrNotFound exactly like every other pull lookup. A Check row is Name, a normalized State, RawState (the backend's own word, verbatim — the ADR-0011/0024 pattern of surfacing a backend's own vocabulary rather than lab's paraphrase of it), Summary, and URL — the URL exists for a human reading the transcript after the fact, not for the agent, which cannot browse the forge.

The pins, decided:

  • The aggregate is a pure, table-tested function, tracker.ChecksState in internal/tracker/checks.go, computed server-side and shipped in the payload — no backend re-derives it, and no client re-derives it either. Vocabulary: pending | success | failure | none. none holds iff zero rows, and it is computed lab-side, never trusted from Forgejo's own combined state: Forgejo reports pending for a commit with zero statuses, which would spin a waiting agent forever on a PR that simply has no CI configured. Precedence is failure > pending > success — a red row is actionable the moment it appears, even while other jobs are still running, which is a deliberate departure from forge-UI combined-state semantics (a badge answers "is everything done and green yet?"; this aggregate answers "is there something to fix right now?", the question the consumer actually acts on). Defensively, any row-state outside the three-word row vocabulary aggregates as failure — a vocabulary bug must be loud, never silently green.
  • Forgejo backend: pull → head.sha → the combined-status endpoint /commits/{sha}/status, mapping only its statuses rows into Checks. Normalization: success → success, pending → pending, everything else (error, failure, warning, and anything unrecognized) → failure.
  • GitHub backend: the union of the Checks API (check-runs — GitHub Actions is invisible to the legacy status API alone) and the legacy combined-status API (external CI still reports there). Conclusions normalize: neutral/skipped → success; cancelled/timed_out/action_required → failure; queued/in_progress → pending; anything unrecognized → failure. The two APIs are not deduplicated — rows are keyed by source+name, so a system that reports to both check-runs and the legacy status yields two rows for one logical check. There is no stable join key across the two shapes, and a doubled row is harmless to an agent reading a list, so no de-dup is attempted. Rate limits surface as tracker.ErrRateLimited per ADR-0015 (typed, no in-client retry); the agent API now maps that error to 503, matching the operator API's existing httpapi mapping for the same sentinel.
  • Builtin backend: no CI machinery exists behind it, so Checks returns zero rows unconditionally — aggregate none. That is a truthful success-shaped answer, not an error: an empty result is "nothing to block on," never a failure to fetch.
  • Wire: GET /agent/v1/prs/{n}/checks, run-token scoped like every agent action (ADR-0014) — repo scope comes from the run row, never from the URL. Response body: {state, checks: [{name, state, rawState, summary, url}]}. An unknown PR number answers 404 via ErrNotFound.
  • CLI: labctl pr checks <n> prints state: <aggregate> then one tab-separated row per check (name, state, rawState, summary, url) — parseable, matching the rest of labctl's output style. --wait polls client-side, inside labctl itself, every ~10s with a hard cap of ~5 minutes — comfortably under the agent Bash-tool's timeout ceiling, because the harness blocks a foreground sleep, so the only way to block on in-flight CI is to do the waiting inside one tool call rather than ask the agent to poll in a loop. On hitting the cap, labctl prints the still-pending state and exits distinctly rather than pretending success or looping forever. Exit codes for this verb specifically: 0 success/none · 1 usage/transport/API error · 2 aggregate failure · 3 still pending at exit. This deliberately folds usage errors into 1 — the binary-wide convention elsewhere reserves 2 for usage errors — so that for this one verb, exit 2 means exactly one thing an agent needs to branch on: the checks are red.
  • Explicitly not a merge gate. pr merge is unamended: it stays attempt-and-surface per ADR-0024, and Checks is a read the agent chooses to act on, not a pre-flight lab imposes.

Out of scope, deferred: fetching job logs through lab (a server-credentialed proxy of provider job logs) — the per-check name is enough for an agent to reproduce the failure locally today, and this is the natural next ADR if that proves insufficient; a web-UI badge; reaper coupling (the reaper's done-signal stays PR-existence, unchanged); and any operator-surface equivalent.

Status

Accepted. Extends ADR-0024's agent PR surface (pr checks joins create | view | list | merge) and ADR-0014's run-token surface (repo-scoped, run-token-authenticated, identical for interactive and AFK sessions). Applies ADR-0015's rate-limit discipline (tracker.ErrRateLimited, typed, no in-client retry) and the ADR-0011/0024 backend's-own-words pattern (RawState verbatim). The aggregate function is table-tested in internal/tracker/checks.go; each backend's mapping is covered where the backend itself is tested.

Considered options

  • Trust the forge's own combined verdict instead of computing an aggregate lab-side. Rejected: Forgejo's combined status reports pending for a commit with zero statuses, which would spin a waiting agent forever on a PR with no CI configured at all — none has to be a fact lab establishes from an empty row set, not a word borrowed from the forge. More generally, forge-UI combined-state precedence answers "is everything green yet," the wrong question for a consumer that needs to act on the first red row it sees, not wait for the whole set to settle.
  • Deduplicate GitHub check-runs against the legacy combined-status rows. Rejected: there is no stable join key between the two APIs' shapes (no shared id, and names collide only by convention, never guaranteed), so any de-dup heuristic would be guessing. A doubled row for a system that reports through both APIs is harmless to an agent reading a list — it just sees the same check named twice — so the correct fix is no fix.
  • Make checks a merge gate or a pre-flight inside pr merge. Rejected: ADR-0024 already pinned pr merge to attempt-and-surface, explicitly rejecting agent-side pre-flighting of mergeability because the forge's state races anything lab could cache. Mergeability stays the backend's call; checks is a separate read an agent consults before deciding to merge, not a gate lab enforces for it.
  • A server-side --wait (a long-polling route on the agent API). Rejected: a stateful long-poll route complicates the agent API for no gain the client can't provide itself. Polling client-side inside labctl keeps the route a plain synchronous read and keeps the blocking behavior inside the one tool call the harness already accounts for, rather than pushing an open connection through the server.
  • Fetch job logs through lab (a server-credentialed proxy of CI log output). Deferred, not rejected: today the per-check name is enough for an agent to reproduce a failure locally by rerunning the equivalent step, so the added surface (auth, size limits, per-provider log formats) isn't justified yet. Worth its own ADR if the per-check name proves insufficient in practice.