PR Review Shift Log: Agent Findings, Checklist Verdicts

The review queue has two tempting shortcuts: ask a coding agent to inspect the patch, or run a written checklist and trust the reviewer to spot everything important. Neither deserves an automatic approval stamp. They solve different problems, and their weaknesses become visible when a pull request crosses service boundaries, changes permissions, or touches data migrations.

This is a workflow comparison, not a model benchmark. The question is which approach gives the reviewer useful evidence, which preserves accountability, and where combining them earns its extra effort.

The review desk needs the same evidence on both sides

Give both approaches the same packet: the diff, the intended behavior, relevant repository guidance, test results, and any rollout constraints. Otherwise, the comparison measures access to context rather than review quality.

Separate candidate findings from verified defects. A useful finding identifies a location, explains a plausible failure, and describes how to check it. An agent comment that sounds authoritative is not evidence; neither is a checked box beside an unexamined requirement.

The coding agent wins at expanding the search

A coding agent can inspect callers, follow changed types, suggest edge cases, and draft a regression test. That makes it useful when the patch looks small but its dependencies are not. Ask it to trace consequences beyond the edited lines rather than produce a general impression.

A practical instruction is: inspect the diff and relevant callers, report only actionable correctness or security concerns, cite file locations, explain the failure path, and distinguish observed evidence from assumptions. Require the agent to say when missing context prevents a conclusion.

The weakness is calibration. Agents can misunderstand framework behavior, overlook operational assumptions, or report speculative defects. Tool access also matters: reading code is different from executing repository scripts. Start with read-only access, withhold secrets, and require approval before commands with side effects.

Verdict: The agent is the stronger first-pass investigator for unfamiliar code, but its findings remain review inputs, not merge authority.

The written checklist wins at retaining obligations

A checklist makes recurring duties explicit. It is particularly valuable for concerns that may not be obvious in the diff: deployment order, compatibility, observability, and recovery. Its strength is consistency, not discovery.

• Confirm that the implementation matches the stated behavior and handles relevant failure paths.
• Check authorization at the enforcement point, including negative cases for denied access.
• Inspect changed interfaces and identify callers that depend on the previous behavior.
• Verify that tests cover the changed behavior and reproduce any confirmed regression.
• Check whether logs, metrics, and errors will make production failures diagnosable without exposing sensitive data.
• Document deployment dependencies and the rollback or recovery path for stateful changes.

Each item should end with evidence or an explicit explanation of why it does not apply. “Tests passed” does not establish that the right cases exist. Likewise, a rollback checkbox is insufficient when a migration destroys information.

Verdict: The checklist is the stronger control for repeatable obligations, but it cannot substitute for tracing the actual code.

The combined pass needs a strict handoff

Start with the human-written checklist, then give the agent a bounded investigation. Have the reviewer verify each substantive finding against code, a focused test, or documented system behavior. Record rejected findings briefly when the reasoning could matter to the next reviewer.

Finish by revisiting the checklist independently. Do not let a long agent report crowd out release safety or business intent. The responsible human still decides whether the evidence supports merging, requesting changes, or escalating to a domain owner.

The relay desk fits tool choice, not approval policy

If your team uses multiple agent clients, Relay365 is worth considering as a pay-as-you-go relay desk across model families. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Its API base is https://api.api365relay.com; OpenAI clients add /v1. Validate repository-data handling, tool compatibility, and costs before routing reviews through it. Create a Relay365 account if that setup fits your workflow.

The shift closes with three clear decisions

Choose agent-first review for exploratory investigation. Choose checklist-led review for consistent coverage of known obligations. For consequential changes, combine them: the agent broadens the search, the checklist preserves duties, and the reviewer owns the final judgment.