The software factory needs a faster review loop: further optimizing the path from PR to merge
TL;DR
In an AI-native engineering organization, generating code is no longer the slowest part of shipping software. Review, verification, repair, and human decision-making become the bottlenecks, and ultimately define how much faster the organization can move. Standalone AI coding tools may deliver a 30–40% speedup, but achieving a step-change in throughput requires optimizing everything that happens after the code is written.
In our previous post about solving code review with Cosmos, we described a review system that helped our engineering organization increase code output by 3× while reducing median merge time and maintaining quality. We have since expanded that system beyond review into the full PR-to-merge loop: specialized agents handle mechanical work, verify correctness, and address feedback, while humans provide judgment and knowledge transfer, and always own the final merge decision.
This is loop engineering: improving the complete system that turns generated code into a verified, understood, merge-ready change rather than optimizing any single tool in isolation.
Why an “AI code review tool” is not enough
A conventional AI code review tool like CodeRabbit or Greptile analyzes a diff and posts comments. Useful, but review is only one step. The real drag is the chain of human handoffs:
A PR-to-merge loop keeps specialized Experts working through review, repair, verification, and decision support until a human can merge with confidence. Each Expert in the fleet owns one or more of these handoffs, turning a fragmented sequence of manual tasks into a coordinated system. The goal is to optimize the entire loop: human time, cost, quality, and merge latency, not time to first comment.
1. Overview: engineering the complete loop
Our original system separated risk analysis (Risk Analyzer), line-by-line correctness review (Deep Reviewer), and human-guided design review (Pair Review). The expanded system adds repair, runtime verification, and expanded auto-approval, reducing the human bottleneck further. For an overview of Cosmos and its configurable Experts, see our previous post about solving code review with Cosmos.
Experts and capabilities, not one general-purpose reviewer
| Expert or capability | Responsibility |
|---|---|
| Risk Analyzer | Classifies risk and applies auto-approval policy |
| Deep Reviewer | Performs exhaustive, line-by-line analysis for objective correctness defects |
| Pair Reviewer | Reconstructs intent, architecture, product context, and tradeoffs |
| Memory Manager | Remember feedback from the PR and Pair Review sessions to improve future runs |
| VerifierNEW | Exercises affected behavior end-to-end in a test environment (Verifier blog) |
| PR FixerNEW | Repairs review findings, CI failures, and merge conflicts |
| Review DashboardNEW | Observes and summarizes expert state |
cosmos approveNEW | Evaluates a configurable approval policy on author request |
The distinction between Deep Reviewer and Pair Reviewer is particularly important:
The PR Author Expert previously did double duty: writing a PR and fixing review comments, CI failures, and other follow-up work. In the decoupled design, PR Author stops at draft PR creation and PR Fixer takes over. This gives users more control over how fixes run and supports PRs created without PR Author.
Human review and verification are becoming the scarce resources in AI-native engineering organizations. The goal is not to remove humans indiscriminately. It is to spend human attention only where it has the highest leverage.
Why humans remain in the loop
Agents can perform a large share of mechanical analysis and execution, but they do not have complete business and organizational context. Humans remain essential for:
The design target is therefore:
Agents do the mechanical work. Humans make the consequential decisions.
The author and reviewer workflow: before and after
| Traditional workflow | Human-in-the-loop workflow |
|---|---|
| Manually triage every PR and identify low-risk changes | Let Risk Analyzer classify risk and apply the organization’s auto-approval policy |
| Read the PR line by line | Trust Deep Reviewer to perform exhaustive line-by-line analysis |
| Reconstruct context, intent, and architecture from the diff | Use Pair Reviewer’s briefing to understand the change and identify judgment calls |
| Manually deploy and exercise the feature | Inspect Verifier evidence: screenshots, logs, traces, and captured outputs |
| Re-review every fix from scratch | Let Pair Reviewer monitor whether authorized comments were addressed |
| Triage feedback, implement fixes, repair CI, resolve conflicts, and explain each change | Let PR Fixer—or PR Author when it owns the full lifecycle—handle the mechanical follow-up and report what changed |
| Search across comments and checks for current state | Use the Review Dashboard as the entry point |
| Manually gather review, ownership, and verification evidence before requesting approval | Invoke cosmos approve to evaluate the configured approval policy against current evidence |
| Decide whether to merge | Still decide whether to merge |
The economic intuition of using multiple experts
Tokens that measurably reduce human time in the review bottleneck are worth the investment. When Experts turn hours of review and PR shepherding into minutes, they free scarce engineering judgment for the consequential decisions only humans can make, while also helping features reach customers faster.
One of Augment’s core commitments is helping organizations optimize costs. That means optimizing total cost per task: human effort plus token cost. It does not mean minimizing token usage at the expense of successful outcomes. A single Expert burdened with six responsibilities will do a substandard job of each and require more human intervention. Six dedicated Experts can each focus on a different part of PR review, produce higher-quality review artifacts, and drive more of the process autonomously.
3. Cost and quality: optimize cost per successful outcome
The cheapest model by token price is often a false economy. A missed defect, bad repair, or retry can cost more than getting the task right once. We benchmark representative work and choose the lowest-cost model that clears each Expert’s quality bar, the same cost-per-success principle we use across Cosmos.
Today, we use GPT-5.6 Sol for judgment-heavy work such as Risk analysis, Deep and Pair review, and code repair. Bounded, mechanically verifiable tasks such as dashboard aggregation and merge-conflict monitoring run on GPT-5.6 Luna. Models with longer cache TTLs also favor long-running agents because cached input is typically discounted at 90%.
4. A configurable approval path with cosmos approve
The Risk Analyzer has always been able to approve intrinsically low-risk changes under a conservative policy. We now support a second, disabled-by-default approval mode for other changes.
The PR author can comment cosmos approve to request an evaluation against an approval policy. A human still owns and performs the final merge.
Organizations define their own approval policy. Our internal policy checks:
5. Customizability is part of the architecture
We designed the fleet so teams can customize the loop with Cosmos Advisor.
The operating model for AI-native PR Review
The lesson from our first version was that code review could not scale by asking humans to read AI-generated code faster. The lesson from this version is broader: no individual review agent can optimize the full path to merge.
A high-functioning PR-to-merge loop needs:
This is loop engineering applied to the PR lifecycle: optimize the system that produces a verified, understood, merge-ready change, not the amount of output produced by any single agent.
Build your own PR-to-merge loop
Cosmos gives engineering teams the shared context, runtime controls, integrations, and human checkpoints to run agents across review, verification, repair, and the rest of the software lifecycle.
