Graph Engineering: A practical guide to building agents that branch, verify, recover and stop
Your agent follows the prompt perfectly.
It still fails.
One researcher misses the source. A second repeats the same mistake. The reviewer sees a polished answer and approves it. By the time the result reaches you, nobody can explain which decision poisoned the run.
That is not a prompting problem.
It is a control-flow problem.
Graph Engineering is the practice of designing the shape of an AI job before asking models to execute it. You decide what can run in parallel, what must wait, what evidence crosses between steps, where failure goes, and which decisions still belong to a human.
The model does the reasoning.
The graph decides how reasoning becomes work.
A conversation hides the architecture
Chat makes every task look linear:
Research -> analyze -> write -> review
But real work rarely has that shape.
Product research and pricing research can run at the same time. A security review should not share the writer's assumptions. A failed source check should return to research, not restart the whole job. A high-risk action may need human approval while routine actions continue.
Once those relationships matter, one long conversation becomes the wrong abstraction.
Anthropic makes a useful distinction: workflows follow predefined code paths, while agents choose their own process and tools at runtime.
Graph Engineering lets you combine both. Keep predictable decisions in code. Give uncertain decisions to models.
This is the first rule:
Do not use an LLM to decide what ordinary code already knows.
If three tasks are independent, start all three. If a score is below 0.8, route it to review. If the retry counter reaches three, stop. Those are graph rules, not reasoning tasks.
The four parts of a useful graph
You do not need graph theory to build one. You need four things.
1. State
State is the job's memory outside the chat window.
It should contain facts the workflow needs to resume or route: the original brief, collected evidence, node status, retry counts, approvals, budgets, and final artifacts.
Do not treat the entire transcript as state. Most of it is conversational debris.
2. Nodes
A node owns one bounded job.
`collect_pricing` is a node.
`verify_claims` is a node.
`write_report` is a node.
"Research everything, decide what matters, write the report, and make sure it is correct" is not a node. It is a hidden workflow stuffed into one prompt.
3. Edges
An edge answers one question: what is allowed to run next?
Some edges are fixed. Others depend on state.
4. Gates
A gate blocks bad work from moving downstream.
It can be a test, schema validator, permission check, deterministic rule, evaluator model, or human approval. A graph without gates is just a faster way to spread mistakes.
Build the graph around failure, not the happy path
Most workflow diagrams show how everything succeeds.
Production systems are defined by what happens when it does not.
Before adding agents, write down five outcomes for every important node:
This changes how you prompt the model. Instead of asking, "Is this good?", the verifier returns a route:
{
"decision": "retry",
"reason": "Two revenue claims have no primary source",
"target": "collect_company_data"
}The output is useful because the graph can act on it.
Anthropic recommends grounding agents in environmental feedback and setting stopping conditions such as maximum iterations.
That matters because agent errors compound: one weak assumption becomes context for the next step, then evidence for the step after that.
A graph for a real research brief
Suppose the task is:
Compare three AI coding products and produce a sourced recommendation for a 20-person engineering team.
A single-agent version searches, reads, compares, and writes inside one context window. It is simple. It also mixes discovery, judgment, and prose, making failures hard to locate.
A graph can separate the work:
Node 1: Scope
Turn the request into explicit criteria: price, privacy, deployment, model support, administration, and migration cost.
The output is a schema, not an essay.
Nodes 2-5: Collect evidence
Run independent workers for documentation, pricing, security, and user evidence.
Each worker returns the same shape:
{
"claim": "The enterprise plan supports SSO",
"source": "https://...",
"source_type": "official_docs",
"published_at": "2026-07-12",
"confidence": "high"
}Node 6: Normalize
Use code to deduplicate URLs, reject missing fields, standardize dates, and group claims by product.
No model call is required.
Node 7: Challenge
Give the strongest claims to a verifier that tries to disprove them. It should search for contradictory documentation, stale pricing, region restrictions, and missing caveats.
Node 8: Human gate
Only unresolved contradictions and high-impact recommendations reach the human. Routine evidence keeps moving.
Node 9: Synthesize
The writer receives verified evidence, decision criteria, and unresolved caveats. It never sees the raw research transcript.
That last detail matters. Context should follow the edges of the graph, not accumulate in one giant window.
Anthropic has shown a related pattern with code execution and MCP: intermediate data can remain in the execution environment while the model sees only what is explicitly returned. This can reduce context load, latency, and unnecessary exposure of sensitive data.
Verification needs its own branch
Do not ask the same agent to create and approve its own work in the same context.
It already knows why it made each choice. That makes it a poor skeptic.
A stronger verification branch has different information and a narrower job:
The verifier should return evidence, not vibes.
Bad:
This looks accurate and well supported.Better:
{
"pass": false,
"failed_rule": "primary_source_required",
"unsupported_claims": [3, 7],
"next_action": "research_again"
}Evals are not something to bolt on after launch. Anthropic notes that early evals force teams to define success, while later they provide baselines for quality, latency, token usage, cost, and regressions.
In a graph, those evals become gates.
Every graph spends three budgets
Agent graphs can improve quality while quietly destroying latency and cost.
Track three budgets per run:
Time
Parallel branches reduce wall-clock time only when they are truly independent. A join forces every downstream node to wait for the slowest branch.
Tokens
Every extra worker, judge, retry, and synthesis call adds cost. Pass structured evidence between nodes instead of entire transcripts.
Risk
Not every action deserves the same autonomy. Reading documentation and sending a payment should not share one permission policy.
The graph should make those tradeoffs visible. Add per-node token caps, workflow deadlines, retry limits, and permission levels to state.
When a budget is exhausted, return the best partial result with a clear failure report. Do not let the agent improvise its way into an infinite loop.
When you should not build a graph
Graph Engineering is not a reason to turn every prompt into infrastructure.
Use one model call when the task is short, low-risk, and easy to inspect.
Use a simple chain when every step genuinely depends on the previous one.
Reach for a graph when at least one of these becomes true:
Anthropic's advice is blunt: start with the simplest solution and add complexity only when it measurably improves outcomes. Graphs are useful because they expose complexity. They are not useful when they manufacture it.
Your first graph can fit on one screen
Start with a task you already run repeatedly.
Draw the current process as boxes. Then annotate each arrow with the actual data it carries. Mark every decision, external action, failure route, and human approval.
Now reduce it:
Frameworks can help, but the diagram comes first.
The framework is replaceable.
The decisions in your graph are the product.
The shift
Prompt engineering asks:
What should the model say or do next?
Graph Engineering asks:
What information should exist now, who should act on it, what proves the result, and where does failure go?
That is a harder question.
It is also the one that turns a clever demo into a system you can trust twice.
Thanks for reading.
Bookmark this before your next "just add another agent" meeting. And follow @iiiichigo_chan
