Anthropic Graph Engineering: 10 steps to the official team workflow (full course)

Anthropic shipped a knowledge graph cookbook. Nobody has broken down these stages from the official team.

The evaluation README states the expected entity recall is 0.70 to 0.85. The guide's own run came back at 0.55 and 0.38.

Same repository. Same pipeline. The shipped output lands at roughly half the documented baseline, and precision comes in at 1.00 against an expected 0.80 to 0.90.

On the Apollo 11 article the extractor missed Apollo program, Saturn V, Kennedy Space Center, Columbia, and the lunar module Eagle. Not edge cases. The main objects in the document.

This is the full pipeline, the code that runs it, and the five places it fails silently.

01. Read both scoreboards before you build anything

The repository publishes expected baselines for Haiku in evaluation/README.md:

precision      recall        F1
Entities      0.80 - 0.90    0.70 - 0.85   0.75 - 0.85
Relations     0.70 - 0.85    0.55 - 0.70   0.60 - 0.75

The guide's actual run, scored against the hand-labeled set, came back like this:

Apollo 11        F1 = 0.71   precision = 1.00   recall = 0.55
Neil Armstrong   F1 = 0.55   precision = 1.00   recall = 0.38

Recall lands below the documented floor on both documents. Precision lands above the documented ceiling.

Be fair about why: the notebook cell scores per document and entities only, while the standalone script aggregates and also scores relations. Different measurements, not a contradiction.

Sourcing note: the run output is from the published guide.

The baseline table lives in evaluation/README.md in the same repo. I read it through an indexer rather than the raw file, so verify it before quoting it.

But the lesson holds either way. Two numbers ship in the same repo and they do not agree.

So neither is your number. Run the scorer on your own corpus before you design anything around expected quality.

Perfect precision with recall this low is not a broken extractor. It is a conservative one, and that shapes everything you build on top of it.

What it means in practice:

That third one is the trap. A high-precision, low-recall graph answers existence questions well and completeness questions badly.

02. The whole thing is four prompts

The old pipeline was a trained NER model, a trained relation classifier, and hand-written resolution heuristics, all maintained separately as your data drifted.

The replacement is four calls with two models.

python
# what the cookbook shipped in March
EXTRACTION_MODEL = "claude-haiku-4-5"   # high volume, schema-constrained
SYNTHESIS_MODEL  = "claude-sonnet-4-6"  # weighing conflicting evidence

# the same split on current models
EXTRACTION_MODEL = "claude-haiku-4-5"   # still the right call, schema does the thinking
SYNTHESIS_MODEL  = "claude-opus-5"      # resolution and summarization

The cheap model does extraction because the schema does the thinking and volume dominates cost. The strong model does resolution and summarization because those need judgment across documents.

Do not promote extraction to Opus.

On a thousand-document corpus that is one frontier-priced call per document just to fill in a Pydantic schema. The schema guarantees the shape, not the model.

Opus earns its price on the two stages that need judgment.

Deciding that "Edwin Aldrin" and "Buzz Aldrin" are one person. And writing a profile that resolves contradictions across six documents.

That split is the single most copyable decision in the guide.

That split is the single most copyable decision in the guide.

03. Extraction: one call replaces two trained models

The trick is structured outputs. You define the shape as Pydantic, pass it to messages.parse(), and the response is a typed object. No regex, no JSON decode errors, no defensive isinstance checks.

class Entity(BaseModel):
    name: str
    type: EntityType          # PERSON | ORGANIZATION | LOCATION | EVENT | ARTIFACT
    description: str

class Relation(BaseModel):
    source: str
    predicate: str
    target: str

class ExtractedGraph(BaseModel):
    entities: list[Entity]
    relations: list[Relation]

def extract(text: str) -> ExtractedGraph:
    response = client.messages.parse(
        model=EXTRACTION_MODEL,
        max_tokens=2048,
        messages=[{"role": "user", "content": EXTRACTION_PROMPT.format(text=text)}],
        output_format=ExtractedGraph,
    )
    return response.parsed_output

Two lines in their prompt do almost all the work, and both are easy to skip when you rewrite it:

"Extract only entities that are central to what this document is about, skip incidental mentions." This is where your recall goes. Loosen it and recall climbs while precision falls.

"For each entity, write a one-sentence description grounded in this document." That description is not decoration. It is the disambiguation context the resolver needs two stages later.

Six Wikipedia summaries produced 36 entities and 34 relations, of which 24 names were unique.

04. Resolution: the stage that can make things worse

String similarity handles typos. It cannot handle "Edwin Aldrin" and "Buzz Aldrin", two names with zero character overlap that are the same person.

So you cluster with Claude, using those one-line descriptions as context.

class Cluster(BaseModel):
    canonical: str
    aliases: list[str]

def resolve(entity_type: str, entities: list[dict]) -> list[Cluster]:
    unique = {}
    for e in entities:
        unique.setdefault(e["name"], e["description"])
    entity_list = "\n".join(f"- {name}: {desc}" for name, desc in unique.items())
    ...

Now the part everyone quoted: 24 unique names collapsed to 22 canonical entities.

That is two merges. Across six documents about the same program, the resolver found two things to unify. The famous Aldrin case is one of them.

If you were expecting resolution to be the heavy lifting, it is not. On a narrow corpus most surface forms are already canonical.

And here is what the guide admits that nobody repeated:

Resolution can lower your recall. The resolver picked "Neil Alden Armstrong" as canonical. That is the more complete form and the correct choice by its own instructions. It is also a name your gold set does not contain, so a name that matched before resolution stops matching after.

Their words: that is not a resolver bug, it is a scoring artifact. Which is true, and also means your dashboard will show a regression on a stage that improved your graph.

05. Three ways nodes disappear without an error

This is the section I would read twice. All three are in the shipped code and none of them raise.

Unclustered names vanish. If Claude leaves a raw name out of every cluster, alias_to_canonical has no entry, and the assembly loop drops it:

canonical = alias_to_canonical.get(e["name"])
if canonical is None:
    continue          # entity silently gone

The guide tells you to add a fallback but does not write it. Here it is:

def resolve_with_fallback(entity_type, entities):
    """Never lose a name. Anything Claude skipped becomes its own cluster."""
    names = {e["name"] for e in entities}
    try:
        clusters = resolve(entity_type, entities)
    except anthropic.APIError:
        return [Cluster(canonical=n, aliases=[n]) for n in names]

    covered = {a for c in clusters for a in c.aliases}
    orphans = names - covered
    if orphans:
        print(f"  {entity_type}: {len(orphans)} names unclustered, keeping as singletons")
        clusters += [Cluster(canonical=n, aliases=[n]) for n in orphans]
    return clusters

Print that orphan count on every run. It is the cheapest health signal in the pipeline.

Over-merging collapses distinct things. A specific mission like Gemini 12 can get folded into the broader Project Gemini, because their descriptions overlap. You lose precision and the graph looks cleaner, which is the worst combination.

Edges to dropped nodes disappear too. Assembly filters both endpoints:

if src and tgt and src != tgt:
    G.add_edge(src, tgt, predicate=r["predicate"], source_doc=r["source_doc"])

Every relation whose endpoint failed to resolve is gone, and nothing counts them. Count them:

dropped = {"unresolved_src": 0, "unresolved_tgt": 0, "self_loop": 0}

for r in raw_relations:
    src = alias_to_canonical.get(r["source"])
    tgt = alias_to_canonical.get(r["target"])
    if src is None:
        dropped["unresolved_src"] += 1
        continue
    if tgt is None:
        dropped["unresolved_tgt"] += 1
        continue
    if src == tgt:
        dropped["self_loop"] += 1
        continue
    G.add_edge(src, tgt, predicate=r["predicate"], source_doc=r["source_doc"])

kept = G.number_of_edges()
total = kept + sum(dropped.values())
print(f"edges kept {kept}/{total}  dropped {dropped}")

If more than a few percent of edges are dropping on unresolved endpoints, your resolver is the problem, not your extractor.

06. Assembly is fifteen lines and two real decisions

G = nx.MultiDiGraph()

MultiDiGraph, not Graph. Two entities can be joined by several distinct predicates, so you need parallel edges. And direction carries meaning: "Armstrong commanded Apollo 11" is not the same fact as "Apollo 11 commanded Armstrong."

Every node carries its type, the documents that mention it, and a mention count. Every edge carries its predicate and source document. That per-edge provenance is what makes citation possible later.

The result on six documents:

Graph: 22 nodes, 34 edges
Connected components: 1

Most connected:
  Apollo program                 degree 9  (EVENT)
  Apollo 11                      degree 9  (EVENT)
  John F. Kennedy Space Center   degree 7  (ORGANIZATION)

One connected component is the signal to watch. Fragmented islands mean variants that should have merged and did not. Check this number before you check anything else.

07. The comparison they ran, and the result nobody expected

They asked the same question twice. Once with no graph, once with a serialized two-hop subgraph.

Without the graph, Claude answered from pretraining: Wapakoneta, Purdue, Edwards Air Force Base, Montclair, West Point, MIT, Rome, Washington DC, Kennedy Space Center, Houston, the Pacific Ocean, Honolulu.

With the graph, it answered: Armstrong walked on the Moon. That is one edge. It then stated plainly that the graph holds no location data for Aldrin or Collins.

The grounded answer was dramatically poorer. It was also the only one where every claim cites a specific edge from a specific document.

That is the actual trade, and the guide states it in one sentence: on a private corpus where Claude has no prior knowledge, only the grounded answer works at all.

If your questions are about public facts, the graph is costing you money and quality. Build it when the corpus is yours and the answer has to be traceable.

08. Where the pipeline breaks at scale

The notebook runs six documents in memory. Four things change past that.

Resolution does not fit in a prompt. Ten thousand PERSON entities in one call does not work. The guide says to block first and stops there. This is the blocker:

from collections import defaultdict

def block(entities, max_block=80):
    """Cheap grouping so Claude only arbitrates within small candidate sets."""
    buckets = defaultdict(list)
    for e in entities:
        tokens = [t for t in e["name"].lower().split() if len(t) > 2]
        # last token catches "Neil Armstrong" / "Armstrong"
        # first token catches "NASA" / "NASA Headquarters"
        keys = {tokens[-1], tokens[0]} if tokens else {e["name"].lower()}
        for k in keys:
            buckets[k].append(e)

    seen, blocks = set(), []
    for _, group in sorted(buckets.items(), key=lambda kv: -len(kv[1])):
        fresh = [e for e in group if e["name"] not in seen]
        if not fresh:
            continue
        seen.update(e["name"] for e in fresh)
        for i in range(0, len(fresh), max_block):
            blocks.append(fresh[i:i + max_block])
    return blocks

# resolve each block independently, then merge the alias maps
alias_to_canonical = {}
for b in block(entities_of_type):
    for cluster in resolve_with_fallback(etype, b):
        for alias in cluster.aliases:
            alias_to_canonical[alias] = cluster.canonical

Two names only meet if they share a token, which is why "Edwin Aldrin" and "Buzz Aldrin" still land in the same block on the shared last name.

String similarity would never merge those two. Token blocking still puts them in front of Claude, and Claude does the rest.

Extraction cost is controllable. Move the fixed instructions into a cached system block so you pay full price only for document text:

response = client.messages.parse(
    model=EXTRACTION_MODEL,
    max_tokens=2048,
    system=[{
        "type": "text",
        "text": EXTRACTION_INSTRUCTIONS,        # fixed across every document
        "cache_control": {"type": "ephemeral"}, # cached, not re-billed
    }],
    messages=[{"role": "user", "content": f"<document>\n{text}\n</document>"}],
    output_format=ExtractedGraph,
)

The Batches API is 50% off for anything that can wait 24 hours, which extraction almost always can. Resolution cannot, because it depends on the full extraction output.

Incremental updates need a different resolve. New documents resolve against the existing canonical set, not against each other:

def add_document(text, title, G, alias_to_canonical, canonical_info):
    result = extract(text)
    for ent in result.entities:
        if ent.name in alias_to_canonical:
            continue                       # already known, nothing to decide
        candidates = [
            {"name": c, "description": canonical_info[c].get("description", "")}
            for c in canonical_info
            if canonical_info[c]["type"] == ent.type
        ]
        # ask Claude only whether this ONE new name matches an existing node
        match = match_against_existing(ent, candidates)
        canonical = match or ent.name
        alias_to_canonical[ent.name] = canonical
        canonical_info.setdefault(canonical, {"type": ent.type, "aliases": [canonical]})
    ...

Re-summarize an entity only when its source-document set actually changes. Summarization is the expensive stage and most new documents touch few hub nodes.

Storage is the last thing to change. NetworkX holds a few hundred thousand edges. Past that the schema maps onto three tables and nothing above the persistence layer changes:

CREATE TABLE entities (
  id       BIGSERIAL PRIMARY KEY,
  name     TEXT NOT NULL UNIQUE,
  type     TEXT NOT NULL,
  summary  TEXT
);

CREATE TABLE aliases (
  entity_id BIGINT REFERENCES entities(id) ON DELETE CASCADE,
  alias     TEXT NOT NULL UNIQUE
);

CREATE TABLE relations (
  source_id  BIGINT REFERENCES entities(id) ON DELETE CASCADE,
  target_id  BIGINT REFERENCES entities(id) ON DELETE CASCADE,
  predicate  TEXT NOT NULL,
  source_doc TEXT NOT NULL,
  UNIQUE (source_id, target_id, predicate, source_doc)
);

CREATE INDEX ON relations (source_id);
CREATE INDEX ON relations (target_id);

The two indexes are what make traversal viable. The unique constraint on relations is what makes re-running extraction idempotent, which you will need the first time a batch job dies halfway.

09. Summarization is what turns labels into knowledge

Until this stage every node carries one sentence from whichever document happened to mention it first.

For hub nodes you pool every mention, add the graph neighborhood as context, and synthesize a profile with structured time ranges and atomic facts.

class TimeRange(BaseModel):
    start: str    # YYYY or YYYY-MM, or "unknown"
    end: str      # YYYY or YYYY-MM, or "ongoing"

class EntityProfile(BaseModel):
    summary: str
    key_facts: list[str]
    time_range: TimeRange

Two instructions in that prompt matter more than the schema:

Run it on hub nodes only. Summarizing every node is where budgets die, and low-degree nodes have nothing to synthesize across.

10. The eval harness is the actual product

Everything above is a demo until you can score it. The guide ships a gold set, an alias map, and a precision/recall scorer.

def prf(predicted: set, gold: set) -> tuple[float, float, float]:
    tp = len(predicted & gold)
    p = tp / len(predicted) if predicted else 0.0
    r = tp / len(gold) if gold else 0.0
    f1 = 2 * p * r / (p + r) if (p + r) else 0.0
    return p, r, f1

Change the extraction prompt, rerun the scorer, watch F1 move. That loop is the difference between a notebook and a system.

Two things about how it scores that change how you read the output.

Relations are matched on endpoints only. A relation counts as correct if both its source and target match a gold pair, and predicate wording is ignored entirely. So "Armstrong commanded Apollo 11" and "Armstrong participated in Apollo 11" score identically. Relation recall is an upper bound on connectivity, not a measure of whether the edge says the right thing.

Canonicalization happens before comparison. The alias map normalizes variants first, mapping things like "Project Apollo" -> "apollo program". Every canonical form your resolver invents that the map does not know about reads as a miss.

That second one is why the resolved recall in section 04 could drop.

Extend alias_map.json whenever you see a canonical form the scorer does not know, or you will spend a week tuning a prompt to fix a lookup table.

When not to build this

Skip the graph when a single document contains your answer. That is retrieval, and retrieval is cheaper.

Skip it when your questions are about public knowledge. The comparison above showed the model already knows more than your graph will.

Skip it when nobody will maintain the gold set. Without the eval loop you cannot tell an improvement from a regression, and every prompt change becomes a guess.

Build it when facts must chain across documents that never mention each other, when the corpus is private, and when every claim has to point at a source.

What to actually copy

The model split. Haiku for schema-constrained volume, Sonnet for judgment. This transfers to every pipeline you build, not just graphs.

Descriptions at extraction time. They cost almost nothing and they are the only thing that makes resolution work later.

Per-edge provenance. Store the source document on every edge or you lose citations forever.

The fallback cluster. One line that stops silent node loss.

The scorer. Before the graph, before the prompt tuning, before anything.

The pipeline is four prompts. The engineering is the five places it fails quietly, and Anthropic documented every one of them in a cell most people scrolled past.