How does prompt chaining work?
You write the steps, and the model fills in each one. Anthropic’s engineering guide describes a pattern that “decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one”, with programmatic checks, a “gate”, on any intermediate step (Anthropic, Building effective agents, 19 December 2024, read 5 November 2026). A support chain might classify an email, extract the order number as structured output, check that the order exists, then draft the reply. The gate is ordinary code, a schema check or a lookup, and LLM output validation is the discipline behind it.
Prompt chaining vs agents vs chain-of-thought: what is the difference?
The difference is who decides the steps. Anthropic separates workflows, where LLMs and tools run “through predefined code paths”, from agents, which “dynamically direct their own processes and tool usage” (same source). Chain-of-thought prompting stays inside a single call.
| Prompt chaining | AI agent | Chain-of-thought | |
|---|---|---|---|
| Model calls | Several, fixed in number | Unknown in advance | One |
| Who picks the next step | Your code | The model | The model, within one answer |
| Where checks run | Between calls | Inside the loop | Nowhere outside the answer |
| Best for | Repeatable tasks with known stages | Open-ended work | Multi-step reasoning in one response |
| Main cost | Added latency per step | Cost and compounding errors | Longer output |
In a chain, your code decides what runs next, which makes it easier to test than an AI agent or a full agentic workflow.
When is prompt chaining the right choice?
When the task splits cleanly into stages you can name in advance. Anthropic calls it “ideal for situations where the task can be easily and cleanly decomposed into fixed subtasks”, trading latency for accuracy by making each call an easier task (same source). It is the first of five workflow patterns in that guide, ahead of routing, parallelization, the orchestrator-worker pattern and evaluator-optimizer. Start with the simplest pattern that holds, and move to an agent only when the number of steps cannot be predicted.
What goes wrong in a prompt chain?
Mostly the joins. A step that drifts in format breaks the next one, a mistake early in the chain is inherited by every later step, and each call spends part of the latency budget. Put a gate wherever a bad output would be expensive to carry forward, and make it deterministic: a schema check or a lookup, never the model grading its own confidence, the argument made in deterministic confidence gates. A chain is only as reliable as its weakest join. Trace each step with LLM tracing and rerun prompt regression testing whenever one prompt changes, because a fix in step two can quietly break step four.