LLM Workflow Patterns: Prompt Chaining, Routing and Parallelization
Learn how prompt chaining, routing and parallelization work in LLM workflows, with business examples and clear rules for when to avoid an autonomous agent.
Want to apply this in your company?
Book a call with Juan →LLM Workflow Patterns: Prompt Chaining, Routing and Parallelization
The most widely used LLM workflow patterns are prompt chaining (each step feeds the next), routing (the input is classified and sent down the right path) and parallelization (independent tasks run at the same time and the results are combined). In all three, your code defines the path and the model performs bounded steps. That is usually more predictable and easier to audit than an autonomous agent. For most business processes, starting with one of these patterns is the safest decision.
Key takeaways
- A workflow is a flow where you define the path, in code or in an automation tool. An agent is when the model decides the next steps on its own.
- Prompt chaining suits tasks that break down into fixed steps, with a check between them.
- Routing suits separating request types and sending each one to a specialized treatment, including a human.
- Parallelization suits speeding up independent parts or comparing more than one answer before deciding.
- An autonomous agent is only justified when the number of steps cannot be predicted and a mistake is cheap or reversible.
Workflow or agent: what is the practical difference?
The distinction comes from Anthropic's documentation on building agents, which separates workflows (models and tools orchestrated through predefined paths) from agents (systems where the model directs its own process and tool use). The split is useful for decision makers because it changes what you need to control.
In a workflow, you know in advance which steps exist, in what order they run and where a person reviews the result. If something goes wrong, the log shows which step failed. In an agent, the number of steps varies, the spend varies and the path changes from one run to the next. That can be exactly what an open-ended problem requires, but it is rare in support, finance or operations routines.
A practical rule, which is our implementation view and not an official rule: start with the lowest degree of autonomy that solves the problem, and move up one level only when a test shows the previous level is not enough. For a complementary view on when to split work across several agents, read Multi-Agent Systems vs Single Agent: When to Use Each.
Prompt chaining: when each step depends on the previous one
In prompt chaining, a task is broken into steps and the output of one model call becomes the input of the next. Between steps you can add a simple check, written in code, that decides whether the flow continues or stops.
The benefit is that each call has a narrow task, which is easier to instruct, test and fix. The cost is more latency, because the steps run in sequence.
Business examples where chaining works well:
- Sales meeting summary: extract decisions and open items, then format them to the CRM standard, then validate required fields. The topic is covered in How to Update Your CRM With AI After a Sales Meeting.
- Sales proposal: gather requirements from the conversation, build the draft, check that all mandatory items are present, send it for human review.
- Reply to a public complaint: summarize the history, write a draft, check tone and forbidden promises, approve.
One design note: put the check between steps in code whenever possible. Verifying that a field exists, that a value is on an allowed list or that the text contains a forbidden promise does not need a model. Reserving the model for work that requires interpretation keeps the flow cheaper and more predictable.
Routing: classify before you answer
In routing, the first step classifies the input and the flow follows a specific path. Each path can have its own instructions, tools, model and permission level. It is the most natural pattern for customer support, because customer messages mix topics that should not get the same treatment.
A well-designed router usually has these traits:
- A closed set of categories, always including a catch-all such as unknown or other that leads to a person.
- Structured output (only the category name), validated in code before the flow continues.
- Deterministic rules ahead of the model, when they exist. If the message carries an order number or arrives from a specific channel, you may not need to ask the model at all.
- A record of the chosen category on every run, so you can measure routing errors later.
A less obvious benefit is risk control. Requests for a copy of an invoice or an order status can go down a path with read-only tools. Requests involving cancellation, refunds or personal data can go straight to a human queue. That way permission follows the type of request, not the goodwill of the model. For how this applies to support, AI Customer Support: What to Automate and What to Keep Human covers the separation in detail.
Another point: routing also helps with cost. Simple questions can go to a smaller, cheaper model, and ambiguous cases to a more capable one. Treat this as a hypothesis to validate with your own data, not a guarantee.
Parallelization: split the work or compare answers
Anthropic's documentation describes two variations of parallelization. In the first, sectioning, independent parts of a task run at the same time. In the second, voting, the same task runs more than once to get different answers and raise confidence in the result.
Examples of sectioning:
- Analyze a contract along independent lines (deadlines, penalties, data protection) and merge the findings into a single report.
- Process several documents from the same customer at once, each with the same instruction.
- Run a policy check (inappropriate content, sensitive data) in parallel with generating the reply, instead of after it.
Examples of voting:
- Classify a risk case with two or more independent calls and send it to human review when they disagree.
- Review a piece of code or legal text with different instructions and compare the warnings.
To decide whether it is worth it, ask whether the parts are truly independent. If one depends on the result of the other, it is chaining. Then ask whether the added cost of extra calls pays for itself: every call consumes tokens, and spend grows in proportion. What Are Tokens in AI and Why They Set the Cost of Using It helps you estimate this.
Illustrative scenario: WhatsApp orders at an auto parts store
Imagine an auto parts store that receives customer messages on WhatsApp. This is an illustrative scenario, unrelated to any real company, that combines all three patterns.
First, a router classifies the message into one of a few categories: parts quote, order status, exchange or warranty, and other. Messages in exchange or warranty and other go straight to a person, with a summary already prepared.
On the quote path, a chain takes over. The first step extracts the vehicle model and the part the customer wants from the conversation. A check in code confirms that both fields exist and that the part is in the catalog. If something is missing, the flow asks the customer one specific question. If everything is there, the second step checks stock and builds the draft reply.
While the draft is built, two checks run in parallel: one confirms the text does not promise a delivery date the system has not confirmed, and the other confirms there is no unnecessary personal data in the reply. If either one raises a flag, the draft goes to human review before it is sent.
No step decides on its own what to do next. The path is known, the logs show each decision and a person steps in where the risk is higher. To connect a flow like this to WhatsApp, n8n WhatsApp Cloud API Workflow: Build a Support Flow Step by Step shows a practical starting point.
When should you avoid an autonomous agent?
An autonomous agent makes sense for open-ended problems, where the steps cannot be predicted and the environment gives clear feedback after each step. Even so, Anthropic itself recommends finding the simplest solution possible and adding complexity only when needed. Use this list as a decision guide:
- If you can draw the process as a flowchart with a few branches, use a workflow.
- If a mistake causes damage that is hard to undo (payment, cancellation, sending personal data, contacting the customer), keep human approval and a fixed path.
- If you must explain to an auditor, a customer or the Brazilian data protection authority (ANPD) why a decision was made, prefer logged steps over free-form reasoning. LGPD Automated Decision-Making: The Right to Review (Art. 20) covers the right to review.
- If cost per run must be predictable, avoid agents with no cap on steps, time and spend. See AI Agent Infinite Loop: Step Limits, Time and Budget Caps.
- If the agent would read external content (emails, web pages, attachments) and could still act through tools, prompt injection risk grows. The OWASP prevention cheat sheet is a good reference, and Prompt Injection Protection for AI Agents: A Practical Guide covers the practical side.
On the other hand, an agent can be worth it for tasks like exploratory research, where a person reviews the final result and no irreversible action happens along the way. Even then, start with read-only tools.
How to put one of these patterns into operation
The method we use follows four moves: map, limit, pilot and hand over. Applied to workflows with a model, it looks like this:
- Map: describe the current process as it really is, with inputs, outputs, exceptions and who decides. Write down a baseline (how the work is done today) before any automation.
- Limit: define which data the flow can see, which actions it can take and where a person approves. When personal data leaves the country because of the model provider, check what LGPD requires for international data transfer or consider running a local model.
- Pilot: pick one real flow, with a record of each step and a success metric agreed in advance. Build a set of test cases (the evaluation guides from OpenAI and Anthropic explain how to think about this).
- Hand over: give the team the documentation, the logs and the review criteria, and decide together whether to expand, adjust or stop.
On failures, treat the flow like any other integration: timeouts, retries with clear criteria and error logging. n8n Error Handling and Retries: AI Workflows That Don't Break covers that side.
Frequently asked questions
Which pattern should I use first? Start with chaining if the process is already a sequence of steps, and with routing if the problem is a mix of request types. In practice the two often appear together: the router picks the path and each path is a short chain. Parallelization comes later, when there is a concrete reason for speed or confidence.
Does an LLM workflow need a specific tool? No. You can build it with a visual automation tool, with code in Python or TypeScript, or with agent platforms. What matters is that the steps, validations and logs are explicit, whichever tool you choose.
Does parallelizing always make the flow faster and better? Not necessarily. Splitting independent tasks reduces waiting time, but it increases the number of calls and the spend. Voting can raise confidence, but only if the answers are truly independent and you have a clear rule for what happens when they disagree.
Does a workflow remove the need for human review? No. It makes review more focused, because you know at which step and for what reason a case was flagged. For decisions that affect people, money or personal data, human approval remains the most reliable control.