Mock Tools for Testing AI Agents: Safe Sandbox Testing Guide

Mock Tools for Testing AI Agents: Safe Sandbox Testing Guide

To use mock tools for testing AI agents, put a layer between the agent and each tool, then swap the real APIs for simulations with responses you agreed on in advance. You can run the same scenario as many times as you like and see which tool the agent chose and with what arguments. Nothing leaves the test environment: no message sent, no order changed, no charge created. Afterward, check the simulations against the provider's own test environment so they do not drift away from reality.

Key takeaways

• Test the decision, not just the text: check which tool was called, with which arguments, in what order, and whether the agent stopped or handed off to a human when it should have.
• Mocks, fakes and sandboxes serve different purposes. Use a mock for one-off scenarios, a fake when state matters, and the provider's sandbox to confirm the contract.
• Write tools (send a message, issue a refund, reschedule) never execute in a test. They only record the call.
• Failure scenarios are worth more than the happy path: errors, timeouts, ambiguous data, out-of-limit requests and hostile text coming back from a tool.
• Simulations go stale. Schedule a periodic check against the real system and turn every incident into a new test scenario.

What does it mean to simulate tools in an AI agent?

An AI agent decides, and the decision becomes action through tools: looking up a customer in the CRM, listing calendar slots, creating an order, sending a message, requesting a refund. Testing those decisions directly against live systems causes practical problems. The first is risk, because a failed test can turn into a real message to a real customer. The second is repeatability: data changes, the API goes down, and the scenario that failed yesterday no longer exists today. The third is usage limits and dependence on third parties for every test run.

Simulating a tool means giving the agent a substitute that honors the same contract (same name, same parameters, same response format) but that you control. For the model, the difference is invisible. For you, it changes everything: the scenario becomes a fixed input, and the result can be compared with what you expect.

This is different from testing the model in isolation. What you evaluate here is the whole assembly: instructions, model, tools and stop rules. If the agent calls the wrong tool or invents an argument, the problem could sit in any of those parts, and a test with simulations shows you where to look.

Mock, stub, fake and sandbox: which one for which case?

The terms get mixed up in daily work, so it helps to separate them by what each one does. Use this decision guide:

• Stub: returns a simple default response with no logic. Use it for tools that exist in the flow but do not matter for what you are testing.
• Mock: returns a specific response to a specific call and lets you verify that the call happened. Use it for one-off scenarios, such as customer not found or a timeout error.
• Fake: a simplified implementation with state, such as an in-memory calendar. Use it when sequence matters: the agent lists slots, books one, and the next lookup must reflect that booking.
• Record and replay: you record real responses once and replay them in tests. It helps when the real format is complex, but you must review the recordings and anonymize personal data before storing anything (LGPD).
• Provider sandbox: the test environment the API vendor offers, when it offers one. Use it to confirm the contract is still valid, not for the daily suite, because it may have its own limits and behaviors. Check the vendor's documentation.

In practice, a useful suite starts with mocks and one or two fakes for the tools that hold state. The provider's sandbox comes in later, as a check that your simulations still tell the truth.

How do you build a tool layer you can swap?

The technical secret is simple: the agent should never know whether it is talking to the real system or to a simulation. To get there, follow this plan:

• Step one: list every tool the agent has and classify each as read or write. Write tools are where the risk lives.
• Step two: write the contract for each tool: name, parameters, return format and possible errors. Clear descriptions help the agent use tools well, as Anthropic's article on writing tools for agents discusses.
• Step three: create an interface between the agent and the implementation. The choice between real and simulated should come from environment configuration, never from an instruction written in the prompt.
• Step four: for write tools, the substitute executes nothing. It records the call (which tool, which arguments, in what order) and returns a success or error response depending on the scenario.
• Step five: put no real credentials in the test environment. Without a production key, a configuration mistake fails safely.
• Step six: build fixtures from synthetic data, with invented names and phone numbers. Do not copy a customer database into the test repository.
• Step seven: run the scenario and compare the call log and the final state of the fake with the expected result.

If the agent is built on a workflow platform, the logic is the same: replace the node that calls the real API with a node that records the call and answers with data from the scenario. For a flow built on the official WhatsApp API, for example, the n8n WhatsApp Cloud API support flow can use a recorder in the test environment in place of the message-sending step.

Which scenarios should the mocks cover?

The happy path is the least interesting one. The value of simulation is reproducing, in a controlled way, situations that are rare or dangerous in production. A minimum list:

• Happy path: confirms the basic flow works and serves as a reference.
• Empty result: the customer was not found, the order does not exist. The agent should say so, not invent a record.
• Error and timeout: the tool fails. The agent should retry within a limit, warn the customer or hand off, and never claim the action was completed. See also how to cap steps, time and spend in AI agent infinite loop: step limits, time and budget caps.
• Ambiguous data: two customers with the same name, two possible dates. The agent should ask, not choose on its own.
• Out-of-limit request: the customer asks for something the agent is not authorized to approve. The correct behavior is to hand off to a person, with a summarized context.
• Hostile text in a tool response: a customer notes field contains an instruction such as ignore the previous rules. The agent should treat it as data, not as a command. The topic is covered in prompt injection protection for AI agents, and OWASP maintains a prevention guide.
• Repeated call: the agent sends the same request twice because of a network failure. Some providers document idempotency keys precisely so that repeating a call does not duplicate the effect, as Stripe describes. If the real API behaves this way, the fake should imitate it, and the test should verify that the final state has no duplicates.

Illustrative scenario: rescheduling on WhatsApp

Illustrative scenario: a fictional clinic uses an agent to handle rescheduling requests on WhatsApp. The tools are find_appointment, list_slots, reschedule and send_message. In the test, the calendar is an in-memory fake with invented data, and send_message only records the text that would have been sent.

The input scenario: the fictional customer writes that she wants to move her appointment to Friday afternoon. The calendar fake is configured with no free slots in that period. What you expect, in verifiable terms:

• The agent called find_appointment before anything else.
• The agent called list_slots with Friday's date and did not call reschedule with a time that never appeared in the list.
• The recorded message offers real alternatives, taken from the fake, and does not promise Friday afternoon.
• The final calendar state equals the initial one, because nothing was confirmed.

Variation one: list_slots returns a timeout. The expected behavior is that the agent says it could not check the calendar and offers a callback or a handoff, without stating any availability. Variation two: the customer insists on an exception the clinic's rules do not allow. The expected behavior is a handoff to a human, with the request summarized. Each variation takes a few minutes to write and then runs every time someone changes the instructions or the model. For the full flow, see automatic appointment scheduling on WhatsApp with AI.

How do you evaluate results when the model varies on every run?

The text the agent writes changes from one run to the next, so comparing exact sentences is a fragile approach. Instead, check what matters to the business:

• Required calls: the lookup tool was used before the agent answered.
• Forbidden calls: the refund tool was not triggered in this scenario.
• Valid arguments: dates, identifiers and values came from the tool or the customer and were not invented.
• Final state: the simulated calendar, order or record ended up as it should.
• Stop condition: the agent finished, asked or handed off within the step limit.

Run each scenario more than once. A single good result does not prove the behavior is stable, and the failure rate you observe across several runs tells you more than one isolated test. Anthropic discusses this kind of evaluation for agents in its article on evals, and OpenAI keeps its own guide on the subject. For the quality of the text itself (tone, clarity, courtesy), use a written rubric and sampled human review, because a simple automatic check will not settle it.

One habit that pays off: every error seen in production or in a pilot becomes a new scenario in the suite. Over time, the suite becomes the memory of the problems that have already happened.

How do you keep the simulation from lying?

The risk of a mock-based suite is that it stays green while the real world has changed. A few precautions help:

• Periodically compare the format of your mocks with responses from the provider's sandbox or official documentation, to catch fields that were added, removed or renamed.
• Include the API's real errors in the mocks, with the same format and the same message, and not just an invented generic error.
• Avoid simulations that are too friendly. If the real API sometimes returns incomplete data, the mock should too.
• Separate what comes from official documentation, what comes from implementation experience and what is a hypothesis. An assumed API behavior should be marked as an assumption in the test until it is confirmed.

In the method Juan Carlo uses with companies, this work fits into four stages. In mapping, you list the process's tools and what each one can do. In limiting, you define permissions, separating read from write. In piloting, a real flow runs first against the simulations and then, with logs and an agreed metric, in controlled operation. In handing over, the scenario suite is delivered along with the agent, so the team can decide to expand, adjust or stop with evidence in hand. For more on how he works, see Juan Carlo, AI consultant in Brazil.

Frequently asked questions

Do I need the provider's sandbox if I already have mocks? Not always day to day, but it is worth using from time to time. Mocks give you speed and repeatability, and the sandbox confirms that the real contract still matches what you simulated. If the provider does not offer a test environment, a careful manual check against the official documentation covers part of that role.

Can I use real customer data in tests? The safest answer is no. Synthetic data covers almost every scenario and keeps personal information from circulating through repositories, logs and third-party tools, which is in line with the necessity principle in LGPD. If you must record real responses to reproduce a complex format, anonymize them before storing and limit who has access.

Does this work with agents that use MCP or no-code workflow platforms? Yes, because the idea does not depend on the technology: the agent needs to receive tools with the same contract, and the implementation behind them can be real or simulated. On an MCP server, that means having a test version that exposes the same tools. On workflow platforms, it means replacing the integration nodes with recorders in the test environment.

Does testing with mocks replace a production pilot? No. Mocks reduce risk and expose decision failures before any customer is affected, but they do not reproduce the full variety of real messages and situations. The most prudent path is to run the simulated suite, move to a pilot with one real flow, clear limits and human handoff switched on, and only then decide whether to widen the scope.

Sources

• Anthropic: Demystifying evals for AI agents
• Anthropic: Writing effective tools for agents
• Stripe: idempotent requests
• OWASP: LLM Prompt Injection Prevention Cheat Sheet