What Is Function Calling in LLMs? How AI Uses Tools and APIs

What Is Function Calling in LLMs? How AI Uses Tools and APIs

Function calling in LLMs is the feature that lets an AI model ask for a tool to be run, such as looking up an order or creating an appointment, instead of only writing text. The model does not execute anything on its own. It returns a structured request with the tool name and its parameters, your system decides whether to run it, calls the API, and passes the result back so the model can write the answer. This is how an AI stops merely chatting and starts working with real company data and systems.

Key takeaways

• Function calling is a structured request: the model says which tool to use and with what data, but your code does the executing.
• The cycle has predictable steps: describe the tools, receive the request, validate, execute, return the result and respond.
• Results depend more on how you design the tools and permissions than on how smart the model is.
• Read-only tools are the best starting point. Tools that change data need validation, logging and, often, human approval.
• Function calling is the foundation of AI agents and relates to standards such as MCP, but it does not replace them or remove the need for governance.

How does the tool-calling cycle work?

A language model only produces text. It cannot reach your ERP, your CRM or your calendar. For that to happen, there is an intermediate layer that you control, and the cycle usually follows these steps:

• Tool description: your system sends the model a list of available tools along with the conversation. Each one has a name, a plain-language explanation and the parameters it expects, such as an order number or a desired date.
• Model decision: after reading the user's message, the model judges whether it needs a tool. If it does, it does not answer the user yet. It returns a call request with the tool name and the parameters filled in.
• Validation by your system: your code checks that the tool exists, that the parameters make sense and that this user is allowed to perform that action.
• Execution: only then does your code call the API or query the database. The credential stays in your system and never appears in the conversation with the model.
• Result returned: the API response goes back to the model as a new piece of the conversation.
• Final answer: with the real data in hand, the model writes a natural-language reply. Sometimes it asks for another tool before answering, and the cycle repeats.

The central point is that the model proposes and your system decides. The word call can mislead: the model does not call the API, it asks for the API to be called. That separation is what makes it possible to apply rules, limits and logs.

Business example: an illustrative WhatsApp scenario

Illustrative scenario: a building materials distributor gets many WhatsApp questions like has my order shipped yet? and when can you deliver to my neighborhood? The support team spends time opening the system just to copy the same information.

The company sets up an agent with three tools:

• Check order status: takes the order number and the customer's phone number, and returns the current state (picked, on the way, delivered).
• Check delivery window: takes the postal code and returns the estimated window, based on the logistics rules the company already has.
• Hand off to an agent: records a summary of the conversation and the reason, and places the customer in the human queue.

The customer writes: hi, has my cement order shipped yet? The model sees that it needs data from the system and prepares a request for the status tool, but the order number is missing. Instead of inventing one, it asks the customer. Once the number is provided, the model issues the call. Your code checks that the order belongs to the phone number in the conversation, queries the ERP API and returns the status. The model then replies in a natural tone: the order has left for delivery and is on the way.

Notice what was left out on purpose: there is no tool to cancel an order, give a discount or change an address. If the customer asks for any of that, the agent uses the handoff tool. In practice, the list of tools is the list of what the AI can do. To see how this kind of flow fits scheduling and bookings, read about automatic appointment scheduling on WhatsApp with AI.

What does the model decide and what does your system decide?

A good way to think about it is to split responsibilities explicitly. Use this list as a decision guide:

• The model decides: whether the message requires a tool, which tool fits best, which parameters to extract from the conversation and how to explain the result to the user.
• Your code decides: which tools exist for each user, whether the parameters are valid, whether the action is allowed, whether it needs approval and how many times it can be repeated.
• The business process decides: what may be automated, when to hand over to a person and who is accountable for the outcome.

When these boundaries blur, the classic problems appear: the agent that looks up another customer's order because nobody checked who owns it, or the one that runs an action twice because the first response was slow. Neither error is a failure of the model's intelligence. Both are design failures in the system around it.

How to design tools an AI can use well

Anthropic's engineering guidance on writing effective tools for agents makes the point that tools are a contract between a deterministic system and an agent that is not deterministic. That changes how you design them. A few practices in that spirit:

• Clear names and descriptions: the model reads the description as an instruction. Say what the tool does, when to use it and when not to.
• Few tools, clearly separated: many similar tools confuse the model. Prefer a small set with distinct purposes.
• Simple, typed parameters: ask for an order number, not free text. The tighter the format, the less room for error.
• Lean responses: return only what the model needs to answer. Returning the customer's entire record wastes context and exposes data for no reason.
• Useful error messages: if the order was not found, the tool should say so clearly, so the model asks for the right information instead of guessing.
• Separate reading from writing: looking up and changing data should be different tools with different rules.

Anthropic's Building effective agents guide also recommends starting with the simplest solution and adding complexity only when it is justified. For tools, that often means a fixed flow with one or two well-defined calls works better than an agent with broad autonomy.

What risks appear when AI starts to act?

Once the model can trigger systems, the risk is no longer just a bad answer. It becomes an improper action. The most common points of attention:

• Excessive permissions: the credential used by the tool has more access than it needs. Apply the principle of least privilege and a service account with a narrow scope.
• No identity check: the tool receives an order number, but nobody verifies that it belongs to the person asking.
• Prompt injection: text from emails, documents or messages can contain disguised commands that try to make the model misuse a tool. The OWASP guidance on preventing prompt injection is a good reference for handling untrusted input.
• Repeated actions: if a write call is retried after a network failure, the effect can be duplicated. Tools that change data should be idempotent or carry a control key.
• Too much personal data: whatever the tool returns becomes part of the conversation. Under LGPD, limit the data to the minimum needed for the purpose and record what was accessed.
• No audit trail: without a record of which tool was called, with which parameters and with what result, there is nothing to audit or improve.

The controls that answer these risks are covered in guardrails for AI agents, which deals with validation at input, output and action. For actions that move money, change records or commit to something with a customer, a human approval rule is a sensible addition.

How do function calling, MCP and agents relate?

Function calling is the basic mechanism: the model asks, the system executes. An agent is an arrangement that repeats this cycle, with the model deciding the next steps until a task is done. MCP (Model Context Protocol) is an open standard for connecting AI applications to tools and data sources in a uniform way. The official MCP introduction explains the idea: instead of building every integration by hand, a server exposes tools and data in a format that many AI clients understand.

In practice, the choice between calling the API directly in your code or exposing an MCP server depends on how many systems and how many AI applications will need the same tools. For a single flow, direct integration is usually enough. If the company wants to reuse access to an internal system across several initiatives, an MCP server is worth considering, starting with read-only tools, as described in how to build an MCP server for internal systems.

How to start without creating a problem: map, limit, pilot and hand over

The method we apply to projects like this follows four moves, and each one ties directly to the tool cycle.

• Map: describe the current process, who asks what, which system holds the answer and how much time it takes today. Without that starting point, there is no way to know whether the AI improved anything.
• Limit: choose a few tools, preferably read-only, with minimum-scope credentials, data reduced to what is necessary and clear rules for when to pass the conversation to a person.
• Pilot: put one real flow into restricted operation, with every call logged and a success metric agreed before you begin.
• Hand over: at the end of the pilot, deliver the documentation, logs and criteria, and decide with the team whether to expand, adjust or stop.

A short checklist before releasing any new tool:

• Does the tool have a single purpose and a description a colleague would understand without context?
• Does it only read, or does it also change data? If it changes data, who approves and how is it undone?
• Does the credential it uses have the least access possible?
• Does the system check that the user is entitled to that data before executing?
• Is every call recorded with parameters, result and time?
• Is there a clear path to a person, as described in WhatsApp AI agent human handoff?
• Is there a retry limit and a maximum run time to prevent loops and unexpected costs?

If the answer to any of these is I don't know, the tool is not ready for production.

What changes for the people making decisions?

For a manager, the benefit of understanding function calling is knowing which questions to ask. Instead of asking whether the AI is smart enough, ask which tools it will have, with what permissions, what happens when a call fails and who can see the log. Those answers say more about a project's risk and value than the choice of model.

It also helps to separate three things when evaluating a proposal: what is described in official documentation (such as the call format of each provider), what comes from implementation experience (such as how to design tools and limits) and what is still a hypothesis to test in the pilot (such as how the agent behaves with your real customers). Mixing the three leads to promises nobody can verify.

Frequently asked questions

Is function calling the same as an API?

No. An API is a system's interface that receives requests and returns data. Function calling is the mechanism by which the AI model indicates that it wants to use an API, supplying the parameters. The actual API call is still made by your code, with your credentials and rules.

Can the AI run a wrong action on its own?

It can propose an inappropriate action, either by misreading the conversation or by receiving malicious instructions hidden in a text. That is why execution passes through validation in your system, with minimum permissions, limits and, for sensitive actions, human approval. The less a tool can do, the smaller the possible damage.

Do I need programming to use function calling?

Yes, at some level. Someone needs to describe the tools, write the code that validates and runs the calls and connect everything to the company's systems. Automation platforms reduce part of that work, but decisions about permissions, data and limits remain the company's. A comparison of options is in n8n vs Make vs Zapier for business.

Where should my company start?

Choose a repetitive, low-risk flow, such as checking status or delivery windows, and build one or two read-only tools. Log every call, define a success metric before you begin and leave a clear path for the human team to take over. After a few weeks of observed operation, decide whether to expand, adjust or stop.

Sources

• Anthropic: Writing effective tools for agents
• Anthropic: Building effective agents
• OWASP: prompt injection prevention
• Model Context Protocol: introduction