What Are Tokens in AI and Why They Set the Cost of Using It

What Are Tokens in AI and Why They Set the Cost of Using It

A token is the smallest unit of text an AI model processes: it can be a whole word, a piece of a word, a punctuation mark or a space. Providers bill and limit usage in tokens, not in words or messages, so the number of tokens going in and out of each call is what sets the cost of an AI system.

Key takeaways

• A token is not a word: the model splits text into pieces, and languages like Portuguese often produce more pieces per sentence than English does.
• Every call has two sides measured separately: input tokens (what you send) and output tokens (what the model writes).
• In a conversation, the history is resent on every turn, so input grows even when the messages are short.
• Input and output usually have different unit prices, and output tends to weigh more per token. Confirm the price table of the provider you use.
• Estimating usage before launch is simple: add up instructions, history, retrieved data and the reply, then multiply by expected volume.

What are tokens in AI, in practice?

A language model does not read letters or words the way a person does. Before processing, a component called a tokenizer converts the text into a sequence of units. Each unit gets a number, and those numbers are what the model works with. When it answers, it generates token by token, and the tokenizer converts everything back into readable text.

The size of each piece depends on the tokenizer's vocabulary. Common, short words tend to become a single token. Long words, rare words, words with accents or words with uncommon suffixes tend to be split into several. Numbers, codes, email addresses and proper names also tend to use more tokens than they appear to.

A hypothetical example, only to illustrate the idea (the real split changes from model to model):

• The Portuguese word "casa" (house) may take one token.
• The word "atendimento" (customer service) may be split into two or three pieces.
• The word "desburocratização" (cutting red tape) may be split into several smaller pieces.
• An order number like "PED-20481-B" may become many short tokens, because the tokenizer does not recognize a common pattern.

The practical consequence: counting words gives you a rough feel, but not the number. Each model family has its own tokenizer, and the same text can produce different token counts across providers. When the figure matters, use the provider's own counting tool, or the usage field that the API response returns.

Why does Portuguese usually use more tokens?

The best-known tokenizers were trained on a lot of English text, so their vocabulary represents English words better. Portuguese has accents, the cedilla, long verb conjugations and lengthy words, which tends to produce more splits. The effect varies by model, and newer models usually handle languages other than English better, but the safe starting point is not to assume that Portuguese text takes the same space as its English equivalent.

This affects two decisions. The first is budget: an estimate built from ready-made rules of thumb in English-language blogs may come in below the real figure. The second is the design of your instructions: a wordy block of instructions, repeated on every call, becomes a fixed cost per message.

The recommendation is to measure with real text from your operation. Take ten typical conversations, run them through the provider's counter and use the average as a baseline, with a margin for long cases.

Why do input and output weigh differently?

Every language model provider splits usage into at least two counters. Input tokens are everything you send: instructions, history, documents, the question. Output tokens are everything the model writes back, including, in some models with extended reasoning, the internal reasoning that the provider counts as output. Confirm this detail in the documentation of the model you contract.

There are technical reasons for the price difference. Reading the input can be done in parallel, while generating the output happens in sequence, one token after another, and takes more computing time. That is why, in most public price tables, the unit price of output is higher than that of input. The values and the ratio change over time and between models, so they are not quoted here: always check the current pricing page.

Two readings come out of the price difference. They look opposed, but they complement each other:

• Volume: input is usually much larger than output, because it carries instructions and history on every call.
• Unit price: output costs more per token, so long, verbose replies inflate the bill disproportionately.

In other words, optimizing only one side leaves money on the table. An agent that writes huge replies weighs on output. An agent with giant instructions and unlimited history weighs on input.

How does a conversation accumulate tokens?

Language models have no memory between calls. For the conversation to make sense, the system resends the whole history (or a summary of it) along with each new message. This means turn ten costs more input than turn one, even if the customer only wrote "ok, thanks".

The technical limit of that sum is the context window, the total number of tokens the model can consider at once. If you want to understand that limit and how to avoid overfilling it, read the article on the context window in AI. The point here is the financial side: before you hit the limit, a long history has already raised the bill.

This is common in WhatsApp customer service. Conversations are chopped into many short messages, and each one triggers a new call carrying everything that came before. To see how that flow is organized on the technical side, the guide on how to integrate AI into your company's WhatsApp shows the pieces involved.

How to estimate the usage of a conversation, step by step

The method below works in a spreadsheet and does not depend on any specific tool.

• List the blocks of each call: fixed instructions, history, retrieved data (catalog, policy, customer record), the new message and the reply.
• Measure each block in tokens with the provider's counter, using real text and not clean examples.
• Calculate the input of the turn: instructions, plus accumulated history, plus retrieved data, plus the new message.
• Add up the input of all turns of a typical conversation, and separately add up the output.
• Multiply each sum by the matching unit price in the provider's current table.
• Multiply by the number of conversations expected in the period and add a margin for long conversations and reprocessing.
• Divide the total by the useful result (conversation resolved, quote sent, appointment booked) to get the cost per unit of work.

The last step matters most for whoever decides. Cost per token only makes sense when converted into cost per result, which is what the unit economics practice described by the FinOps Foundation proposes: tying spend to a unit of business value.

Illustrative scenario: five turns of customer service

The numbers below are hypothetical and only show the mechanics. Suppose a store serves customers on WhatsApp with an AI agent. The agent's fixed instructions add up to 1,500 tokens. Each customer message is 40 tokens and each agent reply is 120 tokens. The conversation has five turns.

The input of each turn is the sum of the instructions, the history and the new message:

• Turn 1: 1,540 input tokens.
• Turn 2: 1,700 input tokens.
• Turn 3: 1,860 input tokens.
• Turn 4: 2,020 input tokens.
• Turn 5: 2,180 input tokens.

Total input for the conversation is 9,300 tokens. Total output is 600 tokens (five replies of 120). There is roughly fifteen times more input than output, but output has a higher unit price, so it cannot be ignored. In this design, what dominates the volume is the repeated instructions: 1,500 tokens sent five times add up to 7,500 of the 9,300 input tokens.

The takeaway for the manager is direct. Before asking for shorter replies, look at the size of the instructions and whether the provider offers caching for the fixed portion. That is the subject of the article on reducing token costs with caching, routing and compaction, and Anthropic's prompt caching documentation describes how this mechanism works at one specific provider.

Where usage tends to get out of control

Once the system is live, a few patterns show up often and deserve a check:

• Instructions that grow with every adjustment, with nobody reviewing what is still needed.
• The full history sent every time, with no summary and no turn limit.
• Long documents pasted into the call when only an excerpt was relevant.
• Replies with no length limit, in situations that call for two lines.
• Automatic retries after a failure, with no cap on attempts.
• Tools that return huge results (complete lists, entire pages) for the model to read.

The last point is less obvious. In agents that use tools, each tool's output becomes input in the next call. Anthropic discusses this concern in its material on context engineering for agents: put into the context only the information the model needs for the next step.

To track all this in production, the ideal is to log the token usage of each call together with the task type. The OpenTelemetry semantic conventions for generative AI already define attributes for this kind of measurement, which helps you avoid depending on a single vendor's dashboard.

How to apply this with method: map, limit, pilot, hand over

Controlling cost per token does not need a separate project. It fits into the four stages of any AI rollout in a business process.

In mapping, you record the current flow and its volume: how many conversations, of what size, with what data. Without that baseline, any token estimate is a guess. In the limit stage, you define the maximum reply length, the number of turns kept in the history, the data the agent may consult and the actions it may take. In the pilot, a real flow runs with usage logged per call and a success metric agreed in advance, and cost per result goes into that metric. The article on how to measure an AI automation pilot details this kind of criteria. In the handover, the team receives the estimation spreadsheet, the configured limits and the decision to expand, adjust or stop.

This separation also helps distinguish what is official documentation (how each provider counts and bills tokens), what is deployment experience (where usage tends to escape) and what is still a hypothesis to validate in your case.

Frequently asked questions

Is a token the same as a word? No. A token can be a short word, part of a long word, a number or a punctuation mark. In Portuguese, it is common for a sentence to produce more tokens than the word count suggests, and the exact figure depends on the model's tokenizer.

How do I find out how many tokens my text has? Use the token counter of the provider you plan to contract, or read the usage field the API returns after each call, which reports input and output tokens. Estimates based on word counts are fine for a first idea, but they do not replace measuring with real text from your operation.

Why does output usually cost more than input? Generating text requires producing one token at a time, in sequence, which takes more computing than reading the input. That is why many public price tables charge more per output token. The values and the ratio change between models and over time, so always check the current pricing page.

Can I reduce usage without hurting the quality of customer service? Generally yes, starting with what adds nothing: lean instructions, summarized history, data retrieved on demand, a length limit on replies and caching of the fixed portion when the provider offers it. Each cut should be tested on real conversations before going to production, because savings that lower the quality of the reply tend to come back as human rework.

Sources

• FinOps Foundation: unit economics
• Anthropic: prompt caching
• Anthropic: Effective context engineering for AI agents
• OpenTelemetry: semantic conventions for generative AI