
Energy Use of AI in Your Company: What to Measure Before Choosing a Model
See which usage numbers to track for AI agents, such as calls, tokens and retries, to compare flows, pick models and set frequency with clear criteria.
Want to apply this in your company?
Book a call with Juan →The energy use of an AI agent depends on three factors that your company controls: how many times the model is called, how much text goes in and out of each call, and which model runs each task. So the useful question is not how much energy AI uses in general, but which usage numbers let you compare flows, switch models and cut unnecessary calls without losing quality. This article lists what to measure, in what order, and how to use those numbers in a decision.
Key takeaways
- Consumption follows usage: call volume, context size and response length matter more than the simple decision to "use AI."
- Measure by flow and by completed task, not only by the total monthly bill from your provider.
- Avoidable calls are the first saving to investigate: fixed rules, ready-made answers and direct lookups in your systems do not need the model.
- The larger model should be the exception for ambiguous tasks, not the default for every message.
- If your provider does not report energy per call, use tokens, retries and latency as indicators, and ask for transparency about anything it publishes.
Why energy shows up in your AI bill
Each model response runs calculations on servers, usually in large data centers. The more text the model processes and the larger the model, the greater the computing effort involved. Commercial APIs typically report usage in tokens, and the energy used per call rarely appears in a standard form in the documentation. That is why whoever operates the system needs proxy indicators that they can control.
Talk about superintelligence does not change this math. For a company, what matters is concrete usage: how many calls the agent makes, with what context, for which task and with what result. This article does not provide an official per-call figure to use as a reference, and any value quoted in news coverage should be checked against the provider's documentation before it enters a decision.
What to measure in each flow
Before looking at energy, a company should build a set of metrics for each agent flow. The items below work as a minimum list:
- Call volume per flow and per period, to separate normal use from peaks or repetition loops.
- Input and output tokens per call, with the average and also the extreme cases.
- Size of the context sent, including conversation history and attached documents.
- The model used at each step, not just the default model configured for the agent.
- Repeats and retries, often caused by timeouts, integration errors or responses outside the expected format.
- Calls resolved by fixed rules, ready-made answers or cache, because they show what did not need the model at all.
- Cost per completed task, not per message, applying the logic of unit cost.
- Handoff rate to a human and correction rate, because a cheap but wrong answer creates rework and costs that never appear on the API invoice.
The last two metrics are the denominator that gives meaning to the rest. Dividing total cost by the number of completed tasks, without first defining what "completed" means and what error rate is acceptable, leads to bad decisions. So the dashboard should be built together with the people who run the process, not only by the technical team.
How to build a minimal dashboard
A useful dashboard does not need to be sophisticated. The steps below fit in a controlled pilot:
- Step 1: list the real flows of the agent, such as order status, quotes or booking, and mark the step in each one that calls the model.
- Step 2: record, for each call, the model used, input and output tokens, the step, the result and whether a retry happened. Use identifiers that link the call to the conversation, without storing personal text beyond what is needed.
- Step 3: calculate a baseline over a representative period of operation or controlled testing, and note unusual events such as campaigns, holidays or integration failures.
- Step 4: define, together with the process owner, the success metric, for example cost per completed task with a minimum correction rate that everyone has agreed on, and review the numbers in every cycle.
The OpenTelemetry semantic conventions for generative AI describe attributes for recording the model and token usage, which helps standardize these records if your company already uses observability tools in other systems.
When to switch models and when to reduce calls
The dashboard is only useful if it leads to decisions. The guide below helps separate the most common cases:
- If the task is classification, simple extraction or answering frequent questions from a fixed knowledge base, test a smaller model before keeping the large one. This topic is covered in small language models for business tasks.
- If the task requires interpreting free text with exceptions, keep a more capable model, but limit the context sent to it to the minimum necessary.
- If the same question arrives many times with the same answer, use a fixed rule or a ready-made response, and evaluate prompt caching when your provider offers it.
- If the agent calls the model before checking hours, stock or status in your system, query the system first and use the model only to write the reply.
- If the number of retries rises after a change, pause the rollout and fix the integration before switching models.
- If there is sensitive data or a need for predictable costs, evaluate a local model with the same dashboard. The approach is described in running Ollama local models in a company.
Illustrative scenario
Illustrative scenario: an auto parts store uses an agent on WhatsApp to answer questions about order status, delivery time and part availability. At the start, every message, including a question about opening hours, calls the larger model with the full conversation history. When the team reviews the dashboard, they notice that many calls return information from a fixed table and that the history sent grows with every exchange. They make three changes: opening hours and fixed questions are answered without the model, order status is looked up in the system before the reply is written, and the history sent is limited to relevant exchanges. The decision rests on evidence, not on a promise of savings: the team knows which calls exist, why they happen and what changed after the adjustment. Complaints still go to human review.
Privacy and records: what to keep carefully
Logs of prompts and responses often contain personal data, such as names, phone numbers and addresses. This is where the LGPD, Brazil's data protection law, becomes part of the consumption dashboard. The company should keep only what is needed to measure and audit, define a retention period, restrict access to logs and verify where the provider processes the data. When processing happens outside Brazil, check the requirements described in international data transfer under LGPD when using AI providers. Measuring consumption does not justify collecting more data than the operation needs.
How we apply this method in practice
In our way of working, the starting point is mapping the current process and the baseline of usage, limiting the data and actions the agent can access, piloting one real flow with records and a success metric agreed before launch, and then deciding, based on the results, whether to expand, adjust or stop. The consumption dashboard is part of the pilot, not a separate report. We do not promise energy or cost savings before measuring the flow in question.
Frequently asked questions
Do I need to measure energy in kWh to decide on models?
Not always. For most operational decisions, tokens per call, call volume, retries and cost per completed task already show where the waste is. Energy measurements make more sense when the company runs its own infrastructure, with a team able to collect that data from the hardware.
Does a smaller model always consume less?
In general, a smaller model needs less processing per call, but that does not guarantee lower consumption across the whole flow. If it needs more attempts, a larger context or human review, the total can go up. That is why the comparison should use the same dashboard and the same quality metric.
How many metrics should I track at the start?
Start with a few: call volume, tokens per call, retries, handoff rate to a human and cost per completed task. These five already help identify waste and evaluate a model switch. Other metrics can be added later, once the flow is stable and someone is responsible for reviewing the numbers.
How do these metrics relate to the LGPD?
Records used for measurement can contain personal data, so the company should define what is stored, for how long and who has access. Measuring consumption does not always require storing the full text of conversations. When a record is needed for auditing, it should have a defined purpose and restricted access.