AI Vendor Model Updates: The Notice Requirements to Demand

AI Vendor Model Updates: The Notice Requirements to Demand

Lists what your AI vendor must disclose before changing or retiring a model, so your customer agent does not shift behavior without notice or review.

•

Want to apply this in your company?

Book a call with Juan →

When your vendor changes or retires the model behind your agent, customer-facing behavior can shift even though nobody in your company touched a single line of configuration. The vendor should therefore tell you in advance the date of the change, the name of the replacement model, the known behavior differences, and what will stop working. This article lists those points and shows how your company can require them, test them, and record the decision before migrating.

Key takeaways

  • A retirement date early enough to test and approve the switch.
  • The exact name of the current model and the replacement, plus whether the change is a new version, a new family, or a different provider.
  • Known behavior differences: tone, response format, tool use, and context limits.
  • Changes to usage terms, call limits, data storage, and where data is processed.
  • An official notice channel, a named contact for questions, and a written record of your company's acceptance.

Why a model change alters your agent even with the same prompt

Many people assume the agent is the prompt. In practice, the agent is the combination of instructions, tools, business rules, context data, and the model that interprets all of it. When the model changes, the way it understands an instruction, chooses a tool, or writes a reply can also change, even if the instruction text stays identical.

This usually shows up in details that seem small: a reply that becomes longer, a handoff rule that fires less often, or a link or list format that the messaging system does not display well. For a store, a hotel, or a clinic, these details affect the customer experience and the workload of the staff who receive the cases.

What the vendor should disclose before the switch

A useful notice is not a generic announcement of new features. It needs to contain information your company can use to make a decision. A minimum list of what should be in writing:

  • The identifier of the current model and of the model that will replace it, with a note on whether it is a new version, a new family, or another provider.
  • A clear timeline: the date the old version stops accepting new calls, the shutdown date, and what happens to requests made after that date.
  • Known differences in context window, output format, tool calling, languages, and refusals.
  • Changes to data policies: where processing happens, how long information is retained, and whether it is used for training.
  • Changes to commercial terms and usage limits, announced in advance and in writing.
  • An official communication channel, a person or team for questions, and documentation for the new version.

If the vendor cannot provide these items, your company has no basis to approve the switch. In that case, the correct approach is to treat the change as a new risk, not as an automatic update.

How your company should organize receipt of the notice

Most problems happen because the notice lands in an email inbox that nobody monitors. A simple sequence prevents that:

  • Record which model is used by each agent, in which flow, and with which tools.
  • Assign an owner for vendor notices, with access to the official channels.
  • When a notice arrives, open a record with the deadline, the expected impact, and the pending decision.
  • Run a set of test conversations on the new model, comparing the results with the baseline of the current model.
  • Decide whether to accept the switch, adjust instructions, or keep the model offline until validation is complete.
  • Update the decision record and brief the operations team before the cutover.

The central point is that your company keeps a record. Without one, nobody can explain months later why the agent started answering differently.

Illustrative scenario: the switch that changed the handoff rule

Illustrative scenario: a furniture store uses an agent on WhatsApp to answer questions about delivery times and exchanges. When the model changes, the agent keeps answering common questions well, but it starts trying to resolve delivery complaints on its own instead of passing them to the team. The transfer rule to a human is still written in the prompt, but the new model interprets it with more room for discretion.

The team notices the problem through customer messages complaining about generic answers. If there had been a set of test conversations including complaint cases, the problem would have appeared before the cutover. Logs of the agent's decisions would also have helped, to see exactly where it stopped calling a human. The lesson is not that the new model is worse. It is that your company needs to measure the behavior that matters to the business, not only the overall quality of the replies.

How to test the replacement model before it reaches WhatsApp

Testing should use real cases, anonymized wherever personal data is involved. Build a set that includes the most frequent questions, edge cases, complaints, cancellation requests, and situations where the agent must stop and bring in a person. The less personal data the test requires, the better. This connects with the controlled testing approach described here, which uses simulations and sandboxes so real customers are not exposed to behavior that is still unknown.

Define what approval means before the test begins. It might be the rate of correct handoffs, the absence of promises outside the policy, compliance with the format required by the channel, or conformity with data rules. Both OpenAI and Anthropic publish guidance on evaluating model and agent behavior, and those materials can help you structure the test. The metric, however, must come from your own process.

Data, LGPD, and your company's responsibility

If the vendor changes the processing location or the retention policy, your company needs to reassess its use. When data from Brazilian customers begins to be handled outside the country, or by another subprocessor, you must check the LGPD requirements for international transfer. This article explains what the law requires in that case. The ANPD publishes official guidance worth consulting when in doubt.

Responsibility for the decision stays with your company, even when the change came from the vendor. For that reason, the acceptance record should include the date, who approved it, what was tested, and which risks were known. That record also makes it easier to answer customers and to pass internal audits.

How we apply the method in this situation

In our work with companies, a model switch follows the method in four steps. First, we map the current flow and record the baseline: which questions arrive, which actions the agent performs, and where it hands off to a human. Next, we limit data and actions, so the new model under test cannot do anything the old model could not do. Then we pilot a real flow with logs and a metric agreed with the company. Finally, we complete the transition with an explicit decision: expand, adjust, or stop. It is not a sophisticated process, but it prevents the situation where the company discovers the change through its customers.

If you want to understand the difference between the kind of AI your company operates today and the broader concepts that show up in the news, this scale of AI for managers helps place what changes when a vendor switches models.

Frequently asked questions

Is the vendor required to notify you when it changes a model? It depends on the contract and the terms of use of each vendor, and there is no standard notice period that applies to everyone. The safest approach is to require in writing the notice commitment and the retirement date before you integrate the model into your process. If the information is not in the contract, ask for it to be formalized.

Can you pin a model version so it does not change? Many vendors offer fixed version identifiers, which gives you more control over when a change happens. Still, fixed versions also reach end of life, so pinning a version buys time but does not remove the need to plan the switch. Confirm in the vendor's official documentation how that policy works.

Where should a company start if it has never switched models before? Start with the agents that handle money, personal data, or commitments to customers, because the risk is highest in those flows. Record the baseline, build a set of test conversations, and define who approves the change. You do not need a large structure to begin, only a process that can be repeated.

Do you need a technical team to handle this? Not always. The company needs a named owner, records of conversations and decisions, and a test set that represents the business. The technical side runs the tests and adjusts the configuration, but the decision to accept or reject the switch should stay with the person accountable for the process.

Sources

AI vendorsModel updatesAI governanceCustomer supportLGPD