Version Control for Prompts in Git: Review, History, Rollback

Version Control for Prompts in Git: Review, History, Rollback

To set up version control for prompts in Git, treat each prompt as a text file in a repository, change it only through a reviewed pull request, and record, for every version, the result of a fixed set of evaluations. Then, when a change makes customer support worse, you can identify the responsible version and roll back knowing which behavior is coming back. The history stops depending on the memory of whoever edited last and becomes a record the company can audit.

Key takeaways

• A prompt is part of the production system: without a version, you cannot know what changed or how to go back.
• A simple repository, with one file per prompt and a configuration file next to it, already covers most cases.
• Pull request review should require the reason for the change, tested examples and an attached evaluation result.
• Each version must be tied to the model, the parameters and the test set used, otherwise the rollback is incomplete.
• Rolling back is easy when the production version shows up in the logs of every conversation and the process is written down before the incident.

Why do prompts need versions like any other code?

A prompt defines how the agent speaks, what it refuses, when it hands off to a person and what information it may cite. Changing one sentence can shift the tone of every answer or make the agent promise something the company does not deliver. When the text lives in a tool dashboard, a spreadsheet or a shared document, nobody knows who changed it, when or why.

The problem usually looks like this: support quality drops in a given week, someone remembers that they "tweaked an instruction", but the previous version no longer exists. Without it, the team tries to rewrite from memory and probably creates a third version, different from both.

Git solves this without a new tool. It already stores authorship, date, line-by-line differences and a message explaining the change. It also allows mandatory review before any change reaches production. If you write support instructions, pair this guide with the article on how to write instructions for an AI support agent, which covers the content of the prompt. Here the focus is its lifecycle.

A prompt without a version cannot be rolled back.

How should you organize a prompt repository?

The structure should be small enough that the team follows it without effort. A dedicated repository works well when several systems share prompts. If there is a single product, a folder inside its repository is enough, as long as the review rules are the same.

One possible layout:

• prompts/support/triage.md: the prompt text, with no customer data.
• prompts/support/triage.config: model, temperature, response limit and other settings that affect behavior.
• prompts/support/CHANGELOG.md: one line per change, with the reason and a link to the pull request.
• evaluations/support/cases: example inputs with the expected behavior, versioned alongside.
• evaluations/support/results: records of each evaluation run, tied to a version.
• docs/OWNERS.md: who approves changes to each prompt.

Three decisions make a difference. First, separate the prompt text from the configuration, because switching the model or the temperature changes behavior as much as changing a sentence, and that has to appear in the history. Second, keep the test cases in the same repository, so a prompt change and a change to the tests are reviewed together. Third, never store access keys, real conversations or personal data. Examples should be fictional or anonymized, which also reduces risk under LGPD (Brazil's data protection law).

To identify versions, use Git tags with a clear name per prompt, and record the commit identifier. The commit is the exact reference; the tag is the readable form for the operations team.

How do you review a prompt change in a pull request?

Prompt review differs from code review, because the text does not fail with an explicit error: it keeps running and simply answers worse. That is why the pull request must carry evidence, not just the text diff.

A pull request description template, as a checklist:

• Which observed problem motivated the change, with one or two anonymized examples.
• What changes in the expected behavior, in one sentence.
• What must not change, such as rules for handing off to a human and limits on promises.
• Evaluation results before and after, on the same test cases.
• New cases added to the test set to cover the problem.
• Data impact: does the prompt now ask for or mention personal information?
• Rollback plan: which version comes back if something gets worse.

The reviewer should be someone who knows the business process, not only someone who understands models. In customer support, that may be the operations lead. In a legal or financial flow, someone from that department. You can require approval from a file owner (for example, with CODEOWNERS on GitHub or an equivalent feature on GitLab) and block merges into the main branch without that approval.

Rules that help: one change per pull request, without mixing a tone adjustment with a new business rule; commit messages that explain why, not only what; and a ban on editing directly in production. If the tool lets you edit the prompt in a dashboard, the dashboard should be fed from the repository, not the other way around.

For the safeguards that a prompt should not replace, such as output validation and action limits, see the article on guardrails for AI agents. A well-versioned prompt does not remove the need for those controls.

How do you link each version to an evaluation result?

This is the step that turns the history into a decision tool. Without the link, Git shows that the text changed but not whether it got better or worse. With it, each version has a record sheet.

Evaluation needs a fixed set of cases, with the expected result defined by someone who knows the process. OpenAI's documentation on evals and Anthropic's article on evaluating AI agents describe ways to build these sets and to combine automatic checks with human review. What matters for versioning is recording each run in a reproducible way.

Each evaluation record should contain:

• The commit identifier of the prompt that was evaluated.
• The model and model version, plus the parameters used.
• The version of the test case set.
• The result by case category, not only an overall summary.
• The cases that failed, with the response obtained.
• Who ran it, when, and whether a human reviewed the responses.

Store this record in the repository or in a location the repository references, and cite it in the pull request. When the version is approved and tagged, the tag points to an evaluation record. That gives you the question that matters on the day of the problem: which was the last version that passed the relevant cases?

In operations, log the prompt version on every conversation or run. Telemetry conventions for generative AI, such as those described by OpenTelemetry, help standardize attributes like model and parameters. Whatever the format, the point is that, when a complaint arrives, you can say which version answered. The article on how to measure an AI automation pilot explains how to define decision criteria before expanding.

How do you roll back safely when a version gets worse?

A rollback is only safe if the process is written down before the incident. In general, it has four steps:

• Step one, confirm the probable cause: compare the production version with the previous one and check the logs to see whether the problem started together with the change. A change of model, knowledge base or integration can have the same effect.
• Step two, roll back through Git: use the revert command (git revert) instead of erasing history. That way the mistake and the fix are both recorded, and the action goes through the same review, even if a quick one.
• Step three, re-evaluate the restored version: run the current set of cases. The old version may have passed the old tests and fail cases added later.
• Step four, record the incident: add the case that exposed the problem to the test set and note the decision in the CHANGELOG.

There are three traps. The first is rolling back the prompt without rolling back the configuration, when the cause was a parameter. The second is assuming the model is stable: if the provider updated the model behind a generic name, the same prompt version can behave differently, so pin the model version whenever the provider allows it. The third is losing track of conversations already sent. If the agent promised something wrong to customers during the problem window, the company needs a human process to correct it, and the version log shows who was affected.

In flows with outbound messages or decisions that affect people, it is also worth having an operational switch to turn the agent off or limit its actions, independent of any change in the repository.

Illustrative scenario: a refund rule that changed the tone

Imagine, as an illustrative example, a services company with a WhatsApp agent for billing questions. The team wants the agent to be warmer and opens a pull request changing two sentences in the triage prompt. The description lists the reason, attaches the before and after evaluation and names the previous version as the rollback plan.

In the evaluation, the tone cases improve, but a group of cases about refund requests starts to fail: the agent, trying to be kind, uses a phrasing that sounds like a promise of reimbursement. The reviewer, who knows the policy, rejects the pull request and asks for an adjustment. The second version fixes the tone without changing the rule for handing off to a person.

If the problem had only been noticed in production, the path would be the same, only slower: see in the logs that the problem conversations used the new version, roll back through Git, re-evaluate and add the refund cases to the test set. What changes is the cost of the mistake, and what keeps it from happening again is the case that is now on record.

How does the map, limit, pilot, hand over method apply?

Versioning follows the stages of the work. In map, the team lists which prompts exist, where they live today, who changes them and which decisions each one influences. In limit, it defines what a prompt may state and do, and who approves changes to each one. In pilot, it runs a real flow with version logs and an agreed success criterion, using the evaluation set as a reference. In hand over, it delivers the repository, the pull request template and the rollback procedure to the company's team, which decides whether to expand, adjust or stop.

If the prompt handles personal data or uses a provider outside the country, also assess what LGPD requires, the topic of the article on international data transfer when using AI providers. Versioning helps here because it records which data the prompt started to mention and when.

Frequently asked questions

Do I need a separate repository just for prompts? Not necessarily. If the prompt belongs to a single system, a folder in that system's repository makes joint review with the code easier. A dedicated repository makes sense when several products or teams share prompts, or when the people who edit them are not the ones who touch the code.

Can non-technical people take part in the review? Yes, and they should. Platforms like GitHub and GitLab have a web interface for reading differences and commenting, with no command line. The business reviewer checks whether the text respects company policy, while the technical team handles testing and release.

Is versioning the prompt enough to guarantee the same behavior? No. Behavior also depends on the model, the parameters, the connected tools and the documents consulted. That is why the record of each version should include those items, and the evaluation must be repeated whenever any of them changes.

What should I do with real conversations that reveal problems in the prompt? Use them as inspiration, but do not copy them into the repository untreated. Remove names, phone numbers and anything that identifies the person, and turn the case into an equivalent fictional example. This keeps the test set useful and reduces the risk of exposing personal data in the Git history, from which it is hard to erase later.

Sources

• OpenAI: guia de evals
• Anthropic: Demystifying evals for AI agents
• OpenTelemetry: convenções semânticas para IA generativa
• NIST AI Risk Management Framework