How to Evaluate a RAG System: Retrieval, Faithfulness, Citations

How to Evaluate a RAG System: Retrieval, Faithfulness, Citations

To evaluate a RAG system, test two stages separately: whether the search brought back the right passages, and whether the answer used only those passages and cited the correct source. Start with a small set of real questions, each with the expected answer and the document that supports it. A single score for the whole system hides where the defect is, which is why the separation matters.

Key takeaways

• Search and answer fail for different reasons, so measure them separately: retrieval on one side, faithfulness and citation on the other.
• A set of real questions, each with an expected answer and a source document, is worth more than any sophisticated metric.
• Three simple questions are enough to start: did the right passage show up, did the answer stay inside the passage, does the cited source support what was said.
• Questions with no answer in the documents belong in the test, because the correct behavior there is to admit the system does not know.
• Evaluation does not end at launch: documents change, and the test set must be rerun after every relevant change.

Why evaluate search and answer separately?

A RAG system (retrieval-augmented generation) works in two stages. First, it searches a knowledge base for passages. Then a language model writes the answer using those passages as context. The AWS documentation on RAG describes this design: the model consults an external knowledge source before generating a response.

The practical point is that each stage has its own defects. If the search does not bring back the passage that contains the answer, no adjustment to the instructions will fix it. If the search brings back the right passage and the answer still comes out wrong, the problem is in generation or in the instructions, and changing the document index will not help.

When a team looks only at the final answer and says it is bad, it does not know where to start. When it separates the two evaluations, the conversation becomes actionable: the search is failing on these questions, or the answer is making things up on those others.

How do you build a test question set?

The test set is the main asset of any evaluation. It does not need to be large at first, it needs to be representative. The best source is questions that customers or colleagues actually asked: support history, emails, WhatsApp messages, internal queries. Before using this material, remove personal data and confirm the legal basis under LGPD.

For each question, record four things: the question text, the expected answer in plain language, the document (and the part of it) that supports the answer, and a flag saying whether the question should be answered or declined.

A simple outline for building the set:

• List the topics the assistant will be asked about and pick questions from each one, not only the easiest.
• Include questions written the way people really write them, with abbreviations, typos and incomplete sentences.
• Include questions whose answer depends on more than one document.
• Include questions whose answer does not exist in the knowledge base, to test whether the system admits it.
• Include questions about information that changed, where an old document and a new one contradict each other.
• Ask someone who knows the subject to validate the expected answer, not the other way around.

That last rule prevents a common mistake: using the model itself to generate the expected answers and then measuring the system against them. The answer key has to come from a person responsible for the content.

How do you measure retrieval?

The retrieval question is objective: for this question, did the passage that contains the answer appear among the passages the search handed to the model? You do not need to look at the final answer for this. Run only the search stage and compare the result with the source document noted in the test set.

Three simple measures cover most of what a team needs:

• Retrieval hit rate: in how many questions the correct passage was among those returned.
• Position of the right passage: whether it came near the top or was buried at the end of the list.
• Noise: how many irrelevant passages came along with the correct one and took up space in the context.

Noise matters because the model's context is limited and useless passages can distract the answer. If this topic is new to your team, it is worth reading about the context window in AI to understand why sending more text does not always improve the result.

When the retrieval hit rate is low, the causes are usually about data, not the model: long documents split in the wrong place, PDFs with poorly extracted text, internal names the search does not connect to the term the customer uses, or old versions of the same document competing with each other. For scanned documents, extraction deserves its own care, as we discuss in extracting data from PDFs with AI.

How do you measure answer faithfulness?

Faithfulness is the question: is everything the answer claims supported by the passages the system received? An answer can be correct in the real world and still unfaithful, because the model filled in something from its own knowledge that the knowledge base does not say. In a corporate assistant this is a problem, since the company answers for what its official knowledge base states, not for what the model assumes.

To evaluate, take the answer and the passages delivered and check claim by claim. A manual method works well at the start:

• Split the answer into short claims (a deadline, a condition, a step).
• Mark each claim as supported, unsupported or contradicted by the passages.
• Classify the whole answer: faithful, partially faithful or unfaithful.
• Note the type of failure: invention, exaggeration, a blend of two rules, or information from an outdated document.

Once the team understands the pattern, you can use a model as a grader to triage larger volume, as long as a sample keeps being reviewed by people. The OpenAI documentation on evals and Anthropic's article on evals for AI agents cover how to structure these tests and why clear criteria and reviewed examples matter. As a working hypothesis, not a documented fact, we recommend treating the automatic grader like a junior colleague: useful for organizing the queue, not for giving the final word.

Do not forget the questions with no answer in the knowledge base. For those, the faithful answer is to say the information was not found and, when it makes sense, hand off to a person. If the system confidently answers something that is not in the documents, count it as a faithfulness failure, even if the answer sounds plausible.

How do you evaluate cited sources?

Citing a source only helps if the citation is true. There are three distinct checks, and teams usually do only the first:

• The cited source actually exists in the knowledge base (it is not an invented name).
• The cited source contains the information the answer attributes to it.
• The cited source is the current version of the document.

The second check is the one that catches the most problems. A system can cite a real document from the knowledge base and still attach to it a claim the document does not make. For the user, the citation creates a false sense of verification. So the useful measure is citation precision: of the sources cited, how many truly support the sentence in which they appear.

A complementary measure is coverage: of the important claims in the answer, how many carry a source. In answers about deadlines, warranties, conditions or contract terms, a claim without a source should be treated as an open item.

If the assistant serves customers on WhatsApp, think about format too: the citation has to fit in a short message and make sense to the person receiving it, for example the document name and section, or a link to the official page. For those deciding the flow design, how to integrate AI into your company's WhatsApp covers the channel in detail.

Illustrative scenario: reading the results together

Imagine, as an illustrative scenario, a services company that built an internal assistant to look up warranty policies and service deadlines. The team runs the test set and sorts the results into four groups.

• Right passage retrieved, answer faithful, citation correct: working, nothing to do.
• Right passage retrieved, but answer unfaithful: the problem is in the instructions or the model. Adjust the instruction text to answer only from the passages and test again.
• Right passage not retrieved: the problem is in the search or the documents. Review how documents are split into passages, their titles and the synonyms customers use.
• Answer faithful, but citation points to the wrong document: the problem is in the step that links each claim to its source. Review how the document metadata reaches the model.

The value is that each group leads to a different action and a different owner. The person who maintains the documents fixes the third group, the person who maintains the instructions fixes the second. Without the separation, everyone argues about the final answer and nobody changes anything in a verifiable way.

How do you track it day to day, and where does our method fit?

After launch, evaluation becomes a light routine. Rerun the test set whenever documents, instructions, the model or the search configuration change. Compare with the previous round and record what improved and what got worse. A drop on questions that were previously correct is the most valuable signal in the process.

In production, log the question, the retrieved passages, the answer and the cited sources for each interaction. These logs let you review real samples and feed new cases into the test set. The observability concepts from OpenTelemetry help you think of these logs as traces that explain the system's behavior. Be careful with what you log: conversation logs can contain personal data and need a retention period and access control. And since the document base is text input to the model, it is worth knowing the risks described in prompt injection in AI agents.

Our method follows four steps that fit well here. Map: gather the real questions, the documents that answer them and the current process as a baseline. Limit: define which documents the assistant may consult and what it never answers on its own. Pilot: run a real flow with logs and a success metric agreed in advance. Transfer: hand the test set and the review routine to the team, which decides whether to expand, adjust or stop.

For the operations team, it is worth starting with a small dashboard of a few indicators reviewed at regular intervals:

• Retrieval hit rate on the test set.
• Share of faithful answers in the reviewed sample.
• Citation precision in the same sample.
• No-answer cases handled correctly (refusal or handoff).
• Correction requests from users or human agents.

If the assistant serves customers, connect these indicators to your support metrics, as we discuss in customer support KPIs with AI. RAG quality explains part of what shows up there, but not all of it: tone, channel and handoff to a human also weigh in.

Frequently asked questions

How many questions do I need to start evaluating a RAG system? Start with a small set that still covers the main topics, the real ways people ask, and the cases with no answer in the knowledge base. A modest set well reviewed by someone who knows the content is better than a large one generated without checking. It grows with the errors you find in production.

Can I use an AI model to evaluate my RAG system's answers? You can, mainly to triage volume and flag likely errors. As a working hypothesis, treat the result as support: keep a sample reviewed by a person and compare the two evaluations from time to time. If they diverge a lot, adjust the criteria before trusting the automatic triage.

What should I do when the answer is right but the cited source is wrong? Treat it as a citation failure, not a success. The cause is usually in how the document metadata reaches the model, or in duplicate passages from different versions. Fix the source of the problem, rerun the test set and confirm that answers that were correct before are still correct.

Do I need to evaluate again after the RAG system goes into production? Yes, because documents, policies and models change, and an improvement in one place can make another worse. Rerun the test set after every relevant change and review a sample of real conversations, respecting LGPD when handling those logs. If the evaluation shows the system does not meet the agreed metric, stopping or narrowing the scope is also a valid decision.

Sources

• AWS: What is retrieval-augmented generation
• OpenAI: evals guide
• Anthropic: Demystifying evals for AI agents
• OpenTelemetry: observability concepts