Extract Data from PDFs with AI: Fields, Rules and Human Review

Extract Data from PDFs with AI: Fields, Rules and Human Review

To extract data from PDFs with AI reliably, it is not enough to ask a model to read the file. You need to define in advance which fields matter, validate every result against objective rules, and send anything doubtful to a person before writing anything into your system. The AI reads and proposes, the rules check, and only then does the data go in.

Key takeaways

• Define the fields for each document type before choosing a tool: name, type, whether required, and where the value usually appears.
• The AI should return each value with the source passage and a flag for doubt, not just the final value.
• Validate with deterministic rules (check digits, dates, totals, master data), not with the opinion of the same model.
• Doubtful cases go to a human review queue, and the system receives the data only after that review.
• Start with one document type, keep a log of decisions, and expand only after you have measured the result.

Why payment slips, contracts and lab reports need different treatment

Every PDF looks like the same problem, but the risk changes a lot with the document. A Brazilian payment slip (boleto) has a predictable structure and data that can be checked by calculation. A contract has free-form language, clauses that vary, and information spread across many pages. A lab or technical report mixes technical text, tables, reference values and, often, scanned images.

So the useful question is not which model is best, but how much an error costs for that document. A wrong amount on a payment slip causes an improper payment. A wrong effective date in a contract can mean missing a renewal or termination deadline. A wrong number in a report can feed a technical or clinical decision. The higher the cost of an error, the stricter the validation should be and the earlier human review should enter.

Another difference is where the file comes from. A system-generated PDF has selectable text and usually extracts well. A scanned PDF, or a photo saved as a PDF, depends on image recognition, and the uncertainty is higher. It is worth separating the two flows from the start, because the confidence threshold should not be the same.

How to define the fields before you turn on the AI

The most common mistake is starting with the prompt. The right starting point is a field sheet, built with the person who types this data into the system today. That person knows which fields actually block the process and which are irrelevant.

For each field, record:

• The field name in your system (for example, due date, amount, payee, tax ID).
• The expected type: text, date, monetary amount, fixed-format identifier, list of options.
• Whether it is required or optional.
• Where it usually appears in the document and how that can vary.
• What to do when it is missing: leave it empty, never invent it.
• Which rule confirms that the value is correct.

Two precautions make a difference. The first is to allow the answer to be not found. Models tend to fill gaps, and an empty field is far less dangerous than a plausible but wrong one. The second is to ask for the exact passage of the document each value came from. That lets the reviewer check in seconds, and it lets an automatic rule verify that the passage really contains the value.

The output should follow a fixed, structured format, with the same field names every time. This makes validation and logging easier. Anthropic's guidance on building effective agents argues for exactly this kind of design: simple, verifiable steps before any greater autonomy.

How to validate extraction against rules

Validation is what separates an impressive demo from a process the finance team can rely on. The golden rule is that the check cannot depend on the same model that did the extraction. If the AI makes a mistake and then confirms its own mistake, nothing was validated.

Types of rules that tend to work:

• Format and check digit: tax IDs and the payment slip's digitable line carry control digits that can be calculated in code.
• Internal consistency: the slip's amount should match the amount encoded in its digitable line, and the due date should be consistent with the one in that line.
• Consistency with master data: does the payee exist in your supplier database? Is the tax ID in the contract that of a known party?
• Consistency with the order: is the amount within what was authorized in the purchase order or contract?
• Plausible ranges: dates in the distant past or absurd future, zero amounts, or amounts far above the history.
• Duplicates: has this same slip already been entered?
• Source presence: the passage cited by the AI really contains the extracted value.

Ideally each field gets a simple result: passed, failed or could not be checked. That result goes into the log together with the value, and it decides the next step. To go deeper into checking inputs and outputs before an action, the article on guardrails for AI agents details the pattern.

When to send a document to human review

Human review is not a sign that automation failed. It is the correct design for any data that will change accounts, payments or obligations. The question is deciding which cases go to the queue and which pass straight through.

A simple decision guide:

• All rules passed and the document is from a known supplier: it can proceed to entry, with periodic sampling by a person.
• Any rule failed: it goes to the queue, with the problem field highlighted.
• A required field is empty or marked as doubtful by the extraction: it goes to the queue.
• A scanned document of low quality, skewed or cropped: the whole document goes to the queue.
• An amount or deadline above a limit the company sets: it goes to the queue, even if everything passed.
• A new document type never seen in the pilot: it goes to the queue until it is mapped.
• A contract with a clause outside the expected standard: it goes to legal or the responsible owner, not to a data entry clerk.

The review screen should show the document next to the extracted fields, with the source passage highlighted and the reason the extraction stopped there. If the reviewer has to open the PDF in another window and hunt for the value, the queue becomes a bottleneck and the team goes back to typing everything. Every correction a person makes is also valuable information: it shows where the field sheet or the rules need to improve.

When the extracted data leads to a decision about a person, such as a credit assessment or the rejection of a benefit, the right to review also comes into play. The piece on automated decisions under the LGPD helps you understand when this applies.

Illustrative scenario: accounts payable with supplier payment slips

Imagine, as an illustrative scenario, a mid-sized company that receives supplier payment slips by email and WhatsApp. Today a person opens each PDF and types the payee, amount, due date and digitable line into the finance system.

In the pilot, the flow would look like this. The file arrives and is classified: payment slip or another document. The AI extracts the defined fields, each with its source passage. Code recalculates the check digits of the digitable line and compares the amount and due date with the data read from the text. The payee is looked up in the supplier master data, and the slip is compared with the open purchase order or contract. If everything matches, the entry is ready for the finance team's normal approval. If the supplier is new, if the amount differs from the order, or if the digitable line does not validate, the item lands in the review queue, with the reason written in plain language.

In this design, the AI never pays anything and never changes master data. It reduces typing and makes what is suspicious more visible. If your finance system is an ERP such as Bling, Omie or Tiny, first look at the integration paths via API and their risks, because writing to the system should go through a controlled interface with minimal permissions.

A staged rollout plan

The method we use for this kind of project has four moves: map, limit, pilot and hand over. Applied to PDF extraction, it looks like this.

• Map: find out how the document arrives, who types it today, into which system, which errors already happen and how long the process takes. This initial picture is your baseline for comparison.
• Limit: choose a single document type, a short list of fields, the data the AI may see and the actions it may not take. Remove from what you send anything that is not needed for extraction.
• Pilot: run on real documents in parallel with the manual process, logging every extraction, validation and human correction, with a success metric agreed in advance.
• Hand over: give the documented flow to the team, with a named owner, and decide from the logs whether the project expands, gets adjusted or stops.

During the pilot, build a set of test documents whose correct answers have already been checked by a person, and run the extraction against it whenever you change the prompt, the model or the rules. Anthropic describes this practice in its writing on how to think about evaluations for AI agents. To choose what to measure and when to decide, the guide on how to measure an AI automation pilot offers decision criteria.

Risks that are often forgotten

The first is malicious content inside the document itself. A PDF can contain text that tries to give instructions to the model, such as telling it to ignore rules or change a field. Treat all extracted text as data, never as instructions, and keep the model without access to tools that write to systems. This type of attack is described in the OWASP Top 10 for LLM applications.

The second is privacy. Contracts and reports often carry personal data, and medical reports can carry health data, which the LGPD treats as sensitive. Define the legal basis with your legal team, send the model only what is necessary, check where and how the provider processes and retains data, and log who accesses the review queue. Brazil's data protection authority (ANPD) is the official reference for guidance on the subject. If the project will process sensitive data at scale, consider a data protection impact assessment before going to production.

The third is drift over time. Suppliers change the layout of their slips, legal changes the contract template, the lab changes the report format. Without monitoring, quality drops and nobody notices. Track the share of items that go to the queue and the most corrected fields: if a field starts being corrected often, something has changed.

Checklist before writing to the system

• Was the field sheet validated by the people who run the process today?
• Does the extraction accept an empty field and return the source passage?
• Is there at least one deterministic rule for each critical field?
• Are the criteria for going to the review queue written down and simple?
• Does writing to the system use minimal permissions and record who approved?
• Is there a set of test documents with checked answers?
• Is someone responsible for reviewing the logs periodically?

Frequently asked questions

Can you extract data from PDFs with AI without human review?

For low-risk documents, with strong validation rules and known suppliers, some items can pass straight through, with periodic sampling. For payments, contractual obligations, and technical or health data, we recommend keeping a person in the final decision on doubtful cases. What goes straight through and what goes to the queue should be set by a written rule, not by impression.

Does a scanned PDF work as well as a PDF with text?

Generally, no. When the file is an image, reading depends on visual recognition, and the chance of error in numbers and dates rises. So separate the two flows, use stricter rules for scanned files, and send low-quality documents to review.

Does the AI need to be trained on my documents?

In most extraction cases, training a model is not necessary. The safest path is usually to describe the fields well, provide examples of the expected format, and measure the result on a test set of real documents. Consider something heavier only if this simple path does not reach the quality agreed in the pilot.

Where should I start in my company?

Choose the document that consumes the most typing time and has clear checking rules, such as recurring supplier payment slips. Map the current process, limit fields and permissions, run a pilot that logs everything, and decide from the data whether to expand, adjust or stop.

Sources

• Anthropic: Building effective agents
• Anthropic: Demystifying evals for AI agents
• OWASP Top 10 for LLM Applications
• ANPD: Autoridade Nacional de Proteção de Dados