Superintelligence for Test Prep Schools: Data and Assessment

Superintelligence for test prep schools, today, is not a intelligence above human level making teaching decisions. In practice, it is a layer of AI agents that reads the question bank, the student's history and the course material to suggest the next study step, with clear limits and teacher review. What decides whether this helps or harms is how the data is organized and whether personalizing study stays separate from assessing the student.

Key takeaways

• An agent only personalizes well if the question bank has reliable metadata: topic, skill, difficulty, origin and review status.
• Performance data should be used for a defined purpose, with restricted access and a record of every recommendation made.
• Personalizing study (what to review, in what order) is different from assessing (grades, pass or fail, ranking). The second function should stay with the teaching team.
• Student support through an agent works with a narrow scope, approved sources and a handoff to a person when the question is pedagogical or sensitive.
• Before opening it to students, test with real cases, agreed success criteria and a pilot on a single workflow.

What superintelligence means in a test prep school

The term attracts attention, so it helps to be precise. The systems available to companies today are language models and agents that carry out bounded tasks. They summarize, explain, classify, suggest and query data sources. They have no pedagogical judgment of their own and no responsibility for a student's result. For a deeper look at the difference, read Is ChatGPT Superintelligence? What It Is and What It Is Not.

For a test prep school, this precision matters for a simple reason: the product being sold is preparation for an exam, and the trust of students and families depends on correct content and fair assessment. An agent that gets an answer key wrong or invents a justification does direct damage. So the useful question is not how intelligent the system is, but which tasks it can take on, with which data, and under what supervision.

Three areas tend to concentrate the value, and this article focuses on them:

• Content: organizing and maintaining the question bank and supporting materials.
• Student: turning performance history into study suggestions.
• Support: answering operational and content questions within limits.

How to organize the question bank before any agent

The most common mistake is to connect an agent to a folder of PDFs and expect it to understand the course. An agent retrieves passages and generates answers from them. This technique is known as retrieval-augmented generation (RAG), and AWS publishes a clear explanation of it. If the source base is disorganized, duplicated or out of date, the answer inherits the problem.

A question bank prepared for agents has, for each item, at least these fields:

• Question text, options and answer key checked by a teacher.
• Topic and subtopic, using the same taxonomy across the whole course.
• The skill or competency the question requires.
• Difficulty level defined by an explicit criterion, not by impression.
• Origin: original question, previous exam or adapted, with the permitted use noted.
• An approved worked solution, stored separately from the question text.
• Review status and the date of the last check.
• A flag for questions reserved for assessment (mock exams and tests), which the agent must not reveal or comment on before they are administered.

That last field is what protects the assessment. Practice questions and exam questions should live in different collections with different permissions. The study agent queries only the practice collection and the approved solutions.

Another precaution: separate what is an official source (exam notice, reference matrix, legislation) from what is the school's own material. When the agent answers, it should say where the information came from. If there is no source in the base, the correct answer is to say it did not find one and refer the student to the teacher.

What performance data the agent can use

Personalization requires student data: correct answers by topic, time per question, attendance, repeated attempts, mock exams taken. This data is personal and, in many schools, belongs to minors. That calls for attention to Brazil's data protection law (LGPD), especially purpose, necessity and the care required with data about children and teenagers. The national data protection authority (ANPD) publishes official guidance that should be reviewed together with the school's legal counsel.

A prudent data design follows these rules:

• Define the purpose in writing, for example suggesting content review. Do not use the same data for another purpose without a new analysis.
• Give the agent only what it needs: a performance summary by topic is usually enough, without full name, ID number or contact details.
• Keep performance data separate from sensitive data, such as health information or specific learning needs, which require their own treatment.
• Log every time the agent queries a student's history and every recommendation it issues.
• Let the student and the guardian see what was suggested and why.

If the agent's recommendation could influence something significant for the student, such as a class change or a mandatory study plan, the discussion about automated decisions and the right to review comes in. The article LGPD Automated Decision-Making: The Right to Review (Art. 20) helps explain the point. The practical rule is that the recommendation serves as support and a person decides whenever the effect is significant.

How to personalize study without compromising assessment

The dividing line is this: the agent helps the student study, and the teaching team assesses. When the agent starts assigning grades, labeling a student as ready or not ready, or predicting approval as if it were fact, it takes on a role that requires human judgment, transparency and the possibility of contesting the result.

A simple decision guide for the team:

• The agent can suggest topics to review based on recent errors, explaining the reason.
• The agent can assemble exercise lists from the practice collection, respecting the progressive difficulty defined by the course.
• The agent can explain an already approved solution in simpler language, without changing the answer key.
• The agent can remind students of study routines and course deadlines.
• The agent should not assign an official grade, rating or ranking.
• The agent should not reveal questions reserved for mock exams and tests.
• The agent should not state a chance of passing as a certainty.
• The agent should not grade essays with official effect without teacher review. It can offer a preliminary comment, labeled as such.

There is also a subtle pedagogical risk: an agent that hands over the answer too quickly reduces the student's effort. It is worth configuring the support style in stages: a hint, then a partial explanation, and only then the full solution. This decision is pedagogical, not technical, and should come from the teachers.

Another point is data bias. If a student's history is short, the agent may infer poorly. The instruction should tell it to treat little evidence as little evidence, without labeling the student. A low-confidence study suggestion should come as a question or a proposed diagnostic, not as a conclusion.

How to structure student support with agents

Support is the most visible part. Students ask in the channel they already use, often WhatsApp, and expect a quick reply. An agent can cover operational questions (office hours, this week's material, how to access the mock exam) and content questions anchored in approved material.

The support design needs three elements. First, a narrow, written scope: what the agent answers and what it never answers. Second, approved sources: the content base and the school's rules. Third, a handoff to a person with context, so the student does not have to repeat the story.

These should go straight to a person:

• Questions about exam correction or grades.
• Requests to review an answer key.
• Reports of anxiety, distress or personal difficulty.
• Financial, contract or enrollment issues that fall outside the script.
• Any message from a minor that involves a sensitive situation.

Illustrative scenario: a student writes in the evening saying they missed several questions on the same topic and wants to know what to do. The agent checks the performance summary by topic, sees that the errors cluster in one subtopic, suggests a short review sequence with questions from the practice collection and cites the matching lesson. It then offers to book office hours with the subject teacher. If the student writes that they feel so discouraged they want to quit, the agent does not give advice: it records the context and alerts the coordination team for human contact. Every step stays in the conversation log.

For the channel and school operations side, AI for Schools: Admissions and Parent Communication Guide covers points that also apply to test prep schools.

How to test before putting it in students' hands

A pedagogical agent should not be evaluated by impression. Build a set of real, anonymized cases with the expected answer defined by the teachers. Include easy questions, ambiguous questions, attempts to obtain reserved questions, requests for grades and emotional messages. Then run the agent on those cases every time the instruction, the model or the base changes. The general approach to evaluating agents is well explained in Anthropic's engineering article on evals for AI agents.

A lean test plan:

• First: gather the cases with the coordination team and define what counts as a correct, acceptable and unacceptable answer.
• Second: test the agent in an isolated environment, with fictitious data and simulated integrations. Mock Tools for Testing AI Agents: Safe Sandbox Testing Guide explains how.
• Third: check whether the agent cites its source and admits when it does not know.
• Fourth: try to break the separation between practice and exam content by asking for reserved questions in indirect ways.
• Fifth: release it to a small group of students, with a teacher following the logs.
• Sixth: review the logs, adjust, and only then decide whether to expand.

For general governance, the NIST AI Risk Management Framework offers a common language to map, measure and manage the risks of AI systems, which is useful when talking with leadership and legal counsel.

How the working method applies

The method we use in AI projects has four steps: map, limit, pilot and hand over. In a test prep school, mapping means finding out how the question bank is maintained today, where performance data lives and which questions reach support most often, recording a baseline before any automation. Limiting means deciding which data the agent sees and which actions it can take, keeping assessment out of its reach. Piloting means choosing a single workflow, such as review suggestions by topic, with logs and a success metric agreed with the coordination team. Handing over means giving the documented workflow to the team and deciding, based on the records, whether to expand, adjust or stop.

When the school also offers training for companies, part of this structure repeats. Superintelligence in Corporate Training shows how learning paths and practice can be organized with the same logic.

Frequently asked questions

Can an AI agent replace the teacher in a test prep school? That is not the recommended design. The agent can absorb repetitive tasks, such as suggesting review, explaining approved solutions and answering operational questions. Pedagogical diagnosis, assessment and follow-up on sensitive cases stay with the team.

Can the agent have access to the questions used in mock exams? Only after they have been administered, and only if the school decides to release them. Reserved questions should sit in a separate collection with their own permission, so the study agent does not reveal them early and compromise the assessment.

Do we need a large question bank to start? No. It matters more that the initial set is small and well organized, with a checked answer key, classified topics and an approved solution. Starting with one subject or one topic lets you test the workflow and fix the structure before expanding.

How should we handle student data, especially for minors? Define the purpose, give the agent only what it needs, restrict access and log its use. For minors, involve legal counsel and the guardian, and consult the official ANPD guidance. If the recommendation could affect the student in a significant way, keep a person in the decision and provide a path for review.

Sources

• Anthropic: Demystifying evals for AI agents
• NIST AI Risk Management Framework
• AWS: What is retrieval-augmented generation
• ANPD: Autoridade Nacional de Proteção de Dados