
Superintelligence and Public Health in Brazil: Limits for SUS
Superintelligence in public health is still a hypothesis. Practical limits for Brazil's SUS: governance, protected data and human clinical decisions.
Want to apply this in your company?
Book a call with Juan →Superintelligence, meaning systems that would outperform humans in nearly every area, does not exist yet, and there is no consensus on when or whether it will. For public health, the useful question today is different: which governance, data and human clinical decision limits should apply to any advanced AI used in Brazil's public health system (SUS), in health insurers or in clinics? The practical answer is to treat powerful models as support tools under control, never as the authority on diagnosis, treatment or access to care.
Key takeaways
- Superintelligence is a speculative concept. Health decisions should rest on systems that exist today, with measured performance and a named owner.
- Clinical decisions stay with a qualified professional, with an identifiable reviewer and a clear path for the patient to contest the outcome.
- Health data is sensitive. A stated purpose, data minimization and access control come before any integration.
- No expansion without a pilot that uses real records, a success metric agreed in advance and a stopping rule.
- A conversational agent can handle administrative flows such as scheduling and documents, but it should not interpret symptoms or give clinical guidance.
What the word superintelligence means here, and what it does not
In the press, the term is used to describe everything from popular chatbots to science fiction scenarios. That mix gets in the way of good decisions. A health manager needs to know which tool is in use, what it does, what data it receives and who is accountable for the result. Current language models are narrow or general purpose systems with uneven behavior: they perform well in some contexts and fail convincingly in others. Treating this technology as if it were already a superior intelligence leads to two common mistakes, delegating too much and rejecting everything out of abstract fear. To separate the concepts calmly, read Is ChatGPT Superintelligence? What It Is and What It Is Not.
Why public health requires stricter limits
An error in a health system does not affect one isolated person in the same way that an error in a customer database does. It can reach an entire queue, a region or a group that already has less access to care. The population served also includes people in vulnerable situations, with low digital literacy or little ability to challenge a refusal. These factors call for limits before the technology is deployed, not after. The larger the scale and the smaller the capacity for individual review, the more important it is to design the process so that a human can notice and correct failures in time.
Governance: who is accountable when the AI gets it wrong
The first question is not technical. Before any pilot, the institution must know who is responsible for the use, who approves changes and who answers to a patient or to a regulator. The NIST AI Risk Management Framework organizes this work into functions of governing, mapping, measuring and managing risk across the system's lifecycle. For a manager, that translates into a simple list of controls:
- An inventory of every AI use, including those teams adopted on their own.
- A risk classification for each use, separating administrative tasks from anything that affects decisions about people.
- A named owner for each system, with authority to suspend its use.
- A record of decisions, instruction versions and incidents, kept for a defined period.
- Periodic review of sampled conversations and decisions, performed by someone outside the team that operated the system.
That last point is the one most often missing. Without independent sampling, the institution discovers the problem through a complaint instead of through measurement.
Data: what can enter the system and what cannot
Health data is among the sensitive categories covered by Brazil's LGPD, which requires extra care with purpose, legal basis, security and sharing. In practice, the most useful principle is to ask, for each field, whether the agent truly needs it to complete the task. An agent that confirms appointments does not need the full clinical history. An agent that explains required documents does not need the diagnosis. It also helps to separate what may be used for training, what may only be consulted during a service interaction and what must never leave the institution's environment. When a foreign provider is involved, the international transfer analysis becomes part of the picture, as explained in International Data Transfer Under LGPD When Using AI Providers. Brazil's data protection authority (ANPD) publishes guidance that should be checked before decisions of this kind.
Human clinical decisions: the limit that is not negotiable
There is an important difference between supporting a decision and making it. A system can summarize records that a professional is authorized to see, organize queues, remind staff of pending exams, prepare reviewed educational material or flag cases that need urgent attention from a nurse. Diagnosing, prescribing, classifying clinical risk as final, or refusing care based on an automated score crosses the line. The LGPD addresses automated decisions and the right to review them, a topic covered in LGPD Automated Decision-Making: The Right to Review (Art. 20). The OECD principles for trustworthy AI reinforce human oversight and accountability, and they are a useful reference for any institutional design.
One warning sign: when the team starts accepting the system's recommendation without reading the case because "it usually gets it right," the limit has already been crossed, even if no formal rule has changed.
Illustrative scenario: scheduling and guidance at a health unit
Illustrative scenario, not a real case: imagine a basic health unit with a backlog of follow-up appointments and many repeated questions over WhatsApp about hours, documents and cancellations. An AI agent answers only those questions, using a list approved by the coordination team. When a message contains signs of urgency, such as severe chest pain, shortness of breath or heavy bleeding, the agent does not attempt any assessment. It tells the person to seek immediate care and transfers the conversation to a human professional, logging the time and reason. Every conversation is recorded, and someone from the quality team reviews a sample each week. The pilot moves forward only if the follow-up backlog shrinks without an increase in complaints and without any response that went beyond the defined scope.
The point of this scenario is not the agent's efficiency. It is the design: a closed scope, a clear route to a human in emergencies and measurement carried out by someone who did not operate the system.
A staged plan to start safely
The sequence below fits public agencies, clinics and insurers that want to test AI without taking on unnecessary risk:
- Map the current process and set a baseline: how long it takes, who does it and where it fails.
- Limit data and actions: what the system can read, what it can answer and what it never does.
- Pilot one real flow with records, an agreed metric and a stopping rule.
- Hand over operations with named owners, incident procedures and periodic review.
- Decide based on the records: expand, adjust or shut down.
The method we apply in consulting projects follows the same logic: map before automating, restrict data and actions before integrating, pilot a single flow and hand over with documentation. In health, the last step carries more weight. Stopping a pilot that cannot deliver adequate safety is a legitimate decision, not a failure.
Frequently asked questions
Is superintelligence already used in public health? No. The term describes a type of system that does not yet exist. What is available today are language models and analysis tools with uneven performance, which require governance, measurement and human oversight before any sensitive use.
Can AI perform triage in the SUS? It can organize administrative queues and flag urgent signals for a professional, as long as the system does not classify clinical risk or refuse care. Clinical triage decisions must remain with the health team, with records and the possibility of review.
Which data should never go to an external model? Identifiable health data, medical records, reports and patient information without a proper legal basis and contract should not be sent without prior analysis. Before any integration, the institution should check purpose, minimization, where processing happens and ANPD guidance.
Where should a city health department or a clinic start? Begin with a low-risk administrative process, such as confirming appointments or sending document instructions. Define owners, limit the scope, measure with records and expand only if someone outside the operation verifies the results.