Company Superintelligence Layer: Memory, Permissions and Evaluations

Company Superintelligence Layer: Memory, Permissions and Evaluations

A company superintelligence layer: memory, tools, permissions, evaluations and human approval around any AI model. Swap models without rebuilding.

•

Want to apply this in your company?

Book a call with Juan →

A company superintelligence layer is the operating structure built around an AI model: memory, tools, permissions, evaluations and human approval. The model provides the ability to interpret and draft, but the layer decides which data it can see, which actions it can take and who checks the result. Because that layer sits apart from the model, a company can switch models when a better option appears without rebuilding the whole process.

Key takeaways

  • The layer sits between the model and the business: it defines context, tools, limits and records.
  • Memory must be curated, because the agent should query what it is authorized to use, not everything that exists.
  • Permissions should be granted action by action, with the minimum needed for the process to work.
  • Evaluations based on real cases show whether a model change improved or worsened the result.
  • Human approval belongs at the points of risk, with a record of what was decided and why.

What the layer contains, piece by piece

The most useful way to think about the layer is as a set of contracts. The model receives a context, may call a few tools, returns a response and leaves a record of what it did. Each piece needs an owner inside the company, a criterion for showing it works and a safe way to be switched off.

Memory covers what the system knows at the moment it responds. It can be the current conversation, a document base consulted on demand (the pattern known as retrieval-augmented generation) or a set of registered facts, such as the status of an order. Tools are the integrations the agent triggers, such as checking the inventory system or creating a proposal draft. Permissions define what each tool is allowed to do. Records keep inputs, outputs and decisions so someone can reconstruct the path later. Evaluations measure quality using known cases. Human approval defines who takes the decision when a case falls outside the rules.

None of these pieces depends on a specific model. That is why, when a new model improves the writing or the reasoning, the company can test it against the same evaluations, the same permissions and the same records. If it gets worse at some point, the change is reverted without drama.

Memory: what the agent should remember and what it should forget

Memory is where companies most often overreach. The temptation is to give the agent access to everything, in the hope that it will answer better. In practice, the more irrelevant context enters the conversation, the greater the chance of a confused answer or of one customer's information appearing in the wrong place.

A simple rule helps: memory should contain only what is needed for the next step of the process. For an after-sales agent, that might be the delivery date, the warranty status and the last contact. A full financial history, for example, usually does not need to be available to it when it answers a question about a deadline.

In the Brazilian context, this decision is also a legal one. The LGPD requires purpose, necessity and security in the processing of personal data, so the agent's memory needs a retention period, a way to correct information and an identified owner. If the knowledge base or the model provider is located outside Brazil, it is worth checking the rules on international data transfer when using AI before releasing any personal data.

Memory quality also depends on the quality of the source data. An agent only retrieves well what is well organized, with consistent field names, dates in the same format and clear document versions. Without that foundation, the layer simply spreads confusion more eloquently.

Permissions: every action needs a reason

The most important principle here is least privilege. The agent should have read access when it only needs to look something up, and write access only for specific actions, with limits on value, quantity or type of record. A single credential with full power over the entire system is the shortest path to an incident.

There is also a specific risk with language models. Text that arrives from outside, such as customer messages, review comments or the content of pages the agent reads, can contain disguised instructions that try to change its behavior. The OWASP guidance on prompt injection recommends treating that content as data rather than as commands, and limiting what the agent can do even when it is manipulated into attempting something improper. For that reason, permission to issue a refund should not depend only on an instruction from the model, but on a rule verified by the system itself.

In practice, this comes down to three decisions the manager must make before any pilot: which tools the agent may call, under what conditions each one can be used, and which actions always stay with a human. These decisions should be written down, not only placed in the prompt.

Evaluations: how to know whether a model change improved the process

An evaluation is a set of real cases, each with the expected answer or an acceptance criterion. Without it, the company only notices a problem when a customer complains. With it, a model change stops being an opinion and becomes a comparison.

A useful set usually includes common cases, edge cases that have already caused confusion, cases where the correct response is to hand off to a person, and some attempts at diversion. The criteria must be observable: the information is correct, the right tool was called, the limit was respected, the handoff happened when it should have.

The evaluation documentation published by OpenAI and Anthropic's guidance on evaluating agents converge on one point: assess the final result and also the intermediate steps, because an agent can reach the right answer through a path the company would not accept. The same tests should run before and after every change to the model, the prompt or the tools.

To test without putting real customers at risk, simulated environments are worth using. This topic is covered in how to test AI agents with mocks and a sandbox.

Human approval: where a person needs to decide

Human approval is not a bottleneck to eliminate. It is a design choice. The useful question is what it costs to get each type of decision wrong, and who has the authority to bear that cost. A simple guide helps make that separation:

  • Informational answers backed by a verified source can be sent by the agent, with a record.
  • Changes to a customer record, deadline or order status can be made by the agent if there is a clear rule and the change can be reversed.
  • Refunds, discounts, policy exceptions and promises of special delivery terms require approval from a person.
  • Complaints showing signs of damage, legal exposure or sensitive data should leave the automated flow immediately.
  • Any case where the agent is uncertain or falls outside the rules should become a handoff with a summary of the context.

The decision record matters as much as the approval. It should state what was requested, which data was used, which rule was applied, who approved it and when. Without it, the company cannot audit or improve the process.

A six-step assembly plan

For a company that wants to move from idea to a controlled pilot, a practical sequence usually works better than starting with the model:

  • Choose a real process and record how it works today, including the exceptions the team already knows.
  • List the data sources and define, for each one, what the agent can read and for how long.
  • Design the tools with minimum permissions and written criteria for use.
  • Build an evaluation set with real cases, including edge cases and handoff cases.
  • Run a pilot with complete records, human supervision at the risk points and a success metric agreed in advance.
  • Based on the results, decide whether the layer expands, adjusts or stops, and transfer the operation with documentation.

Illustrative scenario: switching models without changing the process

Illustrative scenario: a distributor uses an agent on WhatsApp to confirm orders, report delivery status and open exchange requests. The layer limits memory to the customer's own orders, provides lookup and ticket-opening tools, allows write access only for tickets, and requires handoff for any price exception request. When a new model appears, the team runs the evaluation set covering orders, exchanges and ambiguous cases. The new model writes better, but makes more mistakes reading delivery dates around public holidays. The team adjusts the deadline calculation rule, which lives in the system rather than in the prompt, repeats the tests and only then approves the switch. None of this required rewriting the workflow, because the operation was separated from the model from the start.

How Juan's method applies

The method we use follows four steps: map the current process and its baseline, limit data and actions, pilot a real flow with records and an agreed metric, and then transfer the operation and decide whether to expand, adjust or close it. The layer described here is what makes the limiting step concrete, because that is where permissions, memory and approval take shape. When a company also needs ongoing operation, that work can be run by Be Human (be-human.ai), following the same principles of records and supervision.

Frequently asked questions

Is a company superintelligence layer a product you buy off the shelf? No. It is a way of organizing operations around available models, with memory, tools, permissions, evaluations and human approval defined for the company's own process. Some parts can use ready-made platforms, but the decisions about data and actions remain the company's responsibility.

Do I need to wait for a more advanced model before starting? No. The value of the layer lies in organizing the process, the data and the limits, which can be done with current models. When a better model appears, switching is simpler because the operation is already separated from the model.

How do you keep the agent from doing something it should not? The most effective combination is limiting permissions in the system itself, treating external content as data, requiring approval for risky actions and keeping complete records. No instruction in a prompt replaces a rule verified by the system.

Where should a small company start with a layer like this? Start with a repetitive, low-risk process, map the current workflow, define what the agent may read and do, and build a small set of real cases for evaluation. A pilot with an agreed metric and complete records will quickly show whether expanding makes sense.

Sources