Superintelligence Consulting Deliverables: What to Demand From a Provider

Superintelligence Consulting Deliverables: What to Demand From a Provider

The deliverables a serious superintelligence consultancy should produce, from process map to decision log, and how to check each one shows real progress.

•

Want to apply this in your company?

Book a call with Juan →

A serious superintelligence consulting engagement, in practice, is diagnosis, design, and operation of AI systems inside business processes. The value shows up in concrete artifacts: a process map, a measured baseline, boundary rules, a pilot with logs, and a recorded decision to expand, adjust, or stop. If the provider cannot show these items with a named owner and a review method, the manager has little way to know whether real progress happened.

Key takeaways

  • A process map of the current workflow, with a measured baseline taken before any change
  • A boundary matrix that defines accessible data, permitted actions, and mandatory human approvals
  • A pilot on a real workflow, with a success criterion written down before it starts and a named owner for the result
  • A decision log that records what the agent did, which sources it used, and why it handed off to a person
  • A transfer plan that ends in an explicit decision: expand, adjust, or shut down

Why the label matters less than the deliverable

The term superintelligence is used very loosely in the market, which makes the conversation vague from the start. For the person deciding, the useful question is not whether the provider uses the word, but whether they can show documents that let you verify the work. The superintelligence glossary for executives helps separate concept, tool, and operation.

In real operations, an AI agent is a set of parts: a model, instructions, access to data, tools, escalation rules, and records. Each of these parts should appear in some document. When the proposal stays at the level of general promises, the right move is to ask for the artifacts, not for a definition.

The process map and the baseline

The first serious deliverable is the process map as it works today, not as it is supposed to work. It needs to show who performs each step, where requests wait, which exceptions occur, and who decides on them. Without this map, any automation tends to solve the easy cases and push the hard ones onto the team without anyone noticing.

The map should contain, at a minimum:

  • The trigger for each request, whether a WhatsApp message, email, form, or phone call
  • The steps, the owners, and the points where the request gets stuck
  • The known exceptions and who has authority to resolve them
  • The approximate volume by channel and by time of day, taken from records the company already keeps
  • The most frequent errors and how much team time each one takes to correct

The baseline is a snapshot of the process before the change. It can be a sample of conversations read by hand, an extract from a spreadsheet, or the history from the ticketing system. The goal is not a perfect number. The goal is a point of comparison the company accepts as honest. Without a baseline, any improvement later becomes opinion.

Boundary matrix: data, actions, and approvals

This is the artifact that protects the company most, so it deserves the closest reading. The boundary matrix answers three questions: what the agent can read, what it can do on its own, and what requires a person. When it involves customers' personal data, the LGPD requires processing to be limited to its purpose, which in practice means giving the agent only the fields needed for that task.

The matrix should contain:

  • The data the agent can consult, with the purpose of each field
  • The data it does not store, pass on, or use for other purposes
  • The actions it may execute without approval, such as checking order status or suggesting time slots within a fixed rule
  • The actions that require human approval, such as deadline exceptions, discounts, refunds, or changes to customer records
  • The situations that force a handoff to an agent, such as a serious complaint, a cancellation request, or a sign of urgency

It is also worth deciding what does not need AI at all. As Anthropic describes in its guidance on agents, workflows with fixed steps and clear rules tend to be more predictable than agents that decide the next step on their own. In business processes, the safer combination is usually deterministic automation for what is rule-based, with AI only where judgment is needed. Customer messages should also be treated as untrusted input, since they can try to override instructions. The OWASP Top 10 for LLM applications lists this kind of risk and is a useful reference for the review.

Pilot on a real workflow and a success criterion

The pilot should be deliberately small. One channel, one type of request, one defined time window. The temptation to switch everything on at once is common, and it tends to hide what did not work. Before exposing customers, it is worth testing with simulated data in an isolated environment, as described in the article on testing agents with mocks and sandboxes.

A minimum checklist for the pilot:

  • A success criterion written before the start, with one metric and a clear stopping condition
  • A process owner inside the company, with authority to decide on exceptions
  • An observation period with periodic review of conversations chosen at random, not only the ones that went wrong
  • A list of test cases that includes out-of-scope questions and attempts to pull the agent away from its function
  • A rollback plan that returns service to manual mode immediately

Illustrative scenario: a parts store decides to test the agent only for order status questions, during business hours. The criterion is that every status answer is checked against the order system and that any refund request goes to the human queue. After a few weeks of review, the team notices questions about delivery timelines in specific cases that were outside the scope, and it decides to widen the handoff rule before considering any expansion.

Decision log: the record that supports accountability

The decision log is the document that shows what the system did in each case and why. It is the basis for holding the provider to results, because it makes the agent's work auditable. Without it, the company only knows what customers complained about, not what happened in the cases that went right.

Each entry should contain, at a minimum:

  • The conversation identifier and the originating channel
  • A summary of the input and the intent identified
  • The sources consulted and the version of the instructions in force
  • The action taken, including the reply sent and the systems accessed
  • The reason for the handoff to a human, when one occurred
  • The later review by a person and the correction recorded, if any

The log has to be useful without becoming a file of unnecessary personal data. For that reason, the consultant should define how long logs are kept, who can read them, and which fields are masked. For general background on observability, the idea of event tracing is a reasonable starting point.

Evaluations before release and after every change

An evaluation, or eval, is a set of cases with expected results that runs whenever the instructions, the tools, or the model change. It prevents a fix in one place from breaking something else without anyone noticing. The provider should show its test cases and the results of its latest run, not just describe that testing exists.

The NIST AI Risk Management Framework organizes AI risk management into the functions of govern, map, measure, and manage. It works well as a list of questions for the provider: who is responsible for the risks, how they were identified, how they are measured, and what happens when one of them materializes.

Transfer, decision, and Juan Carlo's method

In how we work, the process follows four steps: map the process and the baseline, limit data and actions, pilot a real workflow with logs and an agreed metric, and transfer with a decision to expand, adjust, or stop. The sequence exists so the company knows at every stage what has been delivered and what has not yet been proven. The details of how this work is organized are in the piece on how Juan Carlo works with companies.

A transfer is only serious when it leaves the operation in the hands of someone inside the company, with enough documentation to do so. Items to expect:

  • A runbook with the agent's instructions, its limits, the handoff procedure, and escalation contacts
  • An internal owner and a backup for decisions the agent must not make on its own
  • A cadence for reviewing logs and evals, with who does it and how often
  • A register of incidents and how each one was handled
  • A final decision, written and signed by someone with authority, to expand, adjust, or shut down

Checklist for holding a provider to concrete results

Before approving the next stage of a project, the manager can ask a few objective questions. Good answers usually come with documents, not only verbal explanations.

  • Show the process map used as the base, and who validated that version inside the company
  • Which data the agent accesses, and why each field is necessary for the task
  • What the pilot's success criterion is, and who decides whether it was met
  • Where the logs are stored, and which people in the company can read them
  • Which test cases failed in the latest review, and what was changed afterward
  • Who takes over the operation after the transfer, and how that person is trained

Frequently asked questions

Is superintelligence consulting the same as hiring a chatbot? Not necessarily. A chatbot is a tool, and a consulting engagement should cover what comes before and after it: process diagnosis, boundaries, pilot, records, and transfer. If the scope amounts to configuring a tool, the artifacts needed to evaluate the result are missing.

Do I need to understand programming to ask for these deliverables? No. The manager needs to understand the process, the risks to the customer, and what a business decision means. The documents described above are meant to be read by the person making the decision, and a serious provider should be able to explain them without jargon.

When does it make sense to stop a pilot? It makes sense when the stopping criterion written at the start is reached, or when the review shows errors the team cannot correct within the agreed rules. Stopping is also a valid decision, and it should be recorded. It is not a failure of the project. What matters is that the choice is based on the logs and not on the impression of whoever is closest to the problem.

What should I do if the provider cannot show a decision log? Ask them to explain how the record would be generated before any expansion, which fields it would contain, and who would have access. If the answer stays vague, the expansion should wait. Without a record, the company cannot verify what the system did, and so it cannot hold anyone to the result.

Sources

AI consultingAI governanceAI agentsDecision logBusiness operations