The map is not the architecture: autonomy, authority, and what agentic AI frameworks do not show

It is a well-made material, and the footer brings the most important phrase of the design: start low in the stack and only add a layer when the problem justifies it. I fully agree. The problem is that the diagram itself contradicts this phrase at several points, and those who use this type of map as an architecture checklist end up building exactly the system that the phrase tries to avoid.

In this article, I provide a critical reading of this mental model and propose another one, built around a question that the map does not ask: how much of the decision am I delegating to the model, and how much of the authority remains with the system?

The risk of maps that become shopping lists

Architecture infographics have a legitimate function: to provide common vocabulary for teams that are just starting out. The risk arises when the map ceases to be vocabulary and becomes a requirement. I have seen more than once a team look at a diagram like this and conclude that they need RAG, MCP, sub-agents, long-term memory, and reflection, because all of this is drawn there, side by side, with the same visual weight.

A map does not say what is mandatory, what is optional, and what is costly. It does not indicate what fails first. And it does not say where what the model proposes ends and what the system guarantees begins. This absence is what transforms good teaching material into a source of overengineering.

Three conceptual errors in the layered model

1. The containment is inverted

In the diagram, AI is the smaller circle at the center, and Agentic Systems is the outer circle that encompasses all the others. Visually, this communicates that agentic systems contain artificial intelligence.

In classical taxonomy, the relationship is the opposite. AI is the superset, Machine Learning is a subset of AI, and Deep Learning is a subset of Machine Learning. The author intended to represent a construction stack and drew a containment relationship. For those who are just starting, it seems like a detail. For those who will make architectural decisions, it is the first sign that the design has been optimized for aesthetics, not for precision.

2. The stack is a narrative of the LLM era, not a taxonomy

The sequence of six layers suggests that each step is a prerequisite for the next. This does not hold. Generative AI existed before foundation models: GANs and VAEs generated images years before the large pre-trained models. And agents exist without generative AI: classical planners and reinforcement learning agents make goal-oriented decisions without generating a line of text.

The collateral effect of this narrative is positioning the agent as the evolutionary top, the place where every system should arrive. In practice, a good part of the business problems I solve with AI ends before this point, and ends better.

3. The categories mix different dimensions

The last point deserves attention. Tool calling is the model's ability to issue a structured call to a function. MCP is a protocol for exposing tools and context to a client. In the architecture I use, the MCP interface is part of the trust boundary: it is where the capabilities are exposed to the model. But the protocol alone is not this boundary. Authentication, authorization, credential scope, tenant isolation, sandboxing, and auditing remain the responsibility of the system. A poorly configured MCP server is an attack surface, not control. When the map treats tool calling and MCP as synonyms, it hides precisely this decision.

Autonomy is not capability

Before proposing an alternative, I need to name the underlying confusion that the diagram carries, which appears in many discussions about agents: treating capability and autonomy as the same thing.

Capability is what the system can do: understand natural language, retrieve private knowledge, call an API, read a screen. Autonomy is who decides what happens next. A system can have LLM, RAG, and dozens of tools and not be agentic if all transitions between steps are hardcoded. Having tools does not make a system agentic.

The term agentic is still used with different definitions, sometimes emphasizing goal-oriented behavior, sometimes planning, sometimes iterative execution with feedback from the environment. I do not intend to resolve this discussion. For architectural purposes, I use an operational definition: I consider a system agentic if part of the transition policy between states is delegated to the model.

This distinction is more precise than saying that "in the agent, the model controls the flow." Even in mature agentic systems, the model does not control the system. It selects actions within an execution envelope defined by the runtime. The formulation I use is this: in the workflow, the transition policy between states is explicitly coded; in an agentic system, part of this policy is delegated to the model.

This change alters the nature of the engineering problem. The runtime remains deterministic, but part of the policy that chooses the next transition becomes probabilistic. The consequences are concrete: the state space grows, the cardinality of paths can remain open, the cost per execution becomes variable, reproducing a behavior becomes more difficult, and I start needing execution budgets, semantic tracing, and explicit authorization policies.

Reasoning is not authority

The second distinction missing in the map is between reasoning freedom and operational authority. A model can have high freedom to interpret, plan, and propose, without having the authority to execute.

The model may conclude that it needs to consult process X. Who decides if that user can consult process X is the system. The model proposes the next action. The system continues deciding whether it has permission to be executed.

This defines what I call the decision boundary. The model can participate in the decision: interpret, classify, generate, propose, select the next candidate action. But the system's invariants remain outside of it:

• access control and tenant isolation;
• financial limits and compliance rules;
• schema and domain invariants;
• transaction commits and irreversible side effects.

LLMs can participate in the decision, but invariants need to remain outside the model. This is the correct reading of the separation between deterministic layer and probabilistic layer. It is not a question of where AI exists. It is a question of who holds the final authority.

The layers that the map does not draw

With these two distinctions in hand, it becomes clear what the diagram leaves out.

Data. There is no layer for ingestion, quality, lineage, or governance of the vector index. RAG appears as an application pattern, as if retrieval were a prompt problem. The quality of a RAG starts long before the model receives the context: ingestion, segmentation, metadata, access control, retrieval, reranking, and index updating are part of the problem.

Execution envelope. The map does not treat cost or limits as architectural dimensions. Instead of thinking of iteration ceilings, timeouts, and token budgets as loose mechanisms, I treat them all as a single envelope: token, time, tool call, iteration, and monetary budgets. The engineering question shifts from whether the agent finished to whether it finished within the expected operational envelope. An agent without explicit limits on iteration, time, and consumption turns a decision failure into a potentially unlimited operational failure.

Semantic observability. "Observability & tracing" appears on the map, but in probabilistic systems, trace cannot only log calls and latency. It needs to preserve the retrieved context, the version of the prompt and model, each tool call with arguments and results, the applied policy decision, the triggered fallback, the cost, and the evaluation result. Without this, I can see that the system made a mistake, but not why.

Evaluation as architecture. "Evaluation & monitoring" appears as production control, as if it were post-QA. In AI, eval is part of the engineering system. No increase in autonomy should go into production because it looks better in a demo. The next level needs to surpass the previous one in a set of evaluations representative of the real workload.

Reversibility. No block addresses the ability to revert. If I introduce an agent and quality drops, I need to be able to revert to the workflow, turn off a specific tool, change the model, operate in degraded mode, and re-execute an execution from the trace. Architecture that cannot be reverted is not a decision; it is a gamble.

An architectural escalation heuristic

Instead of the stack, I use an escalation heuristic. I make a point of making clear what it is and what it is not: these levels do not form a taxonomy or a containment relationship. Each level represents the introduction of a new relevant source of operational complexity. The point is not to reach the top. It is to stop at the lowest level that solves the problem with the required reliability.

To avoid repeating the mistake I criticized in the diagram, I separate two dimensions that are often mixed.

The first is capability: what the system can do at each step. It ranges from deterministic logic, through LLM calls with structured output, to grounding (RAG) and tool usage.

The second is orchestration: who decides the next transition. It ranges from single calls, through workflows with fixed graphs, to agents, and finally to multi-agents.

The two combine freely. A workflow without any LLM is legitimate. An agent may not use RAG. A multi-agent system may contain deterministic workflows. The escalation that truly requires justification is that of orchestration, because it is the one that delegates decision-making to the model.

For each step in the orchestration dimension, I ask three questions: what justifies escalation, what starts to cost, and what starts to fail.

The change in nature is between workflow and agent. Up to the workflow, the transition policy is code. From the agent onward, part of it is delegated. It is the step that requires the most evidence, and that is why it should only happen when the problem truly has uncertainty about the path. Autonomy needs to be proportional to the uncertainty that actually exists in the problem.

Real case: where Kazo.ai stopped scaling

Kazo.ai is a legal AI platform for Brazilian law firms. The MVP centers around a WhatsApp assistant that serves the firm's clients.

Looking at the map, it would be natural to design this assistant as an autonomous