Creating an AI agent is easy. Putting it into production is another story.
In recent months, I have spoken with dozens of teams that have built some type of IA agent. The vast majority reach the same point: the prototype works well in a demo, impresses the product team, and even gets an initial deploy. And then, in production, it starts to break in ways that no one anticipated.
This is not a lack of technical competence. It is the natural consequence of treating an agent as if it were just "a well-written prompt running in a loop." This approach has a low ceiling, and that ceiling appears quickly when the environment is real.
What really happens when an agent leaves the lab
When an agent starts operating in production, with real users, real data, and real volume, a series of problems emerge that simply do not appear in controlled tests:
Loss of context. Long conversations or tasks with multiple steps cause the agent to forget previously made decisions, repeat questions that have already been answered, or contradict actions it has executed itself.
Infinite loops. Without clear stopping mechanisms, an agent that relies on itself to decide the next step can enter cycles of trial and error that never converge, and that the user may not even notice until the bill arrives.
Inconsistent state. This is the problem we underestimate the most when moving out of the prototype. An agent can reason correctly and still execute the same action twice, lose the recovery point after a failure, or assume that a step was completed when it was not. Conversational memory and execution state are different problems. The first deals with "what was said." The second deals with "what was done, at what stage, and what can still be safely redone." Treating both as the same thing is one of the most common causes of unpredictable behavior in production.
High token consumption. Each additional call, each retry, each context resent due to lack of structured memory has a cost. At scale, this cost ceases to be marginal and becomes a relevant line item in the engineering budget.
Unchecked responses. An agent that executes actions (calling an API, writing to a database, generating a document) without any layer of validation is, in practice, placing the reliability of the system in the hands of a probabilistic component. This works until the day it doesn't.
Increasing uncontrolled costs. The sum of all this is a system that becomes more expensive as it scales, exactly the opposite of what is expected from a mature software architecture.
Why Prompt Engineering alone doesn't solve this
Prompt Engineering was essential in the first wave, when the central problem was "how to make the model understand what I want." Prompt design, system prompts, structured outputs, tool use instructions, and behavior policies continue to be important components, and will remain so. But they have ceased to be sufficient.
The current problem is different: how to make a system based on probabilistic components operate with predictability, controlled cost, and production reliability. This is not solved by adjusting the wording of a prompt. It is solved with engineering, designing state, execution, observability, evaluation, and safety mechanisms around the model.
In production, the model should not be the system. It should be a probabilistic component within an architecture whose control layer establishes deterministic boundaries of state, authorization, execution, observability, and fault recovery. And the greater the impact of the decision, the less unrestricted autonomy this component should have.
What really requires a reliable agent in production
After bringing multiple agents to production in different contexts (legal, financial, recruitment), I arrived at a set of pillars that I consider non-negotiable:
- Architecture and autonomy limits
- State, memory, and durable execution
- Security, authorization, and access control
- Observability and SLOs
- Continuous trajectory evaluation, not just final response
- Cost control through execution budget
Architecture and autonomy limits
The starting point is the clear separation between the deterministic execution layer and the probabilistic decision layer of the model. The agent decides what, the system ensures how. Or, in higher-risk flows: the agent interprets, plans, and proposes; the system validates, authorizes, and controls execution.
Autonomy also needs to be an architectural decision, not a consequence of what the model is technically capable of doing. Not every tool available to an agent should be able to be executed freely. A mature architecture defines action limits, authorization scopes, budgets, and approval policies by risk level. Instead of asking only "can the agent execute this?", the correct question is "under what conditions is it authorized to execute this?".
In critical operations, such as financial transactions or legal decisions, the agent can interpret, plan, and recommend, while deterministic rules and authorization mechanisms continue to control execution.
State, memory, and durable execution
Memory is about conversational context, structured and with defined scope, not "throwing the entire history into context" with each call. State is about execution, and here comes a concept that I consider central in current agent engineering: durable execution.
Agentic workflows cannot rely on the expectation that the entire flow will finish in a single uninterrupted execution. A process can fail, a worker can restart, an external API can become unavailable, a credential can expire, and human approval can take hours. Therefore, agents in production need to persist state, create checkpoints, and be able to resume execution exactly from where they left off, without repeating side effects that have already occurred. Idempotency is one of the fundamental guarantees in this design, alongside deduplication, retries, and compensation. In practice, in many distributed systems, we work with at-least-once delivery and design idempotent handlers, deduplication, and side effect control to achieve effectively-once semantics at the operation level.
Security, authorization, and access control
When an agent gains tool use, access to databases, APIs, and corporate systems, security ceases to be a checklist item and becomes architecture. Each tool needs to operate with least privilege, specific scopes, parameter validation, and audit trails.
Prompt injection ceases to be just a content problem when the agent has execution capability. A malicious instruction embedded in a document, an email, or an API response can result in unauthorized access to data, unauthorized external calls, or state changes. Treating this as an attack surface, rather than a prompt detail, is what differentiates an enterprise-ready system from an exposed prototype.
Security also includes governance of the data that traverses the agent: what information enters the context, what can be persisted, what can leave the organization's perimeter, and who can retrieve it later.
Observability and SLOs
Observability answers one question: what happened? Latency, cost per interaction, task completion rate, tool error rate, fallback rate, and human intervention rate need to be instrumented from day one.
Some of these metrics appear in both observability and evaluation; the difference lies in what we do with them. Observability explains execution. Evaluations judge its quality.
Production also requires SLOs. It is not enough to know that the agent "works most of the time." We need to define what success means: task completion rate, p95 latency (and p99 when volume justifies), maximum cost per execution, acceptable fallback rate, and percentage of human interventions. Without this, reliability remains a perception, not a measurable property of the system.
Continuous trajectory evaluation
Evaluation answers a different question than observability: was the behavior good? And evaluating an agent does not just mean comparing the final response with an expected response.
In agentic systems, we need to evaluate the trajectory: whether the correct tool was chosen, whether the arguments sent were correct, whether there were unnecessary calls, whether policies were respected, and whether the final goal was truly achieved. It is worth separating two metrics that are often treated as one: tool selection accuracy, whether the right tool was chosen, and tool invocation success, whether that tool executed correctly. An agent can get the second right and the first wrong, and in that case, the call returns technical success on a semantically wrong decision. This involves metrics such as task completion rate, tool selection accuracy, tool invocation success, policy adherence, and, in retrieval scenarios, groundedness. Many failures do not appear in the final response. They appear in the path taken to reach it.
None of this is useful without versioning and traceability. Model, prompts, tool schemas, policies, retrieval configuration, and execution graph need to be associated with the execution they produced, including, in RAG scenarios, which documents and excerpts were used. Without this trail, reproducing a failure or explaining a regression becomes almost impossible.
Cost control through execution budget
Every agent should operate within a defined execution budget before running, not observed afterward. Maximum number of steps, retries, tokens, time, and cost should not be consequences discovered in production, but architectural limits from the flow design. An agent without a budget is, in practice, a probabilistic process with potentially unlimited access to computational resources, and it is exactly this design that produces infinite loops and out-of-control bills.
Containing failures: failure domains and human-in-the-loop
A mature architecture also seeks to contain failures, not just avoid them. A poor model response should not automatically turn into a poor operation in the system. A timeout of a tool should not necessarily take down the entire workflow. Models, tools, memory, retrieval, and external systems need to have clear failure domains, allowing for retry, fallback, controlled degradation, or escalation for human intervention. The goal is not to eliminate all possibility of failure, something impossible in distributed systems, but to limit the blast radius when it happens.
And human-in-the-loop is not necessarily a sign that the agent has failed. In high-risk processes, such as those I often encounter in financial, legal, and recruitment contexts, it can be a deliberate architectural decision. The system should know when to continue autonomously, when to request confirmation, and when to escalate the decision to a person.
What is intentionally left out
I did not place specific frameworks or platforms at the center of this article, whether agentic frameworks like LangGraph and CrewAI, durable execution engines like Temporal, or agent SDKs. They are different categories of tools, solving different problems. Frameworks evolve quickly and stack choices change. The principles remain: state, authorization, durable execution, observability, trajectory evaluation, and cost control are valid regardless of the chosen stack.
The curious thing is that much of this engineering did not originate with AI. Idempotency, observability, retries, circuit breakers, state machines, authorization, and fallback existed long before any LLM. What changed was that we placed a probabilistic component at the center of flows that were previously predominantly deterministic. The discipline of engineering remains the same. What changes is where and how it needs to be applied.
The difference between demo and production
In a demo, the goal is to show that the agent can do something. In production, the goal is to ensure that it does the right thing, within the right limits, at the expected cost, and that the system knows exactly what to do when that does not happen.
It is this difference that separates an interesting agent from a reliable system.
