Architecture & implementation
What a Production AI Agent Deployment Actually Requires
A production AI agent needs a bounded workflow, maintained sources, enforced permissions, behavioral checks, and an owner after launch. These are the five implementation decisions I would resolve before giving a model access to a customer-facing business process.
1. Specify the workflow and its stopping conditions
Start with one customer task. Write down the inputs it needs, the outputs it should produce, and the points where it must stop for a person. For multiple model roles, specify what each role receives, what it returns, and who validates the handoff.
My Support Triage Starter uses three issue types and a one-follow-up allowance. If required information is still missing after that follow-up, the code returns a simulated human escalation. The caller must pass back the previous follow-up state; the prototype does not enforce that limit through a server-owned session.
That distinction changes the production plan. A persistent session store, authentication, and protection against replay or altered state would be part of implementing the intended boundary.
2. Decide which sources can support an answer
Identify the authority for each kind of fact. Service descriptions might come from approved copy; availability might come from a scheduler; account status might require an authenticated system. Those sources have different owners and freshness requirements.
Record what happens when a source is missing, stale, contradictory, or unavailable. The application needs a supported fallback, such as asking a clarifying question or handing the request to a person. Fluent wording cannot substitute for an unavailable fact.
Access also needs a boundary. Specify which records a user may retrieve, what may be sent to a model, and what can appear in an answer or log. A knowledge base is one possible implementation; the source contract matters regardless of retrieval technique.
3. Keep business actions behind application checks
A model can propose an action. Application code should validate the request, authenticated identity, required fields, permissions, and destination before executing it. The approval process should reflect the consequence of an error.
For a booking workflow, distinguish displaying a verified scheduling link, requesting an appointment, and creating a confirmed appointment. Each requires a different integration contract. When an API cannot perform the requested action, the experience needs to say what happens next.
In the triage reference, a suspected duplicate charge opens a mock review ticket; it does not issue a refund. The code’s “resolved” status describes completion of that triage step. Downstream consumers need to understand that meaning before acting on it.
4. Check behavior with representative and adversarial inputs
Build a versioned set of requests with expected outcomes: acceptable answers, relevant sources, required escalation, and forbidden actions. Include ambiguous wording, missing information, source failures, conflicting records, and attempts to cross an access boundary.
Tests and evaluations overlap. A unit test can verify an escalation decision. A scenario evaluation can assess whether a model chooses the correct service or stays within approved information. Measure the relevant stages separately so a good answer score does not hide a routing or action failure.
A small failure in my own triage project illustrates why input variation matters. “My order hasn’t arrived” originally missed the classifier because it used a curly apostrophe. I normalized typographic apostrophes and added a regression test covering straight and curly forms, including the follow-up and escalation path. That test verifies a specific behavior; it does not establish broad language coverage.
Before a model, prompt, source, or routing change ships, rerun the scenarios it could affect. Record the cases, results, and review decision so the next person can inspect what was actually checked.
5. Assign operational ownership before launch
Name the people who maintain source information, receive escalations, review failures, and approve changes. Confirm they have the accounts, documentation, and access needed to do the work.
Define a way to disable actions or route all requests to a person when a dependency fails. Choose monitoring that exposes the workflow: request volume, latency, errors, tool failures, and escalation outcomes, with appropriate handling of customer data.
These are deployment requirements I use to plan and review work. The linked triage project remains a reference implementation with mock integrations. Its source code demonstrates several design choices; it does not establish production operating results.