How to Deploy AI Agents to Production

The demo works fine on your laptop. Then you ship it, and by Tuesday a tool call that started looping at 3am has been burning tokens for hours before anyone notices.

This guide is the list of things to set up so that doesn't happen: where to host the agent, where it keeps its memory, how to test it before each release, how to cap what it can spend, and how to find the bad run when something goes wrong anyway.

What changes when an agent leaves your laptop

On your laptop the agent has one user, you. Nobody is promised a response time, nothing caps the bill, and nothing records what happened. In production you need all three. You need to know when it broke, how often, for which user, what it cost, and whether a new version actually beats the old one before everyone gets it.

The way it fails changes too. A normal web app fails loudly with an error page. An agent fails quietly. It gives a confident wrong answer, calls the same tool forever, or gets a little worse after a model update.

Uptime monitoring won't catch any of that. Tests and run records will.

The seven steps below go in the order that works. Skip one and it usually costs you within the first month of real traffic. After them, Monitor and debug after launch covers what to watch and how to trace a failure once the agent is live.

Step 1: Write down your targets before you pick tools

Before you touch any infrastructure, write down four numbers:

• How long a slow request may take. People measure this as p95, the time that 95 out of 100 requests finish within.
• What share of requests should succeed.
• The most one request may cost.
• The most one user may cost per day.

Every later choice follows from these. An analyst agent that's allowed 30 seconds can run on a completely different setup than a support agent that has to answer in 200 milliseconds. Without targets, you'll overbuild the easy parts and underbuild the risky ones.

A reasonable starting point for an internal tool: p95 under 8 seconds, success above 95 percent, 5 cents per request, 5 dollars per user per day. A tool customers use will need different numbers. Whatever you pick, put it in a doc, share it with whoever owns the budget, and look at it again after a week of real traffic.

Step 2: Pick hosting that fits how long your agent runs

Agents aren't normal web apps. You can't predict how long a run will take, they hold onto information between tool calls, and some need a GPU.

That makes time limits the first thing to check. Put a research agent that usually finishes in four minutes behind a function with a five-minute limit, which is Vercel's maximum on the Hobby plan as of September 2026, and it works right up until one run takes six.

Match your agent to one of three patterns:

• Quick request and answer, under about 60 seconds. Vercel Functions, Cloudflare Workers, or AWS Lambda. Cheap, cost nothing when idle, and fine for chat agents that make one or two tool calls.
• Long runs or GPU work, 1 to 30 minutes. Modal or Railway. Modal lets you deploy a Python function by adding a decorator to it, which makes it the closest thing to "Vercel for backends" right now, and it handles GPUs and long timeouts out of the box.
• Background jobs, anything over 30 minutes or anything that has to survive a restart. Put the work on a queue and have workers pick it up. Options include BullMQ on Redis, AWS SQS plus Lambda, or Temporal or Restate, which remember how far a job got so it can pick up where it stopped after a crash.

If you don't want to make this choice at all, a managed agent platform will make it for you. You pay more per request and it's harder to leave later. As of September 2026 the main options are LangSmith Deployment, which was LangGraph Platform until an October 2025 rename, Microsoft Foundry hosted agents, and Google's Agent Runtime, formerly Vertex AI Agent Engine.

Step 3: Use three kinds of storage, not one

One agent run needs three different kinds of storage, and no single database does all three well.

• Scratch space for the current conversation. Partial results and tool output that the agent needs back in milliseconds. Redis is the usual choice.
• The permanent record. Conversation history, user settings, and audit logs that must never get lost or half-written. Postgres is the usual choice.
• Search by meaning. Past conversations and document chunks the agent finds by similarity rather than by exact match. pgvector, an add-on for Postgres, or a dedicated vector database.

A common way to wire them together: the current turn lives in Redis with an expiry time. A background worker copies finished turns into Postgres and turns the parts worth remembering into embeddings, numeric fingerprints of meaning, stored in pgvector. When a new turn starts, the agent reads from Redis if the session is still fresh, falls back to Postgres if it's gone stale, and searches pgvector for related past context.

Two things to watch. If Redis only saves periodic snapshots, it can lose everything written since the last snapshot if it stops uncleanly, so never treat it as the permanent record. And don't put secrets or personal data into a vector index. Store an ID there instead, and look up the real content in Postgres when you need it.

You can skip the search layer entirely if the agent doesn't recall old conversations or search documents. Postgres plus Redis is enough for many agents. When you do need it, pgvector inside the Postgres you already run is the simplest start. Move to a dedicated vector database only when speed or size forces you to.

Step 4: Guard every way in

Your agent might be reached through an API, a webhook from another service, or a queue. Every one of those doors needs a lock, a speed limit, and a check on what comes through.

Agents cost money every time they run. A webhook with no authentication that starts an agent run is an open tap: anyone who finds the address can run up your bill.

A solid baseline:

• Check who's calling, on every entry point. An API key for server-to-server calls, a signed login token for user sessions, and a signature check for webhooks, which proves the request really came from the service it claims.
• Limit requests per user at the edge, before they reach your code. Upstash Redis, Cloudflare, or your API gateway can do this.
• Check the shape of every request with Zod or Pydantic. Reject a malformed request before it ever reaches a model call.
• Make repeated requests safe. Picture a request that creates a record. The agent creates it, then loses its connection before it records that it succeeded. It tries again, and now there are two. Give each request a unique key so the second attempt is recognized as work already done. That's called an idempotency key, and any endpoint that changes data needs one.

Wrap the agent in a thin handler that checks the caller, validates the input, creates a trace ID, starts the run, and returns. Keep the agent's own logic out of that handler.

Step 5: Record every run before real users arrive

When your agent misbehaves in production, you need to see exactly what that run did. Without a record, you're reading customer screenshots and guessing. With one, you open the run and see the bad tool argument that started the loop.

That record is called a trace. One user request is one trace. Inside it, every model call, tool call, and search step gets its own entry, called a span, and each span points to the one that started it. That parent-and-child tree matters because one request can set off several model and tool calls in a row. Without it, you end up rebuilding what caused what from timestamps.

Every span needs at least:

• A name (for example, agent.run, llm.completion, tool.call:search_crm)
• Start time, end time, and duration
• Parent span ID, so you can rebuild the tree
• What went in (prompt, tool arguments) and what came out (model reply, tool result)
• Token counts, model name, and provider
• Cost in dollars, if you can work it out as you go
• Status: success, error, or truncated

Tag the top-level span with session_id, user_id, and agent_version. When a customer reports a bug, you want to find their exact runs in seconds.

If you're on LangChain, LangGraph, or the OpenAI Agents SDK, setting one environment variable gets you most of this. If you're building your own, use the OpenTelemetry GenAI semantic conventions. They're a shared set of field names, like gen_ai.request.model and gen_ai.usage.input_tokens, so any tracing tool can read your data. They moved to their own semantic-conventions-genai repository in 2026 and are still marked Development, not stable, but they're the closest thing to a vendor-neutral format for model traces.

Then pick one place to send traces and make it the team's single source of truth. Starting prices below are from each vendor's pricing page as of September 27, 2026.

Older versions of this guide listed Helicone. It was acquired by Mintlify in March 2026 and is in maintenance mode, so don't start a new build on it. Datadog now sells what it used to call LLM Observability as Agent Observability.

If you only pick one, Langfuse is a reasonable default. The core is open source, it accepts OpenTelemetry traces, and running it yourself costs only the servers.

Step 6: Test every release and block the ones that get worse

Agents rarely break with an error. They get worse. You change a prompt to fix one bug and quietly break three other behaviors. Without tests, you find out from a customer complaint two weeks later.

The fix is a test set, usually called an eval suite: 30 to 100 real user inputs, each with the outcome you expect. Run the agent against it on every change, and refuse to ship a version that scores worse.

The course covers how to build that suite, so this step only covers where it fits in a deploy. Evaluate agent performance and reliability runs a dataset and keeps failures in the score instead of quietly dropping them. Prompt contracts with evaluation fixtures shows how to write output rules a program can check. Agent evaluation tools compares five tools on the same task.

For deployment, two things matter. First, run the suite automatically on every change and fail the build when scores drop. Braintrust, LangSmith, and Langfuse all support this, and a GitHub Action that runs the suite and posts results to the pull request is enough to start. Second, feed production failures back in. Every failed run worth fixing becomes a new test case, which is the replay loop under Monitor and debug after launch.

Step 7: Roll out slowly, with spending limits

Bad answers aren't the only thing to plan for. A run that loops, a third-party API that goes down, or a prompt injection (hidden text that talks the agent into calling a paid tool over and over) all cost money without showing a visible error.

Start by shipping the new version next to the old one. Send 5 to 10 percent of traffic to it, watch it for at least an hour, and only then send more. That small first slice is called a canary.

Then add these protections:

• Retry carefully. When an outside API fails, wait a little, then longer each time, with a small random delay so every client doesn't retry at once. Stop after 3 tries. Don't retry 4xx errors, which mean the request itself was wrong, except 408 (timeout) and 429 (too many requests).
• Stop calling a tool that keeps failing. After a set number of failures in a row, stop calling that tool for a cool-off period and return an error right away. This is called a circuit breaker. The agent gets the error and can work around it or stop cleanly.
• Cap the steps per run. Pick a ceiling, such as 30 steps, and kill any run that passes it. Log it and alert on it.
• Cap spending per user per day. Before each model call, check a counter in Redis. When a user hits the cap, return a friendly error and tell them.
• Compare the canary to the old version. Look at success rate, response time, and cost per request. Send more traffic only if all three hold or improve.

With those in place, the worst incident is a contained one. Without them, the worst case is a five-figure bill.

Hosting comparison: where to actually deploy

Limits and prices are from each vendor's docs as of September 27, 2026. Use this as a starting point, then check it against your own targets from step 1.

Starting fresh with no strong preference? Modal for long-running work, Vercel for the chat screen users see, and Langfuse for traces is a setup with few moving parts to learn.

Monitor and debug after launch

Most agent failures don't throw an error. The request comes back fine, the agent confidently did the wrong thing, and nobody notices until a customer asks why it booked the same meeting four times.

The traces from step 5 are what let you find those failures. This section covers what to watch, what to alert on, and how to turn a bad run into a fix.

Watch a few numbers every day

Put these on a dashboard someone actually opens:

1. Task success rate. Did the agent finish the job? Score it with an automated judge or by having people review a sample.
2. Tool call success rate. The share of tool calls that come back without an error. A drop usually means an API changed its format or got flaky.
3. Steps per run. Model and tool calls per run. A sudden jump means the agent is looping.
4. Tokens per run. Watch the slow tail (p95), not just the average. The outliers eat the budget.
5. Time per run at p50, p95, and p99. The slowest runs are the ones users remember.
6. Cost per finished task. Total spend divided by successful runs. It's the one number that tells you whether the agent pays for itself.
7. Invalid tool calls. Calls that fail because the model made up a parameter or a tool that doesn't exist.

You don't need all seven on day one. Start with task success, steps per run, and cost per finished task. Those three catch most problems.

Alert on the failures you can't see

Four alerts cover the common quiet failures:

• Loop alert: any run hits the step cap from step 7. One is a bug report. Several in an hour is an incident.
• Tool error spike: the tool error rate jumps to three times its usual level over the past hour. Usually an outside API changed its format or started limiting you.
• Slowdown: p95 time climbs more than 50 percent week over week. Often a longer prompt or a slower model version.
• Spending spike: daily spend passes 1.5 times the average of the last seven days. It catches loops and prompt injections that try to drain the account.

Send them to Slack or PagerDuty, not email, and put the trace ID of the bad run in the alert so whoever is on call can click straight into it. These thresholds are starting points. Adjust them after a couple of weeks of real traffic.

Make a trace fast to open

How fast you can open a bad run decides how fast fixes ship. If it takes three logins and a JSON viewer, nobody looks until something's on fire.

Anyone on the team should be able to open a run in seconds and see, on one screen, the input that started it, every tool call with its arguments and response, tokens and cost per step, and any errors. A "rerun this" button is worth having.

Turn failed runs into test cases

This is the replay loop:

1. Spot a failed run in production.
2. Copy its inputs, including the user message, conversation history, and tool outputs, into your test set.
3. Write down what success looks like: an exact answer, a text pattern, a judge's rubric, or a check on the final tool call.
4. Add the case to the test suite from step 6.
5. When you change the prompt, swap models, or update a tool, run the full suite and compare scores.
6. Block the deploy if scores drop.

Without it, every prompt change is a coin flip. With it, each production failure becomes a test that keeps that bug from coming back. The evaluation lesson covers how to score the suite without hiding failures.

Fix the pattern, not the one bad run

Every two weeks, look at the three most common kinds of failure in your traces and fix what causes them. Four show up in most agents:

Loops. The agent calls the same tool with the same arguments over and over because the answer it gets back is unclear. Fix: keep the step cap from step 7, notice when two tool calls in a row are identical, and have the agent summarize and stop after the second repeat.

Wrong arguments. The right tool, called wrong: text where a number goes, a value that isn't one of the allowed options, a date out of range. Fix: check every tool input against a strict schema, and send back error messages that tell the agent how to fix the call instead of just "invalid input."

Made-up tools and arguments. The agent calls a tool or parameter that doesn't exist. Fix: check every call against a list of allowed tools before running it, and send back an error that lists the real tools and what they accept, so the agent can correct itself on the next step.

Drift in long conversations. The agent gets worse as the conversation grows and the original instructions get buried under dozens of turns. Fix: track how many tokens go in at each step, and past a limit, summarize the older turns.

Each of these is easier to spot in a tracing tool than in raw logs, which is why step 5 comes before real traffic.

What the finished setup looks like

Here's one complete version. Vercel or Modal runs the agent. Postgres on Neon or Supabase holds the permanent record. Upstash Redis holds the current conversation and the rate-limit counters. Langfuse or LangSmith records every trace. Braintrust runs the test suite on every change. A canary on 10 percent of traffic guards every deploy. Per-user spending caps and circuit breakers run on every request.

Price it from the tables above before you commit. At low volume most of those services sit on free or entry plans, which is cheap insurance against the mistake that costs four figures overnight.

Next, build the guardrails this setup assumes. Start with human approval for risky actions, then the evaluation lesson for the test suite in step 6.

Related Guides

• Human Approval for Agents: Store and Check the Exact Decision
• Evaluate a Ticket Agent: Dataset, Scoring Code and Failure Report
• Reactive vs Proactive AI Agents: Architecture Comparison