Deployment & Cost Management

Core concepts

1. Containerizing an agent: Docker is still the default packaging format — one Dockerfile, a few secrets, deploy anywhere. Docker Docs
2. Hosting options: Fly.io, Render, and Railway for general-purpose agent deployments; Modal or Vercel Sandbox specifically when the agent needs execution isolation.
3. Scheduling & triggers: CRON schedulers and tools like trigger.dev for agents that run on an interval rather than on-demand.
4. Token cost tracking: Log input/output tokens and model used per run — the only way to catch a runaway agent before the bill does.
5. Minimizing AI dependency: The best-run agent stacks only invoke a model when a task genuinely needs judgment — deterministic code handles everything else, which is both cheaper and more reliable.

What Is It?

Deployment covers hosting agent loops (containers, serverless, workers), secrets, and scaling.

Cost tracking ties model usage to business outcomes.

Patterns include queue workers, cron agents, and human-in-the-loop approvals.

Why Should I Care?

Agents left always-on burn budgets — architecture choices matter.

SLOs for latency differ from batch report generators.

Real World Analogy

Deploying an agent is opening a food truck — the recipe works in your kitchen, but now you need health permits, cash registers, and a line that does not block traffic.

Visual Mental Model

container/worker -> scheduler -> metrics/cost
  minimize LLM calls per task

Beginner Explanation

Containerize runners, schedule jobs, track token spend, and design flows that do not call the model when a deterministic function suffices.

Smallest Possible Example

COST = {"tokens_in": 1200, "tokens_out": 400}
print("track $ per successful task")

Common Mistakes

❌ No autoscaling limits on tool-heavy workers.

Viral traffic duplicates expensive loops.

❌ Running migrations and agents in same process without isolation.

Long agent tasks starve HTTP health checks.

❌ Ignoring idempotent deployments for in-flight tasks.

Rolling deploys duplicate side effects.

Real Project Example

Fly.io and Railway host small agent workers; Kubernetes for larger fleets.

Vercel cron plus AI SDK for lightweight scheduled agents.

Build This Mini Project

Dockerize a minimal agent CLI, add env-based model selection, and log token counts per run to a CSV.

Interview Question

How do you control agent token costs in prod?

Cache prompts, use smaller models for easy steps, cap loops, batch work, trim context, monitor cost per successful task, and replace LLM steps with code when possible.

Key Takeaways

• Treat agents like other services — containers, secrets, scale.
• Measure cost per successful outcome.
• Prefer code over LLM when deterministic.
• Design idempotent tool effects.

Resources

• Docker Docs
• Fly.io Docs
• trigger.dev Docs
• Render Docs · Railway Docs
• Our own AI agent cost breakdown

YouTube learning

Core concepts from video walkthroughs:

• Deploying coding agents (containers & sandboxes)
• Shipping agents on Vercel with AI SDK
• Gateway observability for cost tracking in prod

The 60-second story

An agent on your laptop is a food truck at your house. Deployment is the permit, the commissary kitchen, and the meter. Docker is the crate: the same recipe, the same jars, a machine that is not your laptop’s mystery shelf. One Dockerfile and a few secrets will get you surprisingly far. The secrets do not go in the crate. They go in the kitchen’s lockbox.

Fly, Render, and Railway are commissaries for a normal long-running service. Modal or a sandbox product is the kitchen you use when the agent must execute code in a room you can throw away. Do not pick a sandbox host just to serve a chat UI. Do not pick a plain web dyno and then subprocess the model’s code on it. Match the kitchen to the mess.

Scheduling is for agents who work when nobody is at the counter. Cron is the diner opening at 6 whether you feel like it. trigger.dev and its cousins are cron with retries and a UI so “it did not run” is a row, not a rumor.

The meter is token cost. Log model, input tokens, output tokens, per run, per user. A runaway loop is a cheerful employee who never clocks out. You want the meter to scream before the card does. The cheapest control is not a smaller model first. It is not calling a model when a rule will do. Deterministic code for tax, thresholds, and routing. The model for judgment. Teams that send every if to a model are paying a novelist to do arithmetic.

Map the permit.

• Docker is the crate.
• A normal host is the commissary.
• A sandbox host is the room you can discard.
• Cron is the diner that opens itself.
• Token logs are the meter.
• A rule in code is a job you do not hand to the novelist.

if status in {"paid", "shipped"}:
    return status
return model.decide(status)

That if is money and reliability. The model should see the weird cases, not the enum you already trust.

Deployment exists because a laptop process dies when the lid closes, and because secrets and dependencies drift. Cost tracking exists because agent loops hide multiplication: steps times tokens times retries. The engineering virtue is calling the model less often, on purpose.

Mental model: a crate, a kitchen, and a meter. Ship the crate. Pick a kitchen that matches whether code must run loose. Watch the meter. Let boring code stay boring.

You have the page if you can say where secrets live, how a scheduled run becomes visible when it fails, and why a token log is not optional once a loop exists.