Every agent needs a model behind it. This chapter covers both the direct providers and the gateway layer that lets you avoid hardcoding your whole stack to one vendor's pricing and uptime.
Start with direct providers when you know your primary model (OpenAI, Claude, Gemini, or a specific open-weight family). Add a gateway when you need one integration surface for many models, centralized billing, or fallbacks — common once you ship beyond a single-provider demo.
What Is It?
Every agent needs a model provider behind the loop.
Direct APIs give first-party features; gateways unify routing, caching, and keys.
Child pages compare vendors and gateway features.
Why Should I Care?
Token spend and rate limits dominate agent unit economics.
Vendor outages happen — abstraction layers save sleep.
Real World Analogy
Direct providers are calling the chef; gateways are food delivery apps that let you reorder from many restaurants with one account.
Visual Mental Model
app -> gateway -> OpenAI / Anthropic / local
app -> provider API directBeginner Explanation
This hub splits direct providers (OpenAI, Anthropic, Google, open weights) and gateways (OpenRouter, LiteLLM, Portkey). You need both maps for cost and failover.
Smallest Possible Example
MODEL = "gpt-4.1-mini" # abstract name; swap via gatewayCommon Mistakes
❌ Hardcoding one model string everywhere.
Use config/env so you can swap models without refactors.
❌ Ignoring rate limits on tool-heavy loops.
Each step may be another completion — budget tokens.
❌ Storing provider keys in frontend bundles.
Keys belong on server-side runners only.
Real Project Example
OpenRouter and LiteLLM appear in startups mixing cheap and premium models.
Helicone and Portkey add observability on top of gateways.
Build This Mini Project
Move model name and base URL to environment variables; document fallback model in README.
Interview Question
Why use an LLM gateway?
Unified API across providers, centralized keys, routing/fallbacks, caching, and spend controls — reduces lock-in and simplifies ops.
Key Takeaways
- Hub covers direct + gateway paths.
- Configure models externally.
- Plan for rate limits and cost.
- Never expose keys client-side.
YouTube learning
Core concepts from video walkthroughs:
