AI agent developer questions

Interview-style questions on agent architecture, memory, RAG, safety, and production.

Core concepts and architecture

  1. What is an AI agent, and how is it different from a chatbot?
  2. Explain the difference between an LLM and an agent framework
  3. What is the agent loop, and what are its core steps?
  4. What is function/tool calling and how does the model decide which tool to call?
  5. Explain the difference between JSON tool-calling and code-execution agents (e.g. smolagents' CodeAgent)
  6. What is a handoff, and when would you use one over a single agent with more tools?
  7. What is the Model Context Protocol (MCP), and what problem does it solve?
  8. How does the Agent2Agent (A2A) protocol differ from MCP?
  9. What is RunContext / dependency injection in an agent framework, and why does it matter?
  10. Explain structured output and why it matters for agent reliability

Memory, RAG & data

  1. What's the difference between short-term and long-term agent memory?
  2. Explain a basic RAG pipeline end to end
  3. What is chunking, and why does chunking strategy affect retrieval quality?
  4. What is hybrid search, and when would you combine it with vector similarity?
  5. When would you reach for pgvector instead of a dedicated vector database?
  6. Explain prompt caching and why it matters for cost at scale

Execution, safety & production

  1. Why does an agent that executes code need a sandbox?
  2. What's the difference between a sandbox with persistent storage and one without?
  3. What is operational misalignment, and how is it different from a normal bug?
  4. What's the minimum viable audit trail for a production agent?
  5. Explain the difference between agent-native and LLM-first observability tools
  6. What are evals, and how are they different from unit tests?
  7. How would you estimate and control the token cost of an agent running in production?
  8. What's the difference between a gateway/router and a direct model provider integration?
  9. Name three ways to reduce an agent's dependency on calling an LLM for every step

Why it matters

Agent roles mix backend, ML ops and security, so interviews test breadth. A clear story about one design choice beats a list of framework names.

Where people get stuck

  • Saying "we just used LangChain" with no reasoning. Interviewers will ask why.
  • Describing tool access without mentioning safety. That's a red flag for enterprise teams.
  • No numbers. Production stories need latency, cost and success rate.

Try this

Pick five questions from this page, answer each in writing with a diagram and practise saying it in under two minutes.

Interview question

Walk me through a production agent architecture.

User channel, runner service, model gateway, tool layer (MCP or APIs), memory and RAG stores, a sandbox for code, observability and evals, then deployment and cost controls. Go deep on the piece closest to the role.

Resources

YouTube learning

Core concepts from video walkthroughs:

Threads in the same domain as this chapter — go deeper on iHateReading without leaving the roadmap.

AI Agent cover

AI Agent

What agents are and how they differ from a single chat completion.

Agent harnessing: the part of building AI agents nobody talks about cover

Agent harnessing: the part of building AI agents nobody talks about

Loop, tools, memory, sandbox, and tracing live in your code — not the model weights.

Python AI agent SDKs in 2026: OpenAI vs Pydantic-AI vs smolagents vs Strands cover

Python AI agent SDKs in 2026: OpenAI vs Pydantic-AI vs smolagents vs Strands

Same three-tool agent in four frameworks — pick by execution model and testing needs.

Smallest AI Agent cover

Smallest AI Agent

Minimal file-reading agent to learn LLM tool loops without framework magic.