AI agent developer questions

Core concepts and architecture

1. What is an AI agent, and how is it different from a chatbot?
2. Explain the difference between an LLM and an agent framework
3. What is the agent loop, and what are its core steps?
4. What is function/tool calling and how does the model decide which tool to call?
5. Explain the difference between JSON tool-calling and code-execution agents (e.g. smolagents' CodeAgent)
6. What is a handoff, and when would you use one over a single agent with more tools?
7. What is the Model Context Protocol (MCP), and what problem does it solve?
8. How does the Agent2Agent (A2A) protocol differ from MCP?
9. What is RunContext / dependency injection in an agent framework, and why does it matter?
10. Explain structured output and why it matters for agent reliability

Memory, RAG & data

1. What's the difference between short-term and long-term agent memory?
2. Explain a basic RAG pipeline end to end
3. What is chunking, and why does chunking strategy affect retrieval quality?
4. What is hybrid search, and when would you combine it with vector similarity?
5. When would you reach for pgvector instead of a dedicated vector database?
6. Explain prompt caching and why it matters for cost at scale

Execution, safety & production

1. Why does an agent that executes code need a sandbox?
2. What's the difference between a sandbox with persistent storage and one without?
3. What is operational misalignment, and how is it different from a normal bug?
4. What's the minimum viable audit trail for a production agent?
5. Explain the difference between agent-native and LLM-first observability tools
6. What are evals, and how are they different from unit tests?
7. How would you estimate and control the token cost of an agent running in production?
8. What's the difference between a gateway/router and a direct model provider integration?
9. Name three ways to reduce an agent's dependency on calling an LLM for every step

What Is It?

This page collects interview-style questions on agent architecture and production concerns.

Focus on tradeoffs and failure modes, not acronym dumps.

Practice sketching loops: model, tools, memory, observability.

Why Should I Care?

Agent roles blend backend, ML ops, and security — interviews test breadth.

Strong narratives beat listing frameworks without context.

Real World Analogy

Agent interviews are oral exams in systems design — they want diagrams, not a tour of every framework logo you memorized.

Visual Mental Model

architecture | memory | RAG | safety | prod

Beginner Explanation

Prepare stories on agent loops, memory, RAG, tool auth, sandboxing, evals, and cost — tie answers to incidents or projects you shipped.

Smallest Possible Example

# be ready to whiteboard: loop, memory, tools, safety
print("architecture > buzzwords")

Common Mistakes

❌ Saying 'we just used LangChain' without your design choices.

Interviewers probe why — prepare reasoning.

❌ Ignoring safety when describing tool access.

Red flag for enterprise teams.

❌ No numbers (latency, cost, success rate).

Production stories need metrics.

Real Project Example

OpenAI, Anthropic, and SaaS agent startups ask system design with tools and evals.

Enterprise IT screens for auditability and data residency.

Build This Mini Project

Pick five questions here, answer in writing with a diagram each, and rehearse aloud under two minutes per answer.

Interview Question

Walk me through a production agent architecture.

Start with user channel, runner service, model gateway, tool layer (MCP/API), memory/RAG stores, sandbox for code, observability/evals, and deployment/cost controls — then deep-dive the piece most relevant to the role.

Key Takeaways

• Practice architecture sketches.
• Include safety and cost by default.
• Use metrics from real or toy projects.
• Explain tradeoffs, not logos.

Resources

• Our AI Agent post
• Smallest AI Agent walkthrough
• Python AI SDKs comparison

YouTube learning

Core concepts from video walkthroughs:

• Agent architecture Q&A style workshop
• Tool calling interview-style fundamentals
• LangGraph concepts for system design questions
• MCP & agent protocol questions explained

The 60-second story

An interview about agents is a shift lead asking if you have actually run the floor, not if you can define the floor. Start with the employee, not the model. A chatbot answers. An agent answers, uses tools, and checks the result before it talks again. The framework is the shift procedure around that employee. The model is the employee’s judgment. If you blur those, every later answer wobbles.

The loop is the procedure: read the situation, decide, call a tool or stop, read the tool’s sticky note, repeat. Function calling is the employee requesting a calculator. Your code runs the calculator. Code-execution agents, like smolagents’ CodeAgent, hand the employee a pen and a small Python pad instead of one slip at a time. Faster for loops. More dangerous without a room you can throw away.

A handoff is sending the guest to a specialist when one employee with forty tools becomes a mess. MCP is the standard outlet for tools. A2A is mail between employees who do not share a codebase. RunContext is the backpack of real dependencies so tools do not grab globals. Structured output is the form the answer must fit so you are not parsing charm.

Then the shift lead walks into memory. The tray is this turn. The notebook is next week. RAG is putting company pages on the desk before the question. Sandboxes are hotel rooms. Traces are photos on a string. A green metric with a bad refund is misalignment. You log who asked and which tool ran.

Map the oral exam.

• Chatbot versus agent = talk versus talk plus tools plus a check.
• Model versus framework = judgment versus the procedure.
• The loop = decide, tool, read, repeat.
• MCP versus A2A = outlet versus coworker mail.
• Tray versus notebook = this turn versus next session.
• Eval versus audit = practice exam versus the notebook of what actually happened.

while not done:
    action = model(tray)
    if action.tool:
        tray.add(run(action.tool))
    else:
        return action.final

If you can sketch that loop and then say where a guardrail can break out of it, you can answer half the architecture questions. The other half is storage, safety, and money: what is on the tray, what is in the notebook, what the meter says, what you refuse to let the tool do.

These questions exist because the failure mode is a confident employee with a real side effect. Definitions are the door. The shift is the answer.

Mental model: a floor shift you can draw. Employee, procedure, tray, tools, notebook, meter. If a question does not land on one of those, ask which one they meant before you improvise.

Practice the loop sketch until it is boring. Boring and correct beats a poetic definition of “agentic.”