Topic 3: AI Agents & Orchestration interview questions

An AI agent is a system that uses a model to interpret a goal, decide on actions, use tools or context, and evaluate results. Unlike a simple prompt-response application, it can follow a multi-step loop. A production agent still needs boundaries: explicit tools, authorization checks, time and cost limits, observable state, and a human escalation path for risky or ambiguous actions.

A workflow follows predefined steps and branching rules, so its behavior is predictable and easy to test. An agent chooses among possible actions at runtime based on the task and available context. Many reliable systems combine both: deterministic workflow code controls critical transitions, while an agent handles flexible tasks such as classification, planning, or selecting the most appropriate tool.

A typical agent has a model for reasoning, instructions defining its role and constraints, tools for external actions, memory or retrieved context, a state store, and an orchestration loop. It also needs policy enforcement, validation, retries, tracing, and evaluation. Keeping these concerns separate makes the system easier to change, test, secure, and operate than putting everything into one prompt.

I would start with a bounded loop: receive the task, load relevant state, ask the model for either a final response or a structured tool call, validate that call, execute it, append the result, and repeat. The loop ends on a valid answer, a maximum-step limit, timeout, budget exhaustion, or policy failure. Every iteration should emit trace data for debugging.

Planning is the process of decomposing a goal into actions or subgoals before execution. It can be explicit, such as producing a task list, or implicit, with the model deciding one action at a time. I prefer plans that are short, structured, and revisable. The executor should verify each step rather than blindly trusting a plan generated before new information appears.

Multiple agents are useful when responsibilities are genuinely different, such as research, coding, review, or domain-specific decision support, and when separate context or permissions improve quality. They add latency, cost, coordination complexity, and more failure modes. I would first establish that a single agent or deterministic workflow cannot meet the requirement, then define narrow roles and a clear handoff contract.

Common patterns include a sequential pipeline, a router that selects a specialist, parallel workers followed by aggregation, a supervisor coordinating specialists, and an evaluator-reviser loop. The right pattern depends on dependency structure and risk. I use sequential execution for dependent steps, parallelism for independent work, and explicit aggregation schemas so one worker cannot silently distort another worker’s output.

I enforce multiple independent limits: maximum iterations, wall-clock timeout, tool-call budget, token budget, and repeated-action detection. The agent should receive a structured termination condition and the orchestrator should stop it regardless of model output. On termination, the system should return a useful partial result or request human review, while recording the reason and intermediate state.

Tools should be narrow, typed, deterministic where possible, and documented with clear preconditions and failure behavior. Inputs need schema validation, authorization, and limits such as allowed resources or record counts. Destructive operations should require confirmation or a separate approval tool. Returning concise, structured results helps the model reason accurately and reduces context growth.

I classify failures as transient, permanent, validation, authorization, or business-state errors. Transient failures can use bounded exponential backoff with idempotency protection. Permanent and policy errors should be explained without repeated attempts. The tool response should expose a safe error code and recovery hint, while logs retain diagnostic details. The orchestrator should decide whether to retry, choose another tool, or escalate.

Agent memory is persisted information used across steps or conversations. Short-term memory is current task state; long-term memory may contain preferences, facts, or past outcomes. Risks include stale facts, accidental sensitive-data retention, prompt injection through stored content, and unbounded context. I use explicit schemas, provenance, retention rules, access controls, and retrieval filters instead of treating every past message as trusted memory.

I keep the working context bounded by summarizing completed work, retaining only relevant tool results, and storing detailed artifacts outside the prompt. Summaries should preserve decisions, unresolved questions, identifiers, and evidence links. Retrieval should be selective and permission-aware. I also monitor context size and model quality because aggressive compression can remove a fact needed for a later decision.

Reliability comes from system design, not from prompting alone. I constrain tools, validate outputs, use deterministic code for critical rules, add time and budget limits, and provide retries with idempotency. I test representative tasks and adversarial cases, instrument every step, define fallback behavior, and review failures. For high-impact actions, the agent proposes while a policy or human approves.

I evaluate both final outcomes and intermediate behavior. A test set should include normal, ambiguous, adversarial, and tool-failure tasks. Metrics can cover task success, factuality, tool-selection accuracy, policy violations, latency, cost, and escalation quality. Traces make failures diagnosable. Offline evaluations establish a baseline, while sampled production reviews and regression tests detect drift after prompts, tools, or models change.

Human-in-the-loop orchestration pauses an agent at defined decision points so a person can approve, edit, reject, or clarify an action. It is appropriate for financial, legal, security, or irreversible operations. The approval screen should show the proposed action, inputs, evidence, and consequences. The system must persist the pending state and safely resume or cancel without duplicating the action.

I treat retrieved documents, web pages, emails, and tool outputs as untrusted data, not instructions. System policy remains separate from content, and tools enforce authorization independently of the model. I restrict domains and actions, sanitize inputs where useful, require confirmation for sensitive operations, and test indirect injection scenarios. Logging suspicious instructions and offering a safe refusal path are also important.

I set per-request budgets, choose smaller models for routing and simple steps, cache stable results, limit retrieved context, and parallelize independent calls. I avoid unnecessary planning loops and make tool outputs compact. Traces should attribute tokens, model time, tool time, and retries to each task. If a budget is reached, the system should return a bounded result or escalate predictably.

Idempotency means repeating the same request produces the same intended result rather than duplicating an effect. Agents need it because models, networks, and orchestrators may retry after uncertain failures. I pass an operation or request key to side-effecting tools, persist execution status, and make the tool safely return the existing result on replay. This is essential for payments, emails, and record updates.

I persist a versioned task record containing the goal, current step, validated tool calls, results, status, timestamps, and retry metadata. Each transition should be atomic or recoverable so a process restart cannot lose the workflow position. Sensitive fields need encryption and access controls. State schemas should support migration because prompts, tools, and orchestration logic evolve over time.

An agent should refuse or escalate when the request is outside its authority, evidence is insufficient, a policy blocks the action, the user’s intent is materially ambiguous, or the impact is high and approval is required. The response should explain the limitation briefly and offer a safe next step. Silent guessing is worse than a bounded refusal because it creates untraceable risk.