Topic 1: Generative AI Fundamentals interview questions

Generative AI refers to models that learn patterns from existing data and produce new content such as text, images, audio, code, or structured outputs. Large language models are one example: they estimate likely token sequences from context. The model does not retrieve a guaranteed fact database by default; it generates an answer based on learned statistical representations, which is why grounding, validation, and careful evaluation are important.

A language model converts text into tokens, processes them through neural network layers, and produces a probability distribution for the next token. A decoding strategy selects one token, appends it to the context, and repeats the process. Temperature, top-p, stop sequences, and token limits influence selection. The result is generated sequentially, not composed from a hidden draft of verified statements.

Tokens are the units a model reads and generates; they may be words, parts of words, punctuation, or symbols. Tokenization affects context-window usage, latency, and cost. A long document can exceed the model limit even when its character count looks reasonable. It also explains why unusual words, code, and some languages may consume more tokens and require different chunking or budgeting strategies.

The context window is the maximum number of input and output tokens a model can handle in one request. It includes system instructions, conversation history, retrieved passages, tool results, and the generated response. A larger window is useful but does not guarantee better reasoning. Applications should reserve output space, remove irrelevant history, summarize safely, and prioritize high-value context instead of filling the limit.

Training learns model parameters from a large dataset and is computationally expensive. Inference uses an already-trained model to produce outputs for a request. Fine-tuning continues training on a smaller, task-specific dataset to change behavior or style. Fine-tuning does not automatically add a reliable, current knowledge source, so retrieval or tool use is usually better for changing facts and private documents.

Temperature changes how sharply the model favors high-probability tokens: lower values usually make outputs more predictable, while higher values increase variation. Top-p limits sampling to the smallest group of tokens whose cumulative probability reaches a chosen threshold. They are decoding controls, not intelligence controls. For deterministic extraction, use low randomness and constrained output; for ideation, allow more diversity and evaluate results.

A hallucination is a plausible-looking output that is unsupported, incorrect, or fabricated. It happens because a generative model is optimized to produce likely continuations, not to prove every claim. Risk rises with ambiguous questions, missing context, and requests for obscure facts. Mitigations include authoritative retrieval, citations, tool verification, structured constraints, abstention instructions, and tests that measure factual accuracy rather than fluency alone.

I would state the task, provide relevant context, define the desired audience and constraints, specify the output format, and include examples when the pattern is difficult to infer. I would separate instructions from untrusted input and ask the model to acknowledge uncertainty instead of guessing. Then I would test representative and adversarial cases, because a prompt that works on one example is not production evidence.

Prompt injection occurs when untrusted content attempts to override the application’s instructions, such as a web page telling an agent to reveal secrets or call an unsafe tool. Treat retrieved text, files, and user input as data, not authority. Use least-privilege tools, explicit confirmation for consequential actions, output validation, secret isolation, and monitoring. No prompt alone should be considered a complete security boundary.

Use structured output when downstream code needs dependable fields, such as classifications, extraction results, tool arguments, or UI data. Define a schema with types, required fields, and allowed values, then validate the response before using it. Schema-constrained generation reduces formatting errors but does not guarantee factual correctness. Business rules, authorization checks, and semantic validation must still run outside the model.

An embedding is a numeric vector representing the semantic features of content, commonly used for similarity search, clustering, or classification. Generated text is a sequence of tokens selected by a generative model. Embeddings help find relevant material; they do not themselves answer a question. A typical application combines embedding search with a language model that reads selected content and produces a response.

A model is the learned component that transforms inputs into predictions or generated outputs. An application surrounds it with product logic: authentication, data access, prompts, retrieval, tools, validation, user experience, logging, and fallback behavior. Production quality depends on the whole system. Changing the model may improve one metric, but weak permissions, poor context, or missing monitoring can still make the application unsafe.

I start with a representative evaluation set containing normal, boundary, and adversarial cases, with explicit criteria for correctness, relevance, groundedness, safety, format, and helpfulness. I combine automated checks with expert or human review, track regression results across model and prompt changes, and measure production outcomes such as task completion and escalation rates. Evaluation should use real failure costs, not only a single aggregate score.

Offline evaluation runs a fixed dataset before release, making regressions reproducible and comparisons efficient. Online evaluation observes the feature with real traffic, through monitoring, user feedback, experiments, or shadow requests. Offline tests may miss changing user behavior and integration failures; online signals can be noisy and risk exposing users. A mature process uses offline gates first, then controlled online validation with rollback capability.

I would measure where time is spent across queueing, retrieval, tool calls, model time-to-first-token, and output generation. Then I would reduce unnecessary context, select an appropriately small model, parallelize independent calls, cache stable results, stream responses, and use shorter outputs where acceptable. I would preserve quality with evaluation tests and set explicit latency budgets, because an optimization that harms task success is not a real improvement.

First, record token usage and cost by feature, tenant, request type, and model. Set budgets, quotas, timeouts, and maximum output sizes. Route simple tasks to smaller models, cache reusable work, trim redundant context, batch asynchronous jobs, and avoid repeated retries without limits. Review quality alongside cost: the cheapest response is not valuable if it causes support work, incorrect decisions, or repeated user attempts.

A fallback should be designed around the user’s goal, not merely another model call. Use bounded retries for transient failures, then try a compatible model or cached result when appropriate. If confidence, validation, or service health is insufficient, provide a clear partial result, a manual path, or a retry option without losing user input. Log the failure category and avoid exposing internal prompts, keys, or provider details.

Classify data before sending it to a model, minimize the fields included, and redact secrets or identifiers where possible. Enforce access control before retrieval, encrypt data in transit and at rest, and understand the provider’s retention and training policies. Limit logs and redact prompts and outputs. Add tenant isolation, retention controls, audit trails, and tests that verify one user cannot receive another user’s private context.

Fine-tuning is useful when a stable task needs consistent style, classification behavior, or formatting and a representative, high-quality dataset is available. Prompting and retrieval are usually preferable for rapidly changing requirements or factual knowledge. Fine-tuning adds data preparation, training, evaluation, versioning, and hosting complexity. I would first establish a measurable baseline, confirm the problem is behavioral, and compare the expected quality gain with that operational cost.

I would include authentication and authorization, input limits, timeouts, bounded retries, rate limiting, cost controls, schema validation, content and safety checks, observability, and a user-safe fallback. Requests should carry correlation identifiers, while logs protect sensitive data. Models and prompts need versioning, offline regression tests, controlled rollout, and rollback. For tools, validate arguments and require explicit approval before irreversible or externally visible actions.