Fine-Tuning Models interview questions

Fine-tuning continues training a pretrained model on a task-specific dataset so its behavior, format, or domain performance changes. I would use it when the desired behavior must be consistent across many requests, such as a fixed extraction schema, writing style, or classification task. I would not use it simply to store frequently changing facts; retrieval-augmented generation is usually better for that because knowledge can be updated without retraining.

Prompting changes the instructions and context supplied at inference time, while fine-tuning changes model parameters through additional training. Prompting is faster to iterate, cheaper initially, and useful for testing whether a task is viable. Fine-tuning can produce more consistent behavior, shorter prompts, and better adherence to a format, but it requires curated data, training infrastructure, evaluation, and version management. I would prototype with prompts before committing to fine-tuning.

Fine-tuning teaches a model patterns, behavior, or task mappings; RAG supplies relevant external information at request time. For example, I might fine-tune a model to classify support tickets consistently, then use RAG to retrieve the current product policy. Fine-tuning is a poor fit for rapidly changing facts because updates require another training cycle. RAG also provides source documents that can be inspected, although retrieval quality must be evaluated carefully.

Supervised fine-tuning trains a pretrained model on labeled input-output examples. Each example shows the behavior I want, such as a user request paired with a concise JSON response. The training objective generally increases the probability of the target output tokens. SFT works best when examples are accurate, representative, consistently formatted, and aligned with the real task. It does not automatically teach facts or guarantee safe behavior outside the training distribution.

A good dataset contains realistic inputs, high-quality target outputs, consistent conventions, and enough coverage of normal, edge, ambiguous, and refusal cases. I would remove duplicates, conflicting labels, leaked evaluation examples, and unnecessary personal data. The examples should represent the production distribution rather than only easy cases. I would also record provenance, labeling guidelines, dataset versions, and known limitations so later failures can be traced to data decisions.

There is no universal number because the amount depends on task complexity, model size, desired behavior, and example quality. A few hundred clean examples can reveal whether a narrow formatting task is learnable, while broader behavior may need thousands or more. I would start with a small pilot, hold out validation data, and plot performance against dataset size. More examples are useful only when they add representative coverage rather than repeated noise.

I first define the task contract and canonical output format, then convert examples into the model provider’s supported conversation or instruction schema. I validate required fields, roles, token lengths, encoding, and serialization before training. I normalize equivalent answers where appropriate, remove accidental prompt leakage, and separate training, validation, and test sets by user or source when correlation could inflate results. A small manual review of every sample is valuable for high-impact tasks.

Catastrophic forgetting is the loss of capabilities learned during pretraining or earlier tuning when a model is trained too aggressively on a narrow dataset. It can show up as weaker general reasoning, language performance, instruction following, or safety behavior. I would detect it with a broad regression suite, not only task accuracy. Mitigations include lower learning rates, fewer epochs, diverse examples, regularization, parameter-efficient methods, and retaining a strong base-model fallback.

Overfitting occurs when the model memorizes training examples or narrow patterns instead of learning behavior that generalizes. Training loss may continue improving while validation quality worsens, especially with a small or repetitive dataset. I would monitor held-out metrics, inspect generated outputs, and test paraphrases and unseen cases. Early stopping, deduplication, fewer epochs, lower capacity, better data diversity, and regularization can reduce overfitting. Exact memorization of sensitive examples is also a privacy risk.

LoRA, or Low-Rank Adaptation, freezes the base model and trains small low-rank update matrices in selected layers. This greatly reduces trainable parameters and storage while often preserving useful quality. QLoRA combines this idea with a quantized base model so training uses less memory. Both are parameter-efficient fine-tuning methods. I would still evaluate the adapter against full fine-tuning and the base model, because lower cost does not guarantee the same task performance.

PEFT adapts only a small portion of a model rather than updating all parameters. Common approaches include LoRA adapters, prompt tuning, prefix tuning, and selective layer training. The advantages are lower GPU memory use, faster experiments, smaller artifacts, and the ability to keep multiple task-specific adapters over one base model. The trade-offs are possible limits on the behavior that can be learned and additional complexity in adapter compatibility, serving, and evaluation.

I pay particular attention to learning rate, number of epochs, effective batch size, sequence length, warmup, weight decay, and adapter rank when using LoRA. The right values depend on dataset size, model, and task. I would begin from documented defaults, run controlled experiments, and compare validation quality rather than tuning everything at once. Training loss, gradient behavior, throughput, memory use, and output samples together reveal whether a run is learning productively.

I evaluate against the base model and a prompt-only baseline using a held-out, representative test set. Automated checks can measure exact match, classification F1, schema validity, citation presence, latency, and cost. For generative work, a rubric and calibrated human review assess correctness, completeness, style, and refusal behavior. I also test adversarial, ambiguous, multilingual, and long inputs. A model should meet must-pass safety and format checks before aggregate quality is considered.

The validation set guides choices such as epochs, learning rate, and checkpoint selection, so repeated tuning can gradually overfit it. The test set should remain untouched until the final comparison and release decision. I would create splits that avoid near-duplicate examples and, where possible, separate users, documents, or time periods. This gives a more honest estimate of how the fine-tuned model will perform on future production inputs.

I would start with LoRA or another PEFT method because it is cheaper, faster, and easier to roll back. Full fine-tuning becomes reasonable when the task needs broad parameter changes, sufficient compute is available, and controlled experiments show adapters are inadequate. The decision should use measured quality, memory, training time, artifact size, serving complexity, and maintenance cost. I would also consider whether multiple adapters must share one base model in production.

Fine-tuning can make a model reproduce information present in training examples, but it is not a reliable knowledge database. Facts may be memorized incompletely, become stale, or be recalled inconsistently. For current, traceable information I would use retrieval with access control and citations. If domain terminology or stable patterns must be learned, fine-tuning may help, but I would test factual accuracy and avoid claiming that training guarantees complete knowledge.

I classify and minimize data before training, remove secrets and unnecessary personal information, and confirm that the provider’s data-use and retention terms match the project. Access is restricted, artifacts are encrypted, and logs avoid raw prompts and targets where possible. I search for evaluation leakage and test whether the model reproduces sensitive strings. For private deployments, I document ownership, retention, deletion, and incident procedures for datasets, checkpoints, and adapters.

I first measure the class distribution and the business cost of each error. I would add representative examples for rare classes, review ambiguous labels, and use stratified validation so minority performance is visible. Depending on the training framework, class weighting, sampling, or carefully generated data may help, but synthetic examples must be checked for quality. I report per-class precision, recall, and confusion patterns instead of relying only on overall accuracy.

I version the base model, dataset, training configuration, adapter or checkpoint, tokenizer, evaluation results, and serving image together. I deploy behind a feature flag or canary, compare quality and operational metrics with the current version, and retain a fast rollback path. Monitoring should cover latency, errors, token usage, refusals, schema failures, user corrections, and safety incidents. I would restrict rollout if a regression appears in a high-impact segment, even when average scores improve.

Common failures include unclear task definitions, inconsistent labels, too little representative data, evaluation leakage, overfitting, and using fine-tuning where retrieval or ordinary application logic is more suitable. Operational issues include excessive sequence lengths, unsupported formats, GPU memory limits, incompatible adapters, and missing rollback plans. I would diagnose by comparing a baseline, auditing samples, inspecting training curves, and testing the complete inference path. Fine-tuning should solve a measured gap, not be used because it sounds advanced.