Parameter-Efficient Fine-Tuning (PEFT)
Learn how adapters make model adaptation cheaper and more practical.
The adapter idea
Parameter-efficient fine-tuning keeps the base model largely frozen and learns a smaller set of additional parameters. LoRA is a common method: it represents a weight update through low-rank matrices that are much smaller than the original layer weights. Other methods tune prompts, selected layers, or quantized models. The practical result is a compact adapter artifact that can be loaded with a compatible base model. PEFT changes the cost profile, but it still needs good data, evaluation, licensing review, and production controls.
Why memory falls
Full fine-tuning stores gradients and optimizer state for many base parameters, which can require substantial GPU memory and checkpoint storage. PEFT reduces the trainable parameter count, so the optimizer and gradients are smaller. Quantization-aware approaches can reduce the base model footprint further. These savings make experiments feasible on fewer or smaller GPUs, but activations, sequence length, batch size, and optimizer choices still consume memory. Measure peak usage rather than estimating from parameter count alone.
from peft import LoraConfig
config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.05,
target_modules=["q_proj", "v_proj"],
task_type="CAUSAL_LM",
)Choose adapter settings
Adapter rank, target modules, scaling, dropout, learning rate, and sequence length influence capacity and stability. A very small adapter may underfit a complex transformation, while a large adapter can raise cost and overfit. Start with documented defaults for the model family, then change one important variable at a time. Confirm that target module names actually match the architecture; a configuration that silently attaches to nothing can produce a convincing but useless training log.
Quantized adapter training
Quantized training can keep frozen base weights in lower precision while adapter parameters train at a higher precision. This can make a large model fit on limited hardware, but compatibility depends on the model, kernel support, hardware, and training library. Watch for numerical instability, unsupported operations, and quality loss. Keep the exact quantization configuration in the experiment record. Always compare the resulting adapter against an unquantized or stronger baseline on the same evaluation set.
Adapter composition
Multiple adapters can represent different domains, languages, tasks, or customers, but combining them is not automatically additive. Their assumptions may conflict, loading order can matter, and inference memory or latency can increase. Define ownership and compatibility rules before allowing runtime composition. If adapters are merged into the base model, preserve the original artifacts and test that the merged output matches expectations. A modular adapter strategy is useful only when versioning and selection remain understandable.
Serving and packaging
A deployable PEFT model includes the exact base-model revision, tokenizer, adapter weights, configuration, runtime version, prompt template, and license information. Decide whether to load adapters per request, keep a warm pool, or merge them for simpler serving. Measure cold-start time, memory, concurrency, and quality under realistic workloads. Reject a mismatched base model rather than attempting a best-effort load. Artifact metadata should make it possible to reproduce the model a user actually received.
Evaluate the trade-offs
Compare PEFT with full fine-tuning and prompt-only approaches on target quality, generalization, safety, latency, storage, memory, and operational complexity. Test hard negatives, domain shifts, longer inputs, and adapter switching if the product supports it. Look for catastrophic forgetting, overconfident errors, and behavior that appears only under certain phrasing. PEFT is a cost-efficient training technique, not a quality guarantee. Keep a base-model fallback and a rollback path when deploying a new adapter.