AI Glossary

Looking for a deeper explanation instead of a one-sentence definition? Most of these terms link out to a full lesson in AI Topics.

Agent

An AI system that can autonomously plan a sequence of steps and call tools to accomplish a goal, rather than just answering a single prompt.

Attention

The mechanism inside a transformer that lets the model weigh how relevant every other token is when processing a given token, regardless of distance between them.

Backpropagation

The algorithm used to compute how much each weight in a neural network contributed to its error, so the weights can be updated during training.

Batch size

The number of training examples processed together before the model's weights are updated once during training.

Bias (statistics)

Systematic error in a model's predictions caused by an oversimplified model or unrepresentative training data, as opposed to random noise.

Chunking

Splitting a document into smaller pieces before indexing it for retrieval, so a search system can return focused, relevant passages instead of whole documents.

Context window

The maximum amount of text (measured in tokens) a language model can consider at once, including the prompt, conversation history, and any retrieved documents.

Cross-entropy loss

A loss function commonly used to train classification models, which heavily penalizes confident predictions that turn out to be wrong.

Embedding

A numeric vector representation of text, images, or other data, positioned so that similar inputs have similar vectors — the foundation of semantic search.

Epoch

One complete pass of a model through the entire training dataset.

Feature

An individual measurable input variable used by a machine learning model, such as a customer's age, a word count, or a pixel value.

Fine-tuning

Further training a pretrained model on a smaller, task-specific dataset to adjust its behavior, as opposed to training a model from scratch.

Grounding

Constraining a model's response to be based on specific, verifiable evidence (such as retrieved documents) rather than relying only on its internal training.

Hallucination

A confident but false or unsupported statement produced by a language model, which can sound plausible despite not being true.

Inference

The process of running a trained model on new input to produce a prediction or output, as opposed to training the model.

Label

The known, correct answer attached to a training example in supervised learning, such as "spam" or "not spam."

Latency

The time delay between sending a request to a model or system and receiving its response.

LLM (Large Language Model)

A neural network, typically based on the transformer architecture, trained on vast amounts of text to predict and generate language.

LoRA (Low-Rank Adaptation)

A parameter-efficient fine-tuning technique that trains a small number of additional weights instead of updating an entire model, reducing cost and memory use.

MCP (Model Context Protocol)

An open protocol that standardizes how AI applications connect to external tools, data sources, and services.

Multimodal

A model or system capable of processing and generating more than one type of data, such as text, images, and audio together.

Overfitting

When a model learns the noise and specific quirks of its training data instead of the general pattern, resulting in strong training performance but poor performance on new data.

Parameter

A learned numeric value (typically a weight or bias) inside a model; the parameter count is often used as a rough proxy for a model's size and capacity.

Precision

Of everything a model predicted as positive, the fraction that was actually positive — a measure of how trustworthy a positive prediction is.

Prompt engineering

The practice of designing and refining the instructions given to a language model to reliably produce a desired kind of output.

Prompt injection

An attack where untrusted input (such as a document or web page) contains hidden instructions designed to manipulate a language model into ignoring its original instructions.

Quantization

Reducing the numeric precision used to store a model's weights (for example, from 16-bit to 4-bit numbers) to shrink its memory footprint, usually with some accuracy trade-off.

RAG (Retrieval-Augmented Generation)

An architecture that retrieves relevant documents from a knowledge base and supplies them to a language model as context before it generates a response.

Recall

Of everything that was actually positive, the fraction a model correctly identified — a measure of how much a model misses.

Reranking

A second-stage process that re-scores and reorders a set of candidate search results using a more precise (and usually more expensive) relevance model.

RLHF (Reinforcement Learning from Human Feedback)

A training technique that uses human preference judgments to fine-tune a model's behavior, commonly used to make language models more helpful and safe.

Temperature

A setting that controls the randomness of a language model's output; low temperature produces more predictable, deterministic text, while high temperature produces more varied and creative text.

Token

A chunk of text (often a word, part of a word, or punctuation mark) that a language model processes as a single unit.

Training data

The dataset used to teach a model patterns, whether through supervised labels, unsupervised structure, or reinforcement signals.

Transformer

The neural network architecture, built around self-attention, that underlies most modern large language models.

Vector database

A database optimized for storing embeddings and performing fast similarity search over them, commonly used as the retrieval layer in RAG systems.

Zero-shot / few-shot

Zero-shot means asking a model to perform a task with no examples in the prompt; few-shot means providing a small number of examples to guide its response.