Classification vs. Regression
Learn how to tell the two core supervised-learning problem types apart and pick algorithms that match the shape of your output.
Start from the output, not the algorithm
The fastest way to choose the right approach is to describe the prediction in one sentence and look at the noun at the end. "Predict which category" is classification. "Predict a number" is regression. Teams that skip this step often force a regression model to predict a category by rounding, or force a classifier to predict a continuous value by bucketing it — both work poorly compared to using the family designed for the job.
Binary, multiclass, and multilabel classification
Binary classification chooses between two outcomes, like fraud or not fraud. Multiclass classification chooses exactly one label from several, like which of five product categories an item belongs to. Multilabel classification allows more than one label per example at once, like tagging a support ticket as both "billing" and "urgent". Each variant needs a different output layer and loss function, so misclassifying which one you have leads to a model architecture that cannot represent the real problem.
from sklearn.linear_model import LogisticRegression, LinearRegression
# Regression: predict the exact price
reg = LinearRegression().fit(X_train, prices_train)
print("Predicted price:", reg.predict(X_test[:1]))
# Classification: predict "expensive" vs "affordable"
is_expensive_train = [p > 300000 for p in prices_train]
clf = LogisticRegression().fit(X_train, is_expensive_train)
print("Is expensive:", clf.predict(X_test[:1]))Common regression pitfalls
Regression models are sensitive to outliers, because a single extreme value can dominate the error the model is minimizing. They also assume, in their simplest forms, a fairly smooth relationship between inputs and outputs — a sudden jump (like a price cliff at a subscription tier boundary) can confuse a plain linear model. Log-transforming skewed targets, capping outliers deliberately, and choosing tree-based models for non-smooth relationships are all standard fixes.
Loss functions shape behavior
A classifier is typically trained to minimize cross-entropy loss, which penalizes confident wrong answers heavily. A regressor is typically trained to minimize mean squared error, which penalizes large errors disproportionately more than small ones. Swapping the loss function changes what the model considers a "good" prediction, so the loss function is a design decision, not an implementation detail — mean absolute error, for example, is far less sensitive to occasional large misses than mean squared error.
When the line blurs
Some problems sit between the two families. Predicting a star rating from 1 to 5 could be treated as regression (a continuous approximation) or as ordinal classification (five ordered categories). Predicting a probability of churn is technically a regression output between 0 and 1, but it is usually produced by a classifier and thresholded into a decision. There is rarely one objectively correct choice — pick the framing that matches how the prediction will actually be used downstream.
Practical exercise
Take five prediction ideas from a domain you know — will a customer renew, how many units will sell next week, is this transaction fraudulent, what genre is this song, how long will a delivery take — and classify each as classification or regression before writing any code. For one you are unsure about, write one sentence explaining how the prediction will be used, since that usually resolves the ambiguity.
Sources and further reading
These primary or specialist references informed the concepts in this guide. Product details can change, so verify current documentation before implementation.