Supervised Learning Explained
Understand how a model learns a mapping from inputs to known outputs, and why labeled data is the foundation of most production ML.
What supervised learning actually does
Supervised learning starts with a dataset where every example already has the answer attached: a house has a known sale price, an email is already marked spam or not spam, a scan is already diagnosed. The algorithm searches for a function that turns the input features into that known output as accurately as possible, then reuses that function on new, unlabeled inputs. Everything downstream — accuracy, bias, and usefulness — traces back to how representative and correctly labeled that original dataset was.
Features and labels
A feature is any measurable input the model can use — square footage, word frequency, pixel values, transaction amount. A label is the answer you want predicted. Choosing features is not a formality: a feature that leaks the answer (like including the final grade when predicting whether a student passed) produces a model that looks accurate in testing and fails in production. Good feature selection asks what information will realistically be available at prediction time, not just what correlates well in a spreadsheet.
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
hours_studied = [[1], [2], [3], [4], [5], [6]]
test_scores = [50, 55, 65, 70, 78, 85]
X_train, X_test, y_train, y_test = train_test_split(
hours_studied, test_scores, test_size=0.33, random_state=0
)
model = LinearRegression()
model.fit(X_train, y_train)
print("Predicted score for 4.5 hours:", model.predict([[4.5]]))
print("R^2 on held-out data:", model.score(X_test, y_test))The train/test split
A model that is only ever evaluated on the data it trained on will always look better than it is, because it can partly memorize that data. Standard practice is to split data into a training set the model learns from and a held-out test set it never sees until evaluation. A common variant, k-fold cross-validation, repeats this split several times over different partitions and averages the results, which gives a more stable estimate when the dataset is small.
Classification vs. regression at a glance
Supervised problems fall into two broad families. Classification predicts a category — spam or not spam, which of ten digits, which risk tier. Regression predicts a continuous number — a price, a temperature, a wait time. The same underlying workflow (features, labels, split, train, evaluate) applies to both, but the algorithms, loss functions, and evaluation metrics differ, which is why picking the right family before picking an algorithm matters.
A minimal worked example
Predicting a test score from hours studied is a simple regression problem: the model looks for a line (or curve) through the (hours, score) points that minimizes the average error. With enough examples, it can predict a plausible score for a number of hours it has never seen, because it has generalized the pattern rather than memorized specific points.
Practical exercise
Take a dataset you understand — even something informal like your own weekly running distance versus how tired you felt out of 10 — and manually split it 80/20. Fit a straight line to the 80% by eye or with a spreadsheet trendline, then check how far off it is on the 20% you held back. This makes the train/test gap concrete before you ever touch a machine-learning library.
Sources and further reading
These primary or specialist references informed the concepts in this guide. Product details can change, so verify current documentation before implementation.