Feature Engineering Basics
Learn the practical techniques for turning raw data into inputs a model can actually learn from.
Why features matter more than algorithms
A well-chosen feature set often improves model performance more than switching to a fancier algorithm. Raw data is rarely in a form a model can use directly — timestamps, free text, and categorical labels all need transformation before a numeric model can learn from them. Feature engineering is where domain knowledge about the problem gets encoded into a form the math can use.
Encoding categorical variables
Models generally expect numbers, not text categories. One-hot encoding turns a category like "color" into separate binary columns ("is_red", "is_blue", "is_green") so the model does not wrongly assume an ordering between categories. Ordinal encoding assigns a single number and is appropriate only when the categories have a genuine order, like "small, medium, large" — using it on unordered categories like city names silently introduces a false ranking that misleads the model.
import pandas as pd
from sklearn.preprocessing import OneHotEncoder, StandardScaler
df = pd.DataFrame({
"signup_date": pd.to_datetime(["2026-01-05", "2026-03-14"]),
"plan": ["pro", "free"],
"monthly_spend": [49.0, None],
})
df["signup_month"] = df["signup_date"].dt.month
df["monthly_spend_missing"] = df["monthly_spend"].isna().astype(int)
df["monthly_spend"] = df["monthly_spend"].fillna(df["monthly_spend"].median())
encoded_plan = OneHotEncoder(sparse_output=False).fit_transform(df[["plan"]])
scaled_spend = StandardScaler().fit_transform(df[["monthly_spend"]])
print(df, encoded_plan, scaled_spend)Scaling and normalization
Many algorithms, especially those based on distance (like k-nearest neighbors) or gradient descent (like neural networks), are sensitive to the scale of input features. A feature ranging from 0 to 1,000,000 can dominate one ranging from 0 to 1 even if it is less predictive. Standardization (subtracting the mean, dividing by standard deviation) and min-max scaling (rescaling to a fixed range) are the two most common fixes, and tree-based models are largely unaffected by scale.
Handling missing data
Missing values are common in real datasets and most algorithms cannot handle them directly. Simple imputation fills gaps with the mean, median, or mode of the column; more sophisticated approaches predict the missing value from other features. It is also worth adding a binary "was this missing" indicator column, since the fact that a value is missing can itself be predictive — for example, a missing income field might correlate with a specific customer segment.
Creating new features from existing ones
Some of the most useful features are derived rather than raw: extracting day-of-week from a timestamp, computing a ratio between two existing columns, or counting the number of words in a text field. These derived features can expose patterns that the raw columns hide from a simple model, especially a linear one that cannot combine features on its own the way a tree-based model or neural network can.
Practical exercise
Take any tabular dataset you can find, even a small CSV of your own expenses or workouts. List every raw column, then write down at least one derived feature you could create from it (a ratio, a date part, a binned category, a missing-value flag). Implement two of them and check, informally, whether they visually separate your target variable better than the raw column did.
Sources and further reading
These primary or specialist references informed the concepts in this guide. Product details can change, so verify current documentation before implementation.