How Neural Networks Work
Build an intuitive, then mechanical, understanding of neurons, layers, weights, and forward passes.
The single neuron
A single artificial neuron takes a set of numeric inputs, multiplies each by a learned weight, adds a learned bias, and passes the sum through a nonlinear activation function. This is close to a tiny logistic regression. One neuron alone can only represent a simple, roughly linear decision boundary — the power of neural networks comes from stacking many of them together.
Layers and depth
Neurons are organized into layers: an input layer that receives the raw features, one or more hidden layers that transform those features, and an output layer that produces the final prediction. Each neuron in a layer typically connects to every neuron in the next layer, which is why this design is called a fully connected or dense network. "Deep" learning simply means a network with enough hidden layers to learn increasingly abstract representations at each stage.
import numpy as np
def relu(x):
return np.maximum(0, x)
X = np.array([0.6, 0.2]) # two input features
W1 = np.array([[0.3, -0.1], [0.5, 0.8]]) # hidden layer weights
b1 = np.array([0.1, -0.2])
W2 = np.array([0.7, -0.4]) # output layer weights
b2 = 0.05
hidden = relu(X @ W1 + b1)
output = hidden @ W2 + b2
print("Hidden activations:", hidden)
print("Network output:", output)The forward pass
A forward pass is the process of pushing one input example through the network, layer by layer, to produce a prediction. At each layer, the previous layer's outputs become the current layer's inputs, get multiplied by that layer's weight matrix, summed with a bias vector, and passed through an activation function. The final layer's output is the network's prediction, which is then compared against the true label to compute a loss.
Why nonlinearity matters
If every layer only did a weighted sum with no activation function in between, stacking any number of layers would mathematically collapse into a single linear transformation — depth would add no representational power at all. Nonlinear activation functions are what let deep networks approximate complex, curved decision boundaries and represent things like "this pixel pattern is a cat" that no straight line could separate.
What weights actually represent
Weights are the numbers the network adjusts during training; they encode how strongly each input to a neuron influences its output. A trained network's "knowledge" is entirely contained in millions or billions of these weight values — there is no separate rulebook. This is why neural networks are often called black boxes: the pattern is real and learned, but it is distributed across weights in a way that resists simple human explanation.
Practical exercise
By hand or in a spreadsheet, build a network with two inputs, one hidden neuron, and one output neuron. Pick arbitrary weights and a bias, and compute the forward pass for three different input pairs using a simple step activation (output 1 if the sum is positive, else 0). Notice how changing a single weight changes which inputs get classified as 1 — this is the mechanical core of what training adjusts automatically.
Sources and further reading
These primary or specialist references informed the concepts in this guide. Product details can change, so verify current documentation before implementation.