What is a neural network and how does it learn?

A neural network is a large stack of simple mathematical operations with adjustable numbers in between, and it 'learns' by repeatedly measuring how wrong it is and nudging every number slightly in the direction that reduces the error. That's genuinely the whole idea; the calculus is just how you compute 'which direction'. The structure: information flows through layers of nodes. Each connection between nodes has a *weight* — a number that says how much this input matters to that node. A node adds up its weighted inputs, applies a simple non-linear function, and passes the result on. Stack enough layers and the network can represent extremely complex relationships between input and output. The learning loop, in four steps: 1. **Forward pass.** Feed in an example — say an image of a handwritten digit. It flows through the layers and produces an output: 'I think this is a 3'. 2. **Measure the error.** Compare to the correct answer ('it was an 8') using a loss function that turns the wrongness into a single number. 3. **Backward pass (backpropagation).** Work backwards through the network computing, for every single weight, whether nudging it up or down would have reduced the error, and by how much. This is the clever part and it's what makes training large networks feasible. 4. **Update.** Adjust every weight a tiny amount in the error-reducing direction. The 'tiny amount' is the learning rate — too large and it overshoots chaotically, too small and it takes forever. Repeat millions of times across millions of examples. Each individual update is almost imperceptible; the accumulation is what produces capability. The intuition that helps most: imagine standing on a foggy hillside trying to reach the lowest valley, able to feel only the slope directly under your feet. You step downhill repeatedly. That's gradient descent — the loss is the altitude, the weights are your position, and backpropagation tells you which way is down. You might land in a local dip rather than the global minimum, which is fine in practice for reasons that are still not fully understood theoretically. What makes deep networks powerful: early layers learn simple features (edges, in an image), middle layers combine them into parts (an eye, a wheel), later layers into concepts (a face, a car). Nobody designs this hierarchy — it emerges because it's an efficient way to reduce error. The same principle applies to text, where early layers capture syntax and later ones capture meaning and context. The honest limitation to hold onto: this is pattern-fitting to the training distribution. It's remarkably powerful, and it's also why these systems fail strangely on inputs unlike anything they were trained on.