The Idea That Changed Everything: How Neural Networks Actually Work
Neural networks power everything from your phone camera to GPT-4. Here's what they are, how they work, and why they changed computing forever.
The Question That Started It All
In the 1940s, Warren McCulloch and Walter Pitts asked a simple question:
Can we model how the human brain works — using math?
That question led to one of the most consequential ideas in computing history.
What a Neural Network Actually Is
Your brain has roughly 86 billion neurons. Each neuron receives signals, processes them, and fires — or doesn't — based on whether the signal is strong enough.
A neural network borrows this exact idea.
Instead of biological neurons, you have nodes. Instead of synapses, you have weights — numbers that determine how strongly one node influences another. Nodes are organised into layers:
- Input layer — receives raw data
- Hidden layers — find patterns
- Output layer — produces the answer
A simple example: you want to teach a network to recognise a handwritten "3".
You feed it thousands of images of the number 3. Each pixel is an input. The hidden layers learn which pixel combinations matter. The output layer says: "This is a 3 — with 97% confidence."
Nobody programmed the rules. The network learned them.
Why It Stayed Quiet for 50 Years
The idea existed since the 1950s. So why did neural networks only explode recently?
Three things were missing:
Data. Neural networks need massive amounts of training data. The internet created it.
Compute. Training a deep network requires billions of calculations. GPUs made it feasible.
The backpropagation paper. In 1986, Rumelhart, Hinton, and Williams published Learning Representations by Back-propagating Errors — the algorithm that finally made training deep networks practical. This is the foundational paper.
For decades, the field waited for the hardware to catch up to the idea.
The Moment Everything Changed
In 2012, a neural network called AlexNet — built by Geoffrey Hinton's team at University of Toronto — entered the ImageNet competition.
It didn't just win. It destroyed the competition. Error rate dropped from 26% to 15% — a gap that shocked the entire research community.
Deep learning went from academic curiosity to industry priority overnight. Hinton later won the Nobel Prize in Physics for this work in 2024.
Where Neural Networks Run Today
Your phone camera — every computational photography feature (portrait mode, night mode, scene detection) runs a neural network on-device.
Google Search — BERT, a transformer-based neural network, understands the meaning of your query, not just the keywords.
Medical imaging — networks detect tumours in X-rays with accuracy matching senior radiologists. Stanford's CheXNet paper demonstrated this in 2017.
Language models — GPT-4, Claude, Gemini are all neural networks at their core. Transformers — a specific architecture introduced in the 2017 paper Attention Is All You Need — are what made modern LLMs possible.
The Core Idea Worth Remembering
A neural network doesn't follow rules. It finds patterns.
You don't tell it "a cat has pointy ears and whiskers." You show it 10 million cats and it figures out what makes a cat — represented as billions of decimal numbers called weights.
That's the shift. From programming logic to learning from examples.
Key Papers Worth Reading
- A Logical Calculus of Ideas Immanent in Nervous Activity — McCulloch & Pitts, 1943
- Learning Representations by Back-propagating Errors — Rumelhart, Hinton, Williams, 1986
- ImageNet Classification with Deep CNNs (AlexNet) — Krizhevsky, Sutskever, Hinton, 2012
- Attention Is All You Need — Vaswani et al., 2017
Where This Goes
Neural networks didn't replace programming. They added a new kind of programming — one where the machine writes its own rules from data.
Every major AI system you use today traces back to McCulloch and Pitts asking whether a brain could be modelled with math.
It can. And we're still figuring out how far that goes.