How does a machine learn?

“The model learned it from the data.” It’s a phrase that gets said a lot, usually with a wave of the hand. But what does it actually mean for a machine to learn? There’s no student sitting at a desk, no textbook, no moment of understanding.

In the previous post we saw that machine learning means learning patterns from examples instead of following hand-written rules. This time I want to open the box a little and show you how that learning works — without a single equation. Once you’ve got this, a lot of AI’s strengths and weaknesses start to make sense.

A definition worth remembering

My favourite definition comes from Tom Mitchell’s classic 1997 textbook Machine Learning:

It sounds academic, but it’s wonderfully practical. Take a spam filter: the task is sorting emails, the performance measure is how many it sorts correctly, and the experience is the pile of past emails people have marked as spam or not. If it gets better at sorting as it sees more examples, it’s learning. That’s it. No magic required.

The basic recipe: data in, model out, predictions

Almost every machine learning project follows the same recipe:

1. Collect examples. Past data where you know the answer — last year’s customers and whether they cancelled, say.
2. Train a model. The computer searches for patterns that connect the inputs (how often they logged in, how many support tickets they raised) to the answer (cancelled or stayed).
3. Test it. Check the model on examples it hasn’t seen before.
4. Use it. Feed in new customers and get a prediction about who’s at risk.

The “model” at the end is simply the learned pattern, stored as a set of numbers. When people talk about “training a model”, they mean steps 1 to 3.

How the learning actually happens: guess, measure, adjust

Here’s the heart of it. Training is a loop that repeats thousands or millions of times:

• Guess. The model makes a prediction for an example. At first, its guesses are basically random.
• Measure. It compares the guess with the real answer and calculates how wrong it was. This “wrongness” score is called the loss.
• Adjust. It nudges its internal numbers slightly in whichever direction would have made the loss smaller.

Picture tuning an old radio: you turn the dial a little, listen to whether the static gets better or worse, and keep adjusting until the signal is clear. Free courses like Google’s Machine Learning Crash Course walk through this loop — loss, small adjustments, repeat — in more detail if you’d like to go a step deeper^[3].

For the neural networks behind modern AI, the “adjust” step uses a technique called backpropagation, popularised in a 1986 paper by David Rumelhart, Geoffrey Hinton and Ronald Williams. It works out how much each internal connection contributed to the error by passing that error backwards through the network, so every connection can be tweaked in the right direction^[2]. Nearly four decades later, it’s still how deep learning models are trained.

Three ways machines learn

Not every problem comes with neat answers attached. Machine learning is usually split into three styles — the first two are how libraries like scikit-learn organise their guides^[4]:

Most business projects I see are supervised learning. If you have historical records with the outcome you care about, you’re most of the way to a supervised problem.

Where it goes wrong

Understanding the recipe also explains the classic failure modes:

Overfitting: memorising instead of learning

A model can get so good at its training examples that it memorises them, quirks and all, and then stumbles on anything new — like a student who memorised last year’s exam answers. That’s why step 3 matters: you always hold back data the model never saw during training, and judge it on that.

Garbage in, garbage out

A model can only learn patterns that exist in its data. If your records are incomplete, mislabelled or out of date, the model will faithfully learn the wrong lessons.

Learning the past, including its biases

Models learn from history. If past decisions were unfair or the world has changed since, the model will repeat yesterday’s patterns unless someone checks.

What I want you to take away

1. Learning means getting better with experience. Task, performance measure, experience — that’s the whole definition.
2. Training is guess, measure, adjust — repeated. The model shrinks its error a little at a time.
3. A model is a stored pattern. It’s a set of numbers, not understanding.
4. Supervised learning is the business workhorse. If you have past outcomes, you can often predict future ones.
5. Always test on unseen data. It’s the only honest measure of whether a model learned or just memorised.
6. Your data sets the ceiling. No model can learn what your data doesn’t contain.