If you’ve used ChatGPT, Claude or Gemini, you’ve used a large language model, or LLM. They can draft a proposal, summarise a 40-page report, explain a tax concept or write code — and for a lot of people, they are AI now. But what’s actually going on inside?
This post explains LLMs in plain English: what they do, how they’re built, and why understanding the mechanics helps you use them well. It builds on the previous post on how machines learn — an LLM is that same guess-measure-adjust loop, just at an enormous scale.
The core trick: predicting the next word
At its heart, an LLM does one deceptively simple thing: given some text, it predicts what comes next. Type “The capital of Australia is” and it predicts “Canberra”. Then it adds that word to the text and predicts the next one, and the next, building a full response one piece at a time.
(Strictly speaking it predicts tokens — chunks of words — rather than whole words. We’ll cover tokens in the next post.)
That sounds too simple to produce an essay or working code. The surprise of the last few years is that when you train next-word prediction on enough text with a big enough model, the model has to pick up grammar, facts, writing styles and patterns of reasoning, simply because they help it predict better.
What makes it “large”
“Large” refers to two things: the amount of text it learns from, and the number of parameters — the internal numbers adjusted during training. When OpenAI introduced GPT-3 in 2020, it had 175 billion parameters, and the researchers showed that a model that size could tackle new tasks from just a few examples in the prompt, without retraining[2].
Why go so big? Research by Kaplan and colleagues at OpenAI found that language model performance improved predictably — following smooth “power laws” — as model size, training data and computing power increased[3]. That finding helped kick off the race to build ever-larger models.
It’s worth being clear about what those parameters are not. The model doesn’t store the text it read like a library of documents it can look things up in. It stores patterns — statistical associations between words and ideas — spread across all those numbers. That’s why it can write fluently about almost anything, and also why it can’t reliably tell you where a particular “fact” came from.
The engine: the transformer
Almost every modern LLM is built on a design called the transformer, introduced by Google researchers in a 2017 paper with the memorable title “Attention Is All You Need”[1]. You’ll recognise the T in GPT: Generative Pre-trained Transformer.
The key idea is attention: when processing each word, the model can look at every other word in the text and decide which ones matter most. In “The trophy didn’t fit in the suitcase because it was too big”, attention is what helps work out that “it” means the trophy. Earlier designs read text strictly one word after another, while transformers process words in parallel — which made training on huge amounts of text far more practical[1].
From text predictor to helpful assistant
A model trained only to predict the next word is not yet a helpful assistant. Ask it a question and it might continue with more questions, because that’s what it’s seen online. So building an LLM typically happens in two broad stages:
| Stage | What happens | Result |
|---|---|---|
| 1. Pre-training | Next-word prediction over a vast collection of text | A model with broad knowledge of language and the world |
| 2. Fine-tuning for instructions | Further training on examples of good responses, plus feedback from people ranking answers | An assistant that follows instructions and answers helpfully |
The second stage matters more than you might think. In a 2022 OpenAI study, people preferred responses from a 1.3-billion-parameter model fine-tuned with human feedback over those from the 175-billion-parameter GPT-3, despite it having over 100 times fewer parameters[4]. This technique is known as reinforcement learning from human feedback, or RLHF. Size isn’t everything; how a model is taught to behave matters too.
What LLMs are good at (and not)
Because an LLM is a pattern predictor trained on language, its strengths and weaknesses follow directly:
| Usually strong | Watch out for |
|---|---|
| Drafting and rewriting text | Stating false “facts” confidently |
| Summarising long documents | Knowledge that stops at its training date |
| Explaining concepts in different styles | Precise arithmetic and counting |
| Extracting information from messy text | Knowing your organisation’s private information |
| Writing and explaining code | Consistency on the same question asked twice |
Think of an LLM as a brilliant, extremely well-read new colleague who has never worked at your organisation and sometimes bluffs. Give them clear instructions and the right documents, and they’re remarkably productive. Take their word as gospel on anything important, and you’ll eventually get burned.
Later in this series we’ll dig into each of these: why LLMs sometimes make things up, how much they can “remember” at once, and how to connect them to your own documents.
What I want you to take away
- An LLM predicts the next piece of text. Everything it does is built on that one skill.
- Scale unlocked surprising abilities. More data, parameters and compute brought predictable improvements.
- The transformer is the engine. Attention lets the model weigh every word against every other.
- Assistants are trained twice. Pre-training builds knowledge; instruction tuning makes it helpful.
- Fluent isn’t the same as correct. Use LLMs for drafting and thinking, and check anything that matters.
Working out where AI fits in your business?
I help Australian teams cut through the hype and pick AI projects that actually pay off. If you'd like a second opinion on an idea, I'm happy to talk it through.
Get in touch →References
- Vaswani, A., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems 30. https://arxiv.org/abs/1706.03762
- Brown, T. B., et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33. https://arxiv.org/abs/2005.14165
- Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361. https://arxiv.org/abs/2001.08361
- Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35. https://arxiv.org/abs/2203.02155