What are tokens?

If you’ve looked at the pricing page for any AI service, you’ve probably seen a line like “$3 per million input tokens” and wondered what on earth a token is. It’s one of those words that gets thrown around as if everyone already knows it — and it turns out to matter a great deal, because tokens are how AI models read, how they’re limited, and how you’re billed.

The good news is that the idea is simple. In this post I’ll explain what a token is, why AI models use them instead of words, and the handful of practical things every business person should know about them before signing up for an AI tool or budgeting for an AI project.

A token is a chunk of text

A large language model (we covered those in the previous post, What Is a Large Language Model?) doesn’t actually see words or letters. Before your text reaches the model, a piece of software called a tokeniser chops it into small chunks called tokens, and turns each chunk into a number. The model only ever works with those numbers.

A token can be a whole word, part of a word, a single character or a piece of punctuation. Common words like “the” or “business” are usually a single token. Longer or rarer words get split into pieces. Something like “unbelievable” might become “un”, “believ” and “able”. The exact split depends on the tokeniser each model uses.

For everyday English, OpenAI’s rule of thumb is that one token is roughly four characters, or about three-quarters of a word — so 100 tokens is around 75 words^[1]. That’s the number I use for quick estimates.

Why not just use words?

It seems odd at first. Why not give the model a dictionary of words? The problem is that language is messy. There are millions of words once you count names, typos, technical jargon, product codes, slang and every other language on earth. A word-only dictionary would either be impossibly large or constantly missing words it had never seen.

Splitting text into sub-word pieces solves this neatly. A popular technique called byte-pair encoding was shown to let translation models handle rare and unseen words by building them out of smaller, familiar pieces^[2]. Frequent words get their own token; unusual words are spelled out from fragments. The model ends up with a manageable vocabulary that can still represent almost any text you throw at it. OpenAI’s open-source tokeniser, tiktoken, is a fast implementation of exactly this approach^[4].

Why tokens matter to your business

Here’s where it stops being trivia. Tokens show up in three places that directly affect what you can do and what you pay.

1. Pricing

Most AI APIs charge per token, usually quoted per million tokens. They typically price input tokens (everything you send: your question, instructions, any documents) separately from output tokens (what the model writes back), and output is often the more expensive of the two. A long prompt with a short answer and a short prompt with a long answer can cost very different amounts.

2. Limits

Every model has a maximum number of tokens it can handle in one go — its context window. I’ll cover that properly later in this series, but the short version is: the limit is measured in tokens, not pages or words.

3. Speed

Models generate their answers one token at a time. More output tokens means a longer wait. If you’re building something customer-facing, shorter answers feel snappier.

Because you’re charged per request, token costs grow in step with usage. I’ve written a whole article on what that means once a product scales: The Hidden Risk of Pure AI Products: Token Cost at Scale.

Not all text costs the same

One thing that surprises people: the same message can use a very different number of tokens depending on the language it’s written in. Tokenisers are mostly trained on large amounts of English text, so English tends to be encoded efficiently. Researchers found that the same text translated into different languages can produce tokenised versions that differ in length by up to 15 times in some cases^[3]. That means higher costs, slower responses and less room in the context window for speakers of those languages^[3].

For Australian organisations serving multilingual communities, that’s worth knowing. A chatbot answering in Vietnamese, Arabic or Hindi may cost noticeably more per conversation than the same chatbot answering in English. It’s worth testing with real examples before you set a budget.

Code, numbers and unusual formatting can also be token-hungry. Long strings of digits, tables full of IDs and heavily formatted text often break into more tokens than you’d expect from the character count.

Simple ways to use fewer tokens

You don’t need to obsess over every token, but a few habits keep costs and delays down:

• Trim your instructions. Prompts tend to grow as people add “just one more rule”. Review them and cut what isn’t pulling its weight.
• Send only the relevant part of a document. If the answer is on page 3, don’t paste in all 40 pages.
• Ask for the length you need. “Answer in three bullet points” is cheaper and usually more useful than an open-ended essay.
• Test with your real data. Estimate tokens using a sample of your actual documents and conversations, not a guess based on word counts.

What I want you to take away

1. A token is a chunk of text — a word, part of a word or a symbol. AI models read and write in tokens, not words.
2. In English, 1 token is roughly ¾ of a word. Use that for quick estimates, then measure properly when it matters.
3. Tokens drive cost, limits and speed. Input and output tokens are usually priced separately, and output is often dearer.
4. Language matters. The same content can take many more tokens in some languages than in English.
5. Small habits add up. Shorter prompts, relevant context only and requested answer lengths keep your bill and wait times down.