Understanding AI Tokens: What You’re Actually Paying For.

For about fifty years, software had one defining economic property, and it shaped everything about how we bought it. Once you’d written the program, making another copy cost essentially nothing. The millionth copy of a word processor cost the same to produce as the second: zero. That single fact gave us licences, subscriptions, free tiers, and the whole idea that software, once built, is basically free to run.

Artificial intelligence breaks that. Every answer an AI gives you is manufactured fresh, on the spot, burning real electricity through real silicon in a building that needs cooling. The millionth answer costs exactly what the first one cost. Nothing is copied. Everything is produced.

This is why people in the industry have started describing the machines that run AI not as computers but as factories, and why the unit they produce has a name you’ll hear more and more: the token. If you want to understand what you’re actually paying for when you use AI, and why the bills surprise people, you need to understand tokens. It’s simpler than it sounds.

What a token actually is

A token is the smallest chunk of text an AI model deals with. Not a letter, not quite a word. Something in between.

When you type a sentence, the model can’t read it the way you do. It first chops your text into pieces from a fixed vocabulary it learned during training. Common words usually survive intact, “the,” “cat,” “running” are each one token. Longer or rarer words get broken up: “unbelievable” might become “un,” “believ,” and “able.” A very unusual name, or a word in a language the model saw rarely, could shatter into four or five pieces. Punctuation and spaces count too.

The rough conversion worth memorising: a token is about four characters, or roughly three-quarters of an English word. A hundred words of text is around 130 tokens. This page you’re reading is a couple of thousand.

Once your text is in tokens, each token becomes a number, and the model, which is fundamentally a machine for predicting what comes next, works entirely in those numbers. It never sees your words. It sees a sequence of tokens and calculates which token should come next.

The life of a single exchange

Here’s what happens when you ask an AI a question, in the order it happens.

1. Your words are broken into tokens. The sentence becomes a list of numbered chunks. These are your input tokens.
2. The model reads them, all at once. It takes in the entire prompt in a single pass, working out how each token relates to every other. This step is fast, because it can be done in parallel, everything is already there in front of it. In the trade this is called the prefill.
3. The model writes, one token at a time. This is the crucial part. It predicts the single most likely next token, adds it to the end, then looks at everything again, including what it just wrote, and predicts the next one. Then again. And again. Each new token depends on every token before it, so this cannot be done in parallel. It happens strictly in sequence. These are your output tokens.
4. The tokens are reassembled into words and appear on your screen.

That’s it. Everything else about the economics falls out of that fourth step.

Why writing costs more than reading

Here is the single most useful thing a non-technical person can understand about AI costs: output tokens cost several times more than input tokens. Across the industry, generating text typically costs three to five times as much as reading it, sometimes far more.

It isn’t arbitrary pricing. It’s mechanical, and the reason is the difference between prefill and decode. Reading a page is like taking it in with your eyes, you can absorb the whole thing more or less at once, and a machine can spread that work across thousands of processors simultaneously. Writing a page by hand is different: you cannot write the fiftieth word before the forty-ninth, because what you write next depends on what you’ve written so far. Each word waits for the one before it.

Which means the machine spends far more of its expensive time producing your answer than consuming your question.

The practical consequence for anyone spending money: the length of the answer matters more than the length of the question. Asking a model to be concise isn’t just good style. It’s the highest-leverage cost control you have.

Context Window: The desk, not the filing cabinet

Every model has a context window: the maximum number of tokens it can hold in mind at once. Think of it not as a filing cabinet, where things are stored and retrieved, but as a desk. Everything the model can consider must be on the desk at the same time. When the desk is full, something falls off.

This produces the most counterintuitive cost in AI, and it catches almost everyone.

The model has no memory between messages. So when you send the tenth message in a conversation, the entire conversation, all nine previous exchanges, has to be handed back to it as input, every single time. Your tenth question isn’t billed as one question. It’s billed as one question plus the whole transcript that came before.

Imagine a consultant who forgets everything the moment you stop speaking, so before each new sentence you must read them the complete minutes of the meeting so far.

It’s worth being precise here, because this is the source of a widespread misunderstanding. The model doesn’t have a weak memory, or a short one. It has none at all. Every mainstream AI model is what engineers call stateless: it looks at what’s in front of it, produces an answer, throws away everything it was thinking, and ends. Nothing is retained. So when a chat application appears to remember you across weeks, no model is remembering anything. A separate piece of software stored some facts about you in an ordinary database, fetched the relevant ones, and slipped them into the prompt before the model ever saw it. Memory is a feature of the system around the model, never of the model itself.

Which leads to the sentence I’d most like you to take away. Continuity is not free. It is purchased, every single turn.

Left unmanaged, that gets expensive quickly, because turn twenty pays to re-read turns one through nineteen. Fortunately, there is a mechanism that stops this from being ruinous, and it is the most important cost lever in the whole system.

The discount for repeating yourself

If every conversation re-sent its entire history at full price, anything long would be prohibitively expensive. It isn’t, because of caching.

When you send the same block of text repeatedly, the same instructions, the same document, the same conversation so far, the provider can keep it in a ready-processed state instead of chewing through it again from scratch. Reusing that stored version costs a fraction of the normal price, on some platforms as little as a tenth.

So there are really three prices, not two, and the distance between the cheapest and the dearest is not a few percent. It is multiples.

• Cached input: text the machine has already digested. Cheapest by far.
• Fresh input: text it must read for the first time.
• Output: text it must write, one token at a time. The most expensive by a distance.

Once you can see those three tiers, two rules follow, and they explain a lot of otherwise baffling bills.

Keep the unchanging things at the front. Caching only works if the beginning of your prompt is identical to last time. Put your stable instructions and your conversation history at the front, everything matches, and you ride the discount. Put something that changes on every request at the front, a timestamp, a session number, a personalised greeting, and you break the match, so everything after it must be reprocessed at full price. Organisations lose real money to a single variable sitting in the wrong place.

Caches go cold. Stored text doesn’t sit there indefinitely. It expires, typically within minutes. A brisk back-and-forth stays cheap. The same conversation with a long pause in the middle costs noticeably more, because the cache expired and the whole transcript has to be read again from scratch. Time is now a variable in your bill, which is not something software ever did.

There’s a second approach, which you’ll meet in AI products that claim to remember you. Rather than re-sending fifty turns of conversation word for word, the system stores a short summary and injects a couple of hundred tokens instead. Far cheaper. But something, usually another AI, decided what was worth keeping, and you are now reasoning over its impression of the conversation rather than the conversation itself. You bought the saving with fidelity.

And notice what caching does and doesn’t do. It does not give the model a memory. The model still forgets everything, every time, and still receives the whole transcript on every turn. Caching only means that re-reading is cheaper than reading. The forgetting is permanent and total. All that moves is the price of the reminder.

Where the tokens actually go

Once you can see tokens, you can see where the money goes. There are four main consumers, in roughly ascending order of appetite.

• Ordinary chat is the cheap case: a short question, a moderate answer, a growing transcript.
• Documents inflate the input side dramatically. When an AI system answers questions about your files, contracts, policies, a knowledge base, it works by stuffing the relevant chunks of those documents into the prompt. You’re now paying to feed it a small library on every request. This is why systems that answer questions about your own data cost more per query than plain chat, even though the answers look similar.
• Reasoning models introduce a cost that’s genuinely hidden. Some models “think” before they answer, generating long internal chains of reasoning that they work through and then discard, showing you only the conclusion. Those thinking tokens are billed as output, and there can be far more of them than the visible reply. You pay a consultant for their afternoon of thinking and receive a one-page memo. The memo is what you see. The afternoon is what you bought.
• Agents are the big one. An agent doesn’t answer once; it works. It plans, calls tools, reads what came back, tries again when something fails, loops, checks, and continues, and every step in that loop is another round of tokens in and tokens out, with the growing history resent each time. Where a chat message costs you one exchange, an agent running for an hour can cost you thousands. This is where AI budgets go from a line item to a problem, and it’s why the arrival of autonomous agents changed the economics of AI more than any price cut.

Pictures, sound, and video: every medium has an exchange rate

So far I’ve talked about text, but the same machinery handles images, audio, and video. The mechanism doesn’t change: whatever you send gets chopped into discrete numbered chunks, and the model works over those chunks. What changes, dramatically, is the exchange rate. How many tokens does one photograph buy? One second of speech? One second of film?

Images become patches. The picture is laid over a grid of small tiles, and each tile becomes a single token. Different providers cut the grid differently, but the effect is the same: the more pixels, the more tiles, the more tokens.

Run the arithmetic and you get one of my favourite facts in this whole subject. A one-megapixel image – 1024 pixels square, the size most AI tools default to – works out at somewhere between seven hundred and fourteen hundred tokens, depending on whose machine is looking, and a token is about three-quarters of a word. “A picture is worth a thousand words” is not a proverb. It’s a conversion rate.

And here is the part that matters most. The resolution dial and the cost dial are the same dial. Some services offer a low-detail mode that charges a small flat fee for any image, no matter how large. That’s cheap because the model isn’t looking at your image. It’s looking at a thumbnail of it. It will miss the small print, the number on the dial, the face in the background.

Your phone takes photographs of around twelve megapixels. At full resolution, one of them would cost something like sixteen thousand tokens. It never does, because every provider caps what it will look at: if your picture is too large, it gets shrunk before it is ever tokenised. Twelve megapixels go up; one megapixel gets looked at. Nobody asks you.

This explains something people notice and can’t account for: why did the AI misread the text in my screenshot? Because somewhere in the pipeline, something shrank it to save tokens. You didn’t pay for those pixels, so the model never saw them.

It’s worth stating plainly, because it’s the difference between text and everything else. For text, breaking words into tokens loses nothing; it’s just bookkeeping. For images, sound, and video, tokenising is compression, and compression means the model sees less than you sent.

Video is images multiplied by time, and time is merciless. A video isn’t processed as a video. It’s sampled into still frames, each one tokenised as an image, with the soundtrack tokenised alongside it. At a modest sampling rate, one second of footage costs a couple of hundred tokens. A minute runs to tens of thousands. An hour of video can consume a million tokens, which on many models is the entire context window. One hour of footage fills the desk completely, and there is no room left for anything else.

The sampling rate isn’t only a cost setting, either. It’s a blindness setting. If the model looks at one frame per second, it never sees the other twenty-nine. Whatever happened between those frames did not happen, as far as it is concerned. Cheap video analysis cannot see fast events. It doesn’t miss them; for the model, they were never there.

Audio comes in two flavours, and they cost very differently. The cheap approach transcribes speech into text first, then processes ordinary text tokens. It works, and it discards everything that wasn’t a word: the tone, the sarcasm, the pause before the answer, whether that was a laugh or a cough. The expensive approach slices the sound itself into audio tokens, on the order of thirty per second, so the model hears rather than reads. You pay more because you’re buying back everything the transcript threw away.

Music can only be done the second way. There is no transcript of a song that you can turn back into the song, which is why music models generate audio directly, and why they took longer to arrive than text ones.

And when you generate rather than consume, the numbers turn brutal. Remember that output always costs several times more than input. Pictures and video are output too. Making a single image costs on the order of a thousand tokens; making video costs thousands of tokens for every second of footage. A half-minute clip can cost more to produce than a book’s worth of prose.

So the rough exchange rates, worth carrying in your head:

• A word: a little over one token
• A photograph: around a thousand
• A second of speech: around thirty
• A second of video: a couple of hundred at a glance, thousands at a stare
• A second of video you generate: thousands, always

Which reframes the whole business rather neatly. You are not really paying for content. You are paying for attention, and for how much of each medium the machine is permitted to notice. Words are cheap to attend to. Pixels are not. Time is the most expensive of all, because every second demands a fresh look.

Why the bill surprises people

Put it together and you can see why AI spending behaves so unlike software spending.

Software costs were a licence: predictable, flat, indifferent to how much you used it. You bought it and forgot about it. Tokens are a utility. You pay for consumption, and consumption is driven by things nobody is watching. How verbose the answers are. How long the conversations run. How much context gets stuffed in. Whether the cache is being hit, or quietly missed because someone put a timestamp in the wrong place. How many times an agent retried a failing step at three in the morning. Whether someone quietly turned the image resolution down to save money.

None of that appears on a purchase order. All of it appears on the bill.

The meter is always running

So when you hear that a computer is becoming a factory, and that it manufactures tokens, take it literally rather than as marketing. It is a description of a real change in what computing costs.

Every AI answer is a small manufactured object, made to order, and the raw materials are electricity, silicon time, and cooling.

That’s why a token has a price, why writing costs more than reading, why the tenth question costs more than the first, and why an agent left running unsupervised can quietly spend more in an afternoon than a team of people would in a month. Even the machine’s apparent memory of you is a thing you rent, one turn at a time.

You don’t buy AI. You buy tokens, in bulk, and then something decides how fast to consume them. Understanding that is most of what you need to spend wisely, and the beginning of asking the only question that really matters: what, exactly, is producing all these tokens, and does anyone have their eye on the meter?