Tokens and context, without the math.

A token is the unit of text an AI model reads and writes. It is often a whole short word, sometimes a piece of a longer one, a digit or a punctuation mark. In English, a token averages about four characters, or roughly three quarters of a word.

A context window is how many tokens a model can hold in view at once. It covers everything in the conversation: the instructions, your messages, attached files and the reply the model is writing. Together, these two ideas explain most of what an AI costs and why it sometimes forgets.

01What is a token in AI?

Think of tokens as the tiles a model spells with, except the tiles come in many sizes. Common words get a tile of their own. Rare words, names and typos are spelled from several smaller tiles.

The vocabulary is learned from huge amounts of text, so frequent chunks earn their own tile. OpenAI’s tiktoken documentation gives a classic example: “encoding” usually splits into “encod” and “ing”, because “ing” is so common in English.

Here is how one of OpenAI’s tokenizers, the one introduced with GPT-4o, splits a few examples from OpenAI’s own cookbook. The ␣ symbol marks a space.

Two patterns stand out. Everyday English words cost one token each, with the space in front of them included. Anything less common, from a brand name to a sum to another alphabet, breaks into more pieces.

02How many words is a token?

Each AI company publishes a rough conversion for English. They agree on the shape but not the exact number, because every model family uses its own tokenizer.

In everyday terms, a 150-word email is about 200 tokens by OpenAI’s rule. A 1,500-word article is about 2,000 tokens, and a 90,000-word novel about 120,000.

Other languages, numbers and code symbols usually take more tokens per character than plain English, as the table above shows. Leave extra room for them.

03What is a context window?

Picture a desk. What the model learned in training is like everything a person learned in school: broad, but fixed. The desk holds what is in front of it right now, and the model can only work with what fits on it.

Everything in a request takes up desk space:

• The system prompt, the hidden instructions an app gives the model

• Every earlier message, yours and the model’s

• Files, images and documents you attach

• For agents, the descriptions of their tools and every tool result

• The reply being written, including any hidden thinking

Pictures and sound become tokens too. Google’s Gemini docs count a small image as 258 tokens and each second of audio as 32, so an hour-long recording fills about 115,000 tokens.

As of September 2026, Anthropic’s current Claude models take one million tokens, roughly 555,000 words, and can write up to 128,000 tokens in a single reply. That is several novels of desk space. As you will see below, that does not mean you should fill it.

04Input vs output tokens: why output costs more

Services that charge by use bill the two directions separately. Input tokens are everything you send, which in a chat includes the whole conversation so far. Output tokens are what the model writes back.

Output is the expensive side. As of September 2026, every current Claude model charges five times more per output token than per input token, from Haiku 4.5 ($1 in, $5 out per million) to Fable 5.1 ($10 in, $50 out). Google’s Gemini 3.8 Flash uses the same ratio at $0.75 and $3.75. Our API pricing comparison runs the same math across five labs and three everyday tasks.

The reason is how the work happens. A model reads your prompt in one parallel sweep, the way you take in a page at a glance. It writes one token at a time, and each new token needs another full pass through the model. Writing is simply more work per token. For what happens to those tokens next, see how a language model predicts text.

Two details surprise people:

• The whole chat is sent again every turn. The model does not remember earlier turns on its own. Each new message sends the full history as input, so message forty costs more than message one. Prompt caching softens this: on Anthropic’s models, reading a cached prefix costs 10% of the normal input price or less.

• Thinking is billed as output. Models that reason before answering produce hidden thinking tokens, and both Anthropic and Google bill them at the output rate. Our guide to reasoning models covers when that is worth paying for.

To spend less without getting worse answers, see how to cut your AI bill.

05Why long chats get worse

Two things happen as a conversation grows. First, quality slips. Anthropic’s documentation says accuracy and recall degrade as the token count grows, a pattern known as context rot. Early instructions and small details get less attention.

Second, the window eventually fills, and something has to give. Some chat apps quietly drop the oldest messages first. That is why a chatbot can forget a rule you set at the very start.

Watch for these signs:

• It brings back a mistake you already fixed.

• It ignores a preference you stated early on.

• It mixes up old and new versions of a document.

• Its answers get vaguer and more generic.

For the full toolkit on keeping a window lean, read our guide to context engineering.

06How compaction keeps a conversation going

Anthropic’s API can compact a conversation on its own servers, and its docs give the reason plainly: response quality degrades as a conversation grows. Some chat apps and coding agents compact for you as the window fills.

The catch is that a summary loses detail. When the work matters, a little manual care does better:

1. 01

   Start a new chat for each new task

   A fresh window has no leftover clutter. Unrelated history only costs tokens and attention.

2. 02

   Summarize before you switch

   Ask the model for a short brief of the conversation, check it for mistakes, and paste it into the new chat.

3. 03

   Keep the essentials outside the chat

   Put standing rules, facts and decisions in a document or project file you can attach again. Then nothing important depends on the chat’s memory.

4. 04

   Paste the part that matters

   Share the relevant section of a file or log, not the whole thing. Smaller inputs are cheaper, faster and more accurate.

Write a brief I can paste into a new chat so we can continue without this history.
Include the goal in one sentence, the decisions we made and why, the facts I gave you that still matter, and the next steps.
Leave out anything we already solved or abandoned. Keep it under 200 words.

07FAQ

Next, see how to choose an AI model for the job, or learn the wider craft of context engineering.