How AI language models actually work.

A large language model (LLM) is a program that predicts the next piece of text. It splits your words into tokens, works out how each token relates to the others, scores every possible next token and picks one. Then it adds that token to the text and repeats, hundreds or thousands of times per answer.

Everything else, from the knowledge to the tone to the apparent reasoning, comes from training: first on trillions of words of text, then on examples and feedback that teach it to act like an assistant. Here is how LLMs work, stage by stage, with no math.

01How do LLMs work? One token at a time

Your phone’s keyboard suggests the next word from the last few you typed. An LLM does the same job at a vastly larger scale, looking at everything in its context window at once.

The unit it works in is the token, a chunk of text that is often a short word or part of a longer one. Qwen3.5, a popular open model family, has a vocabulary of roughly a quarter of a million tokens. Our guide to tokens and context windows shows real examples of how text gets split.

Here is what happens every time a model writes a single token:

1. 01

   Your text becomes tokens

   “The capital of France is” turns into a handful of token IDs, one number per chunk.

2. 02

   Each token becomes a list of numbers

   Every token ID is swapped for a long list of numbers that captures its meaning, called an embedding.

3. 03

   Layers of attention mix in context

   The lists pass through dozens of layers. In each one, every token gathers information from the earlier tokens that matter to it.

4. 04

   The model scores every possible next token

   At the end, the model gives each token in its vocabulary a score. “Paris” scores very high, “Lyon” lower, and “banana” almost nothing.

5. 05

   One token is picked and added

   The model samples from the top candidates, appends the winner and starts again, now with one more token of context.

02Pretraining: reading a library the size of the internet

A fresh model is a huge grid of random numbers, called parameters or weights. Training adjusts them. The method is a fill-in-the-blank game played trillions of times: show the model some text, hide the next token, let it guess, then nudge every parameter so the right answer becomes a little more likely.

The scale is hard to picture:

A person reading 250 words a minute, eight hours a day, would need hundreds of thousands of years to get through 32 trillion tokens. No human has read that much, which is why models know a little about almost everything.

To predict text well, the model has to absorb grammar, facts, styles and patterns of reasoning, because all of them help it guess what comes next. Nobody programs these in. They emerge from the guessing game.

The result is a base model. It continues text, but it is not yet an assistant. Ask it a question and it may answer with more questions, because that is how many quiz pages on the web continue.

Pretraining also sets a knowledge cutoff. Google’s Gemma 4 training data stops in January 2025, and Anthropic lists June 2026 as the reliable knowledge cutoff for Claude Opus 5.5. Anything later is invisible to the model unless you paste it in or give it a search tool.

03Attention: how the model knows what “it” means

Read this sentence: “The trophy did not fit in the suitcase because it was too big.” You know “it” is the trophy. Change “big” to “small” and “it” becomes the suitcase. Getting that right means looking at the whole sentence, not just the last word.

Models run many attention “heads” side by side, each learning a different kind of connection, such as grammar or who did what to whom. They stack dozens of these layers, so later layers work with richer meanings.

This design, the transformer, was introduced by Google researchers in the 2017 paper “Attention Is All You Need”. Nearly every major language model since builds on it, with many refinements.

Attention also explains why long inputs get expensive and fuzzy. Every token compares itself with every token before it, so the work grows fast as the context grows. Anthropic’s engineers describe the fuzziness as context rot: the more tokens in the window, the harder it is for a model to recall any one of them accurately.

04From autocomplete to assistant: fine-tuning and feedback

The helpful assistant you chat with is shaped in a second phase, often called post-training. It uses far less data than pretraining but changes behavior a lot.

Feedback training, often called RLHF (reinforcement learning from human feedback), matters more than its size suggests. In OpenAI’s 2022 InstructGPT study, people preferred answers from a 1.3 billion parameter model trained with human feedback over answers from the 175 billion parameter GPT-3, a model more than 100 times larger.

The newest step rewards the model for reaching answers that can be checked. DeepSeek showed in early 2025 that this kind of reinforcement learning alone can teach a model to reason, and that habits like verifying its own work emerged without anyone writing them in.

05Is it just autocomplete?

“It just predicts the next word” is true about the output and misleading about the inside. To predict well, models build internal machinery that looks a lot like planning.

Anthropic’s interpretability team traced what happens inside Claude in 2025 and found three striking things:

• It plans ahead. Asked to finish a rhyming couplet whose first line ended in “grab it”, Claude picked “rabbit” as the target before writing the second line, then wrote toward it.

• It chains facts. Asked for the capital of the state where Dallas is, it first represented “Texas”, then used that to reach “Austin”, instead of recalling a memorized answer.

• It seems unaware of its own methods. Claude added 36 and 59 using parallel shortcuts, yet when asked how, it described the carry-the-one method taught in school.

The last point is a useful warning. A model’s account of how it reached an answer is itself generated text, not a readout of what happened inside.

06Why LLMs are brilliant at some things and bad at others

Once you know the mechanism, the strengths and weaknesses become predictable.

Take facts. A model does not keep a database of sentences. It stores patterns spread across billions of parameters, so when it lacks a fact, it produces the most plausible-sounding one. That is the root of AI hallucinations.

The one-direction problem is strange but real. A 2023 study called it the reversal curse. Asked about celebrities’ parents, GPT-4 answered questions like “Who is Tom Cruise’s mother?” correctly 79% of the time, but the reverse, “Who is Mary Lee Pfeiffer’s son?”, only 33% of the time.

The same study found that when the fact sits in the prompt, models can work out the reverse. That is the practical lesson: what is in the prompt beats what is in the weights. Give a model the source text and it reasons over it far more reliably than over its memory.

07What reasoning modes add

A standard model commits to its answer token by token. Reasoning modes, also called thinking, let it first write hidden working notes, like scratch paper. It can try an approach, notice a mistake and backtrack, which helps on math, logic, planning and debugging.

Tools extend the same idea. A model that can call a calculator, run code or search the web does not need to guess at arithmetic or recent news. It writes a request, software runs it, and the result lands back in the context as new tokens to predict from. Put that in a loop with a goal and you have an AI agent.

08FAQ

Next, find out when a small model beats a giant one, or learn how to choose an AI model for your work.