What is next-token prediction? Guessing the one token that comes next

Guessing the one token that comes next, based on all the text so far.

Large language models write in a surprisingly simple way. They look at all the text so far, give every possible next token a probability, and pick one. Then they add it to the text and do it again, hundreds of times, until the answer is done.

Training is the same game. The model reads huge amounts of text with the next token hidden, guesses it, and gets nudged a little whenever it's wrong. Repeated an enormous number of times, this simple drill teaches grammar, facts and style.

But the most likely next token isn't always the true one. That's one reason AIs sometimes make things up.

Think of it like this

Think of the word suggestions above your phone keyboard. If you kept tapping a suggestion, you'd build a whole sentence one word at a time. An LLM does the same thing, just with far better guesses.

For example

After "I ate a slice of …", "pizza" gets a high probability and "homework" a very low one.

Common mix-ups

Mix-upAI comes up with the whole answer at once.
ActuallyIt doesn't even decide in advance how long the answer will be. It stops the moment a special "end here" token is picked, which is why the same question gets answers of slightly different lengths each time.
Mix-upThe most likely next word is the true one.
ActuallyProbability is closer to "how often did this appear in text", so a widespread mistaken belief can score high too. "A goldfish's memory lasts three seconds" — often written, but not true — is one such example.

Try it in class

For teachers at the start of a lesson — or for students working on their own.

Question: Why would doing nothing but "guess the next word" — a huge number of times — teach grammar and style too?
Activity: Write the start of a sentence on the board and have everyone write down the next word. Count how many picked each word to make a "class probability table", then compare it with the probabilities an open AI model gives in Daily's "Today's Next Token" game.