Generation & probability, explained simply
Generative AI makes new things: stories, pictures, music. A text generator works by predicting what piece should come next, again and again.
After "Once upon a", the word "time" is very likely. The model has learned these likelihoods, called probabilities, from huge amounts of example text. It picks a next token, adds it, and predicts again.
It does not always pick the most likely option. A little randomness makes answers varied and creative, which is why the same prompt can give different results. It also means the model can produce things that sound right but are not true.
Example: Guessing the next word
- "The cat sat on the …"
- Likely next words: mat (common), sofa, roof, moon (rare).
- Pick "sofa" and carry on: "The cat sat on the sofa and …"
- Choosing differently each time is how one prompt can grow many stories.
Try this
- Play a story game where each person adds one word. Notice how the likely words keep the story sensible.
- Write the three most likely next words after "Happy birthday to …" and one surprising one.
- Ask why a generator that always picked the most likely word might be boring.