All labs · 💬 How ChatGPT works
How does ChatGPT work?
Build a tiny language model yourself: see how text becomes tokens, how the next word is predicted, what “temperature” does, and how words become numbers. Same ideas as ChatGPT – just much smaller.
AI doesn't read letters or words – it reads tokens: common words are one token, rare words are split into pieces. The tokenizer learns which pieces to join from lots of text.
What it learned last
Drag the slider to 0: every letter is a token. Each step joins the pair seen most often in the stories – first “th”, “e␣”… later whole words.
🎯 Mission
Write a sentence of 6 or more words that needs no more than 2 extra tokens than words (use words the tokenizer knows!).
Click a word to add it – you are the language model!
🎯 Missions
- Set temperature to 0 and write twice – is it the same?
- Try temperature 1.5 or more.
- Train it on your own text.
To a computer, every word is a list of numbers. Words with similar meanings get similar numbers – so they sit close together on a map. (Real models use thousands of numbers; ours uses 11 you can read.)
royal people animals food places
🧮 Word maths
Take a word, subtract one, add another – which word is nearest?
🎯 Find queen, then try paris − france + england.
🤖 Talk to the tiny model
It continues your words using what it learned in step 2. Ask “the capital of mars is” – it makes something up, sounding sure. That's called a hallucination: it predicts likely words, it doesn't check facts.