Text becomes numbers. Attention mixes context. Softmax picks the next token — and we will compute it live on this screen.
Four levels of the same machine: tokenizer, embedding, attention, sampling. Nothing on the following plates is an animation — every number is computed in the browser while you watch.
By 水水 · November 2026
Formally: a function that maps a sequence of tokens to a probability distribution over the next token.
Training is one operation repeated until it stops helping: show it text, hide the last token, ask it to guess, adjust every weight slightly toward the truth.
FIG. 1 — scale is the mechanism, not the marketing: the pattern table has to be big enough to hold the regularities of written language.
2a — greedy longest-match over a subword vocabulary. Try strawberry: two tokens, so the model never counts three separate rs.
The embedding matrix E is learned, not designed. Nobody wrote down what dimension 2,731 means — gradient descent put it there.
2b — 128k rows × 4,096 columns ≈ 500M numbers, just to turn IDs into vectors. Same matrix is reused at the output end.
2c — the vocabulary also reserves special tokens: <|endoftext|>, <|im_start|>. Those IDs carry structure, not text — they are how a chat becomes a single flat sequence.
2d — Why this matters: the unit of thought is the token, not the word. Pricing, context limits and the classic “how many r in strawberry” failure all live here.
3a — a 2-D projection of a 4,096-D space. The two pink arrows are the same vector: king − man + woman ≈ queen.
Similarity is an angle, not a distance. Directions encode relations: gender, tense, plurality, politeness.
This is also why search works: a query and a document can be compared without ever sharing a word.
3b — context-free so far. The vector for bank is one vector until attention splits it into river bank and bank loan.
Logits are raw scores, any real number. Softmax exponentiates them — that is what forces the outputs to be positive and sum to exactly 1.
Temperature divides before exponentiating. T → 0 collapses to argmax (safe, repetitive). T → 2 flattens toward uniform (surprising, sometimes wrong).
The entropy readout is the honest measure of how unsure the model is at this exact step.
FIG. 4 — drag the slider: every bar, percentage and bit below is recomputed by the browser, not pre-drawn.
| key | q·k | ÷√dk | es | weight |
|---|
5a — dk = 4, so √dk = 2. Click a query word above; the dot products, exponentials and weights are all recomputed.
Each token produces three vectors: a query (what I am looking for), a key (what I offer) and a value (what I pass on if chosen).
The dot product is a compatibility score. Divide by √dk to keep variance from exploding as width grows.
{layers} × {heads} heads run in parallel and get concatenated — 64 heads × 128 dims per layer is typical.
5b — the residual stream: each block adds to the vector, never replaces it. That is why 100 layers stay trainable.
5c — causal mask: token i may only look at j ≤ i.
original →
shuffled →
| pair | weight, original | weight, shuffled |
|---|
6a — same query, same keys, different positions: the weights are identical. Permute the sentence and the attention matrix just gets its rows reordered.
A bag of words would be a perfect model of language if order did not matter. It does, so position has to be injected by hand.
Modern models rotate the query and key by an angle proportional to position. Relative position then falls out of the dot product itself — which is why context windows can stretch to a million tokens without retraining the architecture.
6b — This is the single most overlooked line in every “how LLMs work” explainer. Skip this plate if you are short on time; keep it if the room is technical.
Predict the next token on trillions of tokens. The text is its own label — no human annotation needed. L = -log p(target)
Supervised examples of good answers. Teaches format: how a helpful reply is shaped.
Humans rank pairs of answers; a reward model learns the ranking; the policy is pushed toward preferred outputs.
7a — loss falls as a power law in compute, parameters and data. That law, not a clever trick, is why scale works.
Backpropagation gives the gradient: for every one of the 552 billion weights, which direction makes the loss smaller. Learning rate η says how far to step.
7b — Batch of ~4M tokens per step, tens of thousands of steps, thousands of GPUs, weeks. One run: tens of millions of dollars.
It is the objective function doing exactly what we asked. We reward plausibility, then act surprised when we get plausible falsehoods.
what we train
“Which continuation is most likely in text like this?”
A confident sentence and a true sentence look identical to the loss. Nothing in the gradient rewards checking a fact.
what we want
“Which statement is true?”
Two different questions. Closing the gap takes retrieval, tools, and a model willing to say “I don’t know”.
Which is why the useful habit is dull and non-negotiable: treat it as a very well-read colleague who never checks a citation.
FIG. 8 — the same mechanism that lets it write poetry lets it invent a statute. One capability, two faces.
{ "model": "deepseek-chat", "messages": [ { "role": "system", "content": "Be brief." }, { "role": "user", "content": "Hello!" }, { "role": "assistant", "content": "Hi!" }, { "role": "user", "content": "And you?" } // whole history ], "temperature": 0.7, "stream": true}
Stateless. The server keeps nothing. Every call resends the entire conversation — that is why long chats get slower and more expensive, and why “context window” is a real limit and not a marketing number.
Metered by token. Input and output are billed separately. A 1,000-token system prompt is paid for on every single turn.
FIG. 9 — the model never leaves the kitchen. Your key says who is ordering.
One request, four ingredients. The reply arrives token by token — this is the loop from FIG. 4 running in production.
POST /chat/completionsAuthorization: Bearer sk-****{ "model": "deepseek-chat", "messages": [{"role":"user","content":"Hello!"}], "temperature": 0.7, "stream": true }
what to point at on screen
→ the request panel: the exact JSON, key redacted
→ first token latency vs total time
→ output tokens ÷ seconds = the loop rate
→ drag temperature to 1.6 and ask again — same prompt, different answer, exactly as FIG. 4 predicts
FIG. 10 — the demo page prints its own measurements. Nothing is staged.
Same API, same request — recorded before class in case the network fails.
“Write a poem.” → “Write a 4-line poem about the moon.”
→ narrows the distribution: fewer plausible tokens, less drift.
“You are an English teacher. Correct my sentence.”
→ conditions every attention head on a different slice of the corpus.
“cat → kitten, dog → ?”
→ in-context learning: the pattern is read off the context, no weights change.
“Work step by step, then give the answer.”
→ more tokens = more forward passes = more computation per answer.
FIG. 12 — none of these are tricks. Each one changes the input distribution the same loop is predicting from.
Any questions?