← 课件 / 博客
A Field Guide to Large Language Models Opening  ·  PL. 01 / 15
next_token softmax(QKT/√d) embed( ) temperature attention api.call
English Sharing · 4 Minutes · Live Demo

How Does AI Talk?

Text becomes numbers. Attention mixes context. Softmax picks the next token — and we will compute it live on this screen.

Abstract

Four levels of the same machine: tokenizer, embedding, attention, sampling. Nothing on the following plates is an animation — every number is computed in the browser while you watch.

128k
vocabulary
4,096
dimensions per token
1M
context window

By 水水 · November 2026

FIG. 1 · The Object

What is a large language model?

Formally: a function that maps a sequence of tokens to a probability distribution over the next token.

Textwhat you type
Model10¹¹–10¹² numbers
p(next)one score per token
Tokensampled, appended

Training is one operation repeated until it stops helping: show it text, hide the last token, ask it to guess, adjust every weight slightly toward the truth.

≈ 552B weights / 16B active — MoE ≈ 15T training tokens 2×1025 FLOPs to train

FIG. 1 — scale is the mechanism, not the marketing: the pattern table has to be big enough to hold the regularities of written language.

FIG. 2 · Text → Numbers

Tokens: the model never sees letters

Byte-level BPE · live

2a — greedy longest-match over a subword vocabulary. Try strawberry: two tokens, so the model never counts three separate rs.

token id → row i of E
x = E[i] ∈ ℝ4096

The embedding matrix E is learned, not designed. Nobody wrote down what dimension 2,731 means — gradient descent put it there.

2b — 128k rows × 4,096 columns ≈ 500M numbers, just to turn IDs into vectors. Same matrix is reused at the output end.

2c — the vocabulary also reserves special tokens: <|endoftext|>, <|im_start|>. Those IDs carry structure, not text — they are how a chat becomes a single flat sequence.

2d — Why this matters: the unit of thought is the token, not the word. Pricing, context limits and the classic “how many r in strawberry” failure all live here.

FIG. 3 · Geometry of Meaning

Meaning becomes direction

dim 1 dim 2 animals colors verbs king queen man woman

3a — a 2-D projection of a 4,096-D space. The two pink arrows are the same vector: king − man + woman ≈ queen.

cos(a,b) = a·b|a||b|

Similarity is an angle, not a distance. Directions encode relations: gender, tense, plurality, politeness.

This is also why search works: a query and a document can be compared without ever sharing a word.

3b — context-free so far. The vector for bank is one vector until attention splits it into river bank and bank loan.

FIG. 4 · The Core Loop · live computation

One token at a time, from logits to choice

SOFTMAX  /  vocab sliceT = 0.7
0.70
The cat sat on the
pi = ezi / T / ∑j ezj / T

Logits are raw scores, any real number. Softmax exponentiates them — that is what forces the outputs to be positive and sum to exactly 1.

Temperature divides before exponentiating. T → 0 collapses to argmax (safe, repetitive). T → 2 flattens toward uniform (surprising, sometimes wrong).

The entropy readout is the honest measure of how unsure the model is at this exact step.

FIG. 4 — drag the slider: every bar, percentage and bit below is recomputed by the browser, not pre-drawn.

FIG. 5 · Inside the Transformer · live arithmetic

Self-attention: every word asks, every word answers

keyq·k÷√dkesweight

5a — dk = 4, so √dk = 2. Click a query word above; the dot products, exponentials and weights are all recomputed.

A = softmax( QKT / √dk )
out = A · V

Each token produces three vectors: a query (what I am looking for), a key (what I offer) and a value (what I pass on if chosen).

The dot product is a compatibility score. Divide by √dk to keep variance from exploding as width grows.

{layers} × {heads} heads run in parallel and get concatenated — 64 heads × 128 dims per layer is typical.

5b — the residual stream: each block adds to the vector, never replaces it. That is why 100 layers stay trainable.
5c — causal mask: token i may only look at j ≤ i.

FIG. 6 · The Blind Spot · optional

Attention cannot see order

original  → 

shuffled  → 

pairweight, originalweight, shuffled

6a — same query, same keys, different positions: the weights are identical. Permute the sentence and the attention matrix just gets its rows reordered.

A bag of words would be a perfect model of language if order did not matter. It does, so position has to be injected by hand.

RoPE: rotate q, k by angle
θi = pos · 10000−2i/d

Modern models rotate the query and key by an angle proportional to position. Relative position then falls out of the dot product itself — which is why context windows can stretch to a million tokens without retraining the architecture.

6b — This is the single most overlooked line in every “how LLMs work” explainer. Skip this plate if you are short on time; keep it if the room is technical.

FIG. 7 · How It Learns

Three stages, one gradient

stage 1 · 99% of the computePre-train

Predict the next token on trillions of tokens. The text is its own label — no human annotation needed. L = -log p(target)

stage 2 · thousands of examplesFine-tune (SFT)

Supervised examples of good answers. Teaches format: how a helpful reply is shaped.

stage 3 · preference dataRLHF / DPO

Humans rank pairs of answers; a reward model learns the ranking; the policy is pushed toward preferred outputs.

loss = −log p(correct token) training steps → pre-train SFT RLHF power law, not a plateau

7a — loss falls as a power law in compute, parameters and data. That law, not a clever trick, is why scale works.

w ← w − η · ∂L/∂w

Backpropagation gives the gradient: for every one of the 552 billion weights, which direction makes the loss smaller. Learning rate η says how far to step.

7b — Batch of ~4M tokens per step, tens of thousands of steps, thousands of GPUs, weeks. One run: tens of millions of dollars.

FIG. 8 · The Consequence · optional

Hallucination is not a bug

It is the objective function doing exactly what we asked. We reward plausibility, then act surprised when we get plausible falsehoods.

what we train

“Which continuation is most likely in text like this?”

A confident sentence and a true sentence look identical to the loss. Nothing in the gradient rewards checking a fact.

what we want

“Which statement is true?”

Two different questions. Closing the gap takes retrieval, tools, and a model willing to say “I don’t know”.

Which is why the useful habit is dull and non-negotiable: treat it as a very well-read colleague who never checks a citation.

FIG. 8 — the same mechanism that lets it write poetry lets it invent a statute. One capability, two faces.

FIG. 9 · The Interface

An API is a waiter with no memory

Youyour app
WaiterHTTP / JSON
Kitchenthe model
{  "model": "deepseek-chat",  "messages": [    { "role": "system",    "content": "Be brief." },    { "role": "user",      "content": "Hello!" },    { "role": "assistant", "content": "Hi!" },    { "role": "user",      "content": "And you?" }  // whole history  ],  "temperature": 0.7, "stream": true}

Stateless. The server keeps nothing. Every call resends the entire conversation — that is why long chats get slower and more expensive, and why “context window” is a real limit and not a marketing number.

Metered by token. Input and output are billed separately. A 1,000-token system prompt is paid for on every single turn.

FIG. 9 — the model never leaves the kitchen. Your key says who is ordering.

FIG. 10 · Live Demo

Let’s call the API and watch it stream

One request, four ingredients. The reply arrives token by token — this is the loop from FIG. 4 running in production.

POST /chat/completionsAuthorization: Bearer sk-****{ "model": "deepseek-chat",  "messages": [{"role":"user","content":"Hello!"}],  "temperature": 0.7, "stream": true }
Open the chat page

what to point at on screen

→ the request panel: the exact JSON, key redacted

→ first token latency vs total time

→ output tokens ÷ seconds = the loop rate

→ drag temperature to 1.6 and ask again — same prompt, different answer, exactly as FIG. 4 predicts

FIG. 10 — the demo page prints its own measurements. Nothing is staged.

FIG. 10b · Demo Backup · skip unless offline

Recorded earlier

Hello! Introduce yourself in one sentence.
Hi! I am DeepSeek, an AI assistant, and I am happy to help you with questions, writing, and ideas.

Same API, same request — recorded before class in case the network fails.

FIG. 11 · The World Race · snapshot 2026-11-18

Who is best? Depends what you measure

FIG. 12 · Practice

Four tips that follow from the mechanism

1Be specific

“Write a poem.” → “Write a 4-line poem about the moon.”

→ narrows the distribution: fewer plausible tokens, less drift.

2Give it a role

“You are an English teacher. Correct my sentence.”

→ conditions every attention head on a different slice of the corpus.

3Show an example

“cat → kitten, dog → ?”

→ in-context learning: the pattern is read off the context, no weights change.

4Ask it to think first

“Work step by step, then give the answer.”

→ more tokens = more forward passes = more computation per answer.

FIG. 12 — none of these are tricks. Each one changes the input distribution the same loop is predicting from.

Colophon

Remember three things

Thank you!

Any questions?