How Large Language Models Actually Work

The 60-second model

A large language model (LLM) is a giant function that, given a sequence of tokens, predicts the next token. Repeat that prediction step a few hundred times and you get a paragraph.

That's it. Everything else - chat, tools, reasoning, code - is built on top of "predict the next token."

Tokens, not words

Models read tokens, which are usually 3–4 characters in English.

• "prompt" → 1 token
• "Engineering" → 1–2 tokens
• "prompt-engineering" → 3–4 tokens (hyphens and casing matter)
• "私は学生です" → many tokens (Asian languages cost more)

You can inspect any prompt in OpenAI's tokenizer or with the tiktoken library.

How the model was trained (briefly)

1. Pre-training - predict the next token across trillions of tokens of internet text, code, and books.
2. Supervised fine-tuning (SFT) - humans write good answers; the model learns to imitate the answer style.
3. Reinforcement learning from human/AI feedback (RLHF / RLAIF / DPO) - humans rank model answers; the model learns to prefer the kind of answers humans (or constitutional rules) like.
4. Post-training for reasoning - newer models (o-series, Claude with extended thinking, Gemini Thinking) learn to use a private "scratchpad" before answering.

What that means for you as a prompt engineer

• The model has no memory between calls (unless you give it some).
• It's a probabilistic function - same prompt, different runs can give different output.
• It only knows what was in its training data + your prompt + tools you give it. It doesn't browse, remember, or "look things up" unless you wire that in.
• It's optimised to be plausible, not necessarily true. Hallucination is a structural feature, not a bug.

Activity

Take three short prompts (a question, a one-line task, and a paragraph). Paste each into the OpenAI tokenizer. Record the token count, then estimate cost at $5 per 1M input tokens. What does this tell you about how to write prompts cheaply?