wunder beta🤖 How Large Language Models Work
Inside the machines that learned language by predicting the next word
What you’ll learn
- Explain how text becomes numbers, the next-token-prediction objective, the transformer, and parameters.LLMs tokenize text into number vectors and are trained on next-token prediction; the 2017 transformer architecture let them scale to billions of parameters that hold distributed knowledge.
- Describe attention, stacked layers, autoregressive generation, and the role of the context window.Attention uses query, key, and value vectors so each token weighs others; stacked layers build meaning, text is generated one token at a time, and the context window acts as the model's working memory.
- Walk through pretraining, post-training alignment, hallucination, and a realistic view of the technology.Models are pretrained on vast text, then aligned via supervised fine-tuning and RLHF into helpful assistants; they can hallucinate because they optimize plausibility over truth, so outputs need verification.
Questions this course answers
What is a token in a language model?
A token is a unit of text, which may be a whole word or a fragment, that the model converts into numbers to process.
What core task are large language models trained to perform?
LLMs are trained on next-token prediction, learning to estimate the most likely next token given the preceding text.
In what year was the transformer architecture introduced?
The transformer was introduced in 2017 in the paper Attention Is All You Need, and underpins modern LLMs.
In the attention mechanism, what three vectors does each token produce?
Attention transforms each token into query, key, and value vectors to decide which tokens matter most for one another.
What does the attention mechanism let the model do?
Attention lets each token dynamically weigh the importance of other tokens, capturing context and long-range relationships.
What is the context window of a language model?
The context window is the span of text the model can attend to at once; anything outside it is effectively forgotten.
Grounded in trusted sources
- Vaswani et al., "Attention Is All You Need," 2017 (arxiv.org/abs/1706.03762)
- Anyone AI, "The Transformer Architecture: Deconstructing How Large Language Models Work" (anyoneai.com)
- RBC Borealis, "Training and fine-tuning large language models" (rbcborealis.com)
- "A Survey on Hallucination in Large Language Models," arXiv:2311.05232
- Roberto Infante, "An intuitive overview of the Transformer architecture," Medium
Every Wunder lesson is built from real, reputable sources — never invented.
Related Science courses
Wunder is a personalized learn-anything platform — tell it any topic and it builds a beautiful, fact-checked course in minutes, with narration, a knowledge check, and a college-style University track.
Browse more Science courses · All topics · Home
© 2026 Wunder Learning LLC · Terms & Privacy