Intro to Large Language Models — the one-hour talk

Why it matters

Karpathy's first public lecture after leaving Tesla: in one hour he covers what LLMs are, how they're trained, and where they're heading. No math required, yet he explains 'the model as a lossy compression of the internet', how next-token prediction grows intelligence, what RLHF does, and why tool use is the direction. Tens of millions of views and the source text for countless explainers — hand this to any non-technical colleague.

Key takeaways

• An LLM is a lossy compression of internet text into weights, replayed by predicting the next token.
• Two training stages: pretraining yields the base model; fine-tuning (SFT + RLHF) turns it into an assistant.
• Scaling laws: performance improves smoothly with compute/data/parameters — and is predictable in advance.
• The LLM is the kernel of a new OS: the context window is memory, tools (browser/code/retrieval) are peripherals.
• Security matters early: prompt injection, jailbreaks and data poisoning are real attack surfaces.