Deep Dive into LLMs like ChatGPT
Tip: use the player's CC button to enable or switch subtitles; English captions are available on these videos.
| Item | Details |
|---|---|
| Speaker | Andrej Karpathy |
| Channel | Andrej Karpathy |
| Date | Feb 5, 2025 |
| Duration | 3:31:23 |
| Format | Video |
| Topics | #llm · #engineering · #open-source |
Why it matters
The 2025 'full edition': over three and a half hours covering the entire pipeline behind ChatGPT — data, tokenization, pretraining, post-training, inference, tool use, hallucinations and mitigations. A full level deeper than the 2023 one-hour talk.
Key takeaways
- The full pipeline: raw internet data → tokenization → pretraining → supervised fine-tuning → RL post-training.
- Hallucination is a feature, not a bug: models always produce the statistically most plausible answer.
- Reasoning models think before answering, with RL rewarding good chains of thought.
- Capabilities are 'Swiss cheese': hard problems solved, easy ones fumbled.
- Practical ChatGPT: treat context as memory — start a fresh conversation when you should.
Original video
- Speaker
- Andrej Karpathy
- Channel
- Andrej Karpathy
- Venue
- A 3.5-hour deep dive on his own channel
- Date · Duration
- Feb 5, 2025 · 3:31:23
Deep Dive into LLMs like ChatGPT
Watch the original on YouTubeSources & Further Reading
This page is grounded in the authoritative sources below — verifiable and citable by AI engines and readers.
- 👤 Person: Andrej Karpathy
- # Topic: LLMsLLM capabilities, products and best practices
- # Topic: EngineeringAPIs, prompting and system design
- # Topic: Open SourceOpen models, frameworks and ecosystems
Citation: Please attribute Laojin Global (laojinchuhai.com) and keep the original link.
Made by Laojin · AI that ships
AllModelsAPIOne key for many models
AllModelsAPI is a multi-model API gateway: one key reaches many models through an OpenAI-compatible interface — point your existing code at a new base URL and you're migrated. It's not a demo: our own production workloads run on it every day.
More from Laojin: Sellenca · 365AIOrg · 365Loopa · 365 Ops · 365Skill
Related
Linked by topic, people and hubs