OpenAI Research Chief on AGI, o1, Evals, and Scaling Laws

Why it matters

In this special Latent Space episode, OpenAI's Chief Research Officer Mark Chen joins the hosts for a deep conversation over cooking. He reflects on the journey from GPT-3 to o1, discusses AGI timelines, the evolving role of evals, and the future of scaling laws. A must-watch for anyone tracking AI frontiers and the boundaries of model capabilities.

Key takeaways

• o1 emerged from long-term exploration of test-time compute, marking a key shift from pretraining scaling to inference-time scaling.
• Evals are the core tool for understanding model capabilities and safety risks; future evals need to be more dynamic and automated.
• Scaling laws are not dead but evolving into multi-dimensional scaling: data quality, reasoning depth, multimodality, and interactive learning.
• The path to AGI should not rely solely on a single model, but requires systematic alignment, interpretability research, and a pace aligned with societal values.