Next 100x in AI: Inference, Networking, & Self-Optimizing Models

Why it matters

In this episode, Baseten's Philip Kiely and Ali Taha dive into the next decade of AI infrastructure. They argue that inference efficiency, networking, and self-optimizing models will be the key engines driving 100x growth in AI. From GPU cluster scheduling to model self-tuning, this conversation offers forward-looking technical insights and practical lessons for AI engineers and founders.

Key takeaways

• Inference cost and latency are the biggest bottlenecks for AI at scale; optimizing inference systems is more urgent than training larger models.
• Networking technologies like RDMA and intelligent routing are becoming the new battleground for distributed training and inference performance.
• Self-optimizing models that adjust weights and prompts via feedback loops will drastically reduce manual tuning overhead.
• The moat for future AI companies lies in deep infrastructure expertise and automated operations.