Sheng Ying on xAI Infra, SGLang, and Open Source

Why it matters

Sheng Ying is the former head of xAI's inference team, creator of the open-source inference engine SGLang, and co-founder/CEO of RadixArk. She describes AI infrastructure as 'a romantic relationship' — product itself that demands aesthetics. This episode covers why infra is AI's romance, the birth of SGLang, xAI's engineering culture, how inference engines shape model capability, and what 'equal access' means for the industry.

Key takeaways

• Infra is a 'romantic relationship': AI infrastructure is not a sidekick but the product itself, demanding aesthetics and craftsmanship.
• The birth of SGLang: from a researcher's obsession with inference efficiency to core infrastructure for xAI and the open-source community.
• xAI engineering culture: small teams, extremely fast iteration, where the boundary between engineering and research is redrawn at the infra layer.
• Inference engines set the ceiling: the same weights behave differently depending on the serving system.
• Open source as equal access: open infra lets more teams afford frontier models — a key to democratizing AI.