The Programmable Data Center — Evan Conrad, SF Compute

GPU compute is bought and sold like a commodity, but delivering it is a lot messier than trading a contract. Evan Conrad of SF Compute joins swyx at AI Engineer New York to explain the market they’re building—and why he thinks it needs an independent operator. We cover: • Why SF Compute had to build cloud operations to deliver the GPUs it trades • How interruptions, storage and hardware generations affect compute prices • Why batch and live inference need different contract terms • Physical versus cash settlement, and the risk of trading a price disconnected from actual GPUs • Why NVIDIA and the big clouds have different incentives from an independent market • How to turn a data center into a programmable GPU cluster Chapters: 0:00 Why can’t NVIDIA build this? 0:25 Welcome back to Latent Space 2:23 From selling a room to running the hotel 3:47 Physical settlement: actually delivering compute 6:00 Liquidity and the GPU order book 8:42 Trading nonstandard compute 10:00 Interrupting and moving workloads 12:20 Why hardware cohorts matter 14:32 How far out do compute contracts go? 16:19 Batch versus live inference leases 17:58 The shape of compute prices 20:33 GPU generations, specifications and SKUs 23:30 Physical versus cash settlement 27:00 The emerging compute market ecosystem 29:03 What it takes to deliver a GPU cluster 32:24 Sovereign compute 33:26 Why NVIDIA can’t build this market 34:48 A programmable data center SF Compute: sfcompute.com Latent Space: latent.space Sponsorship and business inquiries: business@latent.space