S3 Ep 13 Stephen Balaban from Lambda on building the most cost-effective AI cloud

In this talk

1. 2:27

   Making GPU compute cheaper

   Balaban describes Lambda as an AI infrastructure provider offering cloud GPUs as well as workstations, servers, and on-premise clusters. He says the company aims to lower prices by increasing GPU utilization and supply, and cites a flat H100 price of $1.89 per hour. Lambda also uses published prices for customers of every size, including hyperscalers.

2. 16:22

   Building Lambda around deep learning

   Balaban says AlexNet’s 2012 ImageNet results helped convince him that deep learning would reshape computing. Lambda began as a face-recognition API, with its infrastructure business emerging gradually as the team built systems for its own work. By 2016 or 2017, he says, enough people had adopted deep learning to support a business providing infrastructure to others.

3. 20:50

   Scaling GPU supply

   Balaban says Lambda had grown 60 times in the previous 12 months and planned to deploy more H100 clusters with 3,200 Gbps InfiniBand. He describes GPU availability as a persistent challenge and says deployments follow a queue, with GPUs coming online months after contracts are signed. Lambda’s goal is to compete on price, scale, interconnect speed, and time to launch.

4. 27:45

   Supporting open models and generative AI

   Balaban identifies large language models, diffusion, and image, audio, and video generation as heavy workloads. He argues that commercially usable pretrained models could reduce repeated training and wasted compute, and says Lambda was selecting a team to train an open model with company-provided H100 resources. The company also hosted generative AI demos, including Stable Diffusion projects, with an A10 priced at 60 cents per hour.

5. 35:44

   Imagining AI-powered games

   Balaban expects older GPUs to remain useful for smaller models and for models made more efficient through distillation or quantization. He also describes games with neural-generated worlds and NPCs that respond to open-ended conversation. He imagines these experiences becoming less compute-intensive over time, eventually running on a single GPU.