S3 E14 OpenAI Research Scientist Noam Brown on Solving Poker and Diplomacy with AI

In this talk

1. 3:00

   Games as clear tests for AI

   Brown says games offer established benchmarks that are easy to score and compare across systems and against human experts. Beating top players gives researchers a clear signal of performance, though success in one game may not transfer to another.

2. 6:30

   Why poker needs mixed strategies and search

   Unlike chess and Go, poker hides information: players must reason about unknown cards and their opponents’ beliefs. Brown describes using regret minimization to approach Nash equilibrium strategies, which randomize actions so opponents cannot reliably exploit them. He says adding search dramatically improved poker performance, with river search producing a gain comparable to scaling training and the model by 100,000 times.

3. 51:35

   Diplomacy requires human data and player modeling

   Diplomacy combines simultaneous moves with private negotiations among seven players, so success depends on cooperation, communication, and anticipating human behavior. Brown’s team trained on two decades of online game records, then combined human imitation with self-play and search. Their bot placed in the top 10% of a human tournament over 40 games, and participants did not identify it as AI.

4. 1:11:40

   Scaling language models through inference-time planning

   Brown sees planning as a way to improve language model results by spending more computation at inference time instead of relying only on larger, more expensive pretraining. He points to tasks such as drafting contracts, writing novels, and discovering drugs, where slower and costlier answers may be worthwhile. Applying search is harder when, unlike in games, there is no clear final score to tell which path produced the better result.

5. 1:19:02

   A path from games to AI research

   Brown became interested in AI through programming games, then worked in algorithmic trading before pursuing computer science research. A poker AI project brought together his interests in game theory and building systems, and led to work on poker and Diplomacy agents.