Building agents that work in the real world: Live from SXSW

In this talk

1. 3:43

   Start with durable customer problems

   Amanda Door describes using generative AI to summarize large volumes of product reviews and power Rufus, Amazon’s conversational shopping assistant. At first, customers needed suggested questions to see what Rufus could do; as they gained experience with AI tools, multi-turn conversations increased. Michael GeneAngeli says teams need to stay nimble as technology changes, while holding on to long-term customer needs.

2. 13:17

   Build reusable parts around real needs

   The panel emphasizes talking with customers and starting from problems they already have. Door describes building Shop Direct and Buy for Me from reusable components such as a product catalog, order creation, and data reasoning, so teams could adapt how those pieces fit together as the technology evolved.

3. 17:16

   Measure agents and match the task

   GeneAngeli recommends defining success with evaluations tailored to a customer’s workflow, then comparing models on accuracy, cost, and latency. He describes Nova’s Reinforcement Learning Gym, with hundreds of practice environments where models learn through repeated attempts. Door says purchases call for dependable methods: AI can find familiar items and prepare a cart, while established code handles the transaction.

4. 23:09

   Keep people involved where it matters

   Michael GeneAngeli describes using Nova Act to check event details in the same internal tools as the support team. The agent explains its recommendations in Slack, while people review uncertain cases and provide feedback. Panelists say the right level of delegation depends on risk and personal preference: customers may automate routine purchases, while choices such as concert tickets or expensive products still call for human judgment.

5. 41:24

   Context and expertise remain important

   GeneAngeli says models lack much of the situational context people use, including tone, facial expressions, and what is happening around them; memory and context management may help. Door notes that shopping decisions can be emotional even when they seem logical, and says teams are still learning from customer behavior. The panel also describes AI as helping colleagues prototype ideas and communicate across disciplines while deep expertise remains valuable.