How do you improve AI products when your users number in the hundreds of millions—and soon billions?

In this session, Stuart from OpenAI shares how the ChatGPT team built a system to transform fragmented user feedback into actionable product improvements. Learn how OpenAI aggregates signals from support tickets, thumbs up/down ratings, social media, interviews, and product interactions to identify issues, prioritize fixes, and continuously improve models and user experiences at scale.

The talk walks through OpenAI’s end-to-end feedback infrastructure, including shared taxonomies, clustering systems, implicit feedback extraction, agent-powered workflows, and automated reporting pipelines. You’ll see how OpenAI uses AI agents to surface customer pain points, investigate root causes, generate reports, and even assist in resolving issues across products like ChatGPT.

Key Takeaways
• User feedback is often fragmented across support channels, social media, ratings, interviews, and product interactions, making unified analysis essential.
• Explicit feedback captures only a small portion of user experiences; implicit signals from user behavior can dramatically increase actionable feedback volume.
• Shared taxonomies and clustering systems help teams identify trends, prioritize issues, and discover emerging problems before they become widespread.
• AI agents can automate large portions of the feedback loop, from issue discovery and root cause analysis to reporting and remediation.
• The most effective AI product teams build systems that continuously convert user feedback into measurable product improvements.

#OpenAI #ChatGPT #AIAgents #UserFeedback #ProductDevelopment #AIEngineering #CustomerExperience #LLMOps #AIObservability #ArizeObserve

🔗 Try Arize AX & Phoenix OSS: arize.com
🔔 Subscribe for weekly content on LLMs, agents, and evaluation: https://www.youtube.com/@arizeai?sub_confirmation=1