Helen Toner, former OpenAI board member who voted to fire Sam Altman, discusses the fundamental tension between AI safety commitments and commercial incentives in frontier AI labs. She argues that pinning our hopes on any single person being trustworthy is a dangerous strategy, and advocates for stronger institutional structures, transparency, and risk management standards rather than relying on CEO trustworthiness.
Summarized by Podsumo
Helen Toner argues that trusting AI CEOs is not a viable safety strategy; we need institutional safeguards and accountability mechanisms.
She explains the 'AGI pilled' mindset that drives frontier labs like OpenAI and Anthropic to pursue superintelligence, often at the expense of safety.
Toner discusses the recent demonstrations of misuse and misalignment risks, including the Mythos and Hugging Face incidents, as real-world warnings.
She notes that state-level AI regulation is gaining momentum, with California, New York, and Illinois passing coordinated bills focused on frontier risks.
Toner highlights the challenge of US-China AI competition, suggesting that shared threat perception could lead to cooperation on safety, but trust is extremely low.
"If our hopes for the future depend on any one person being trustworthy, we are in big trouble."
— Helen Toner
"We saw these two big demonstrations of the two major concerns people have: misuse and misalignment. Mythos and the Hugging Face attack are just really vivid and crisp real-world demonstrations."
— Helen Toner
"The concept of AGI is kind of a fuzzy cloud. We're now inside that cloud, having passed some definitions, far from others."
— Helen Toner