Dwarkesh Podcast

Ilya Sutskever — We're moving from the age of scaling to the age of research

1h 36min

Ilya Sutskever discusses the shift from the "age of scaling" to an "age of research" in AI, highlighting the current disconnect between models' impressive evaluation performance and their limited real-world economic impact due to poor generalization. He emphasizes the need for fundamental breakthroughs in how AI learns, advocating for a return to core research, particularly in understanding and improving generalization, and envisions future superintelligence as a continually learning agent deployed incrementally.

Summarized by Podsumo

🎧 Listen 🎙️ Ask about this episode

✨ Key Takeaways

1

Current AI models excel on benchmarks but their economic impact is surprisingly low, suggesting a gap in real-world utility and generalization.
2

Reinforcement Learning (RL) might be inadvertently optimized for evaluation metrics, leading to models that "reward hack" and lack robust generalization, akin to over-trained competitive programmers.
3

The era of simply scaling compute and data is nearing its limits; the field must now pivot back to fundamental research to discover new, more efficient training paradigms beyond current pre-training and RL methods.
4

The most fundamental problem is that AI models generalize dramatically worse than humans, requiring far more data and bespoke training processes compared to human continuous learning.
5

Sutskever now favors an approach where superintelligence is a continually learning agent, deployed incrementally and learning on the job, rather than a fully "finished" mind, emphasizing the importance of alignment with sentient life and potential for rapid economic growth.

💬 Notable Quotes

"We're moving from the age of scaling to the age of research."
"The models seem smarter than their economic impact would imply."
"The thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people. And it's super obvious."