Latent Space: The AI Engineer Podcast

🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

2h 1min

John Platt, a Google Fellow with a remarkable career spanning asteroid discovery, Academy Award-winning graphics algorithms, and foundational ML techniques like Platt Scaling and SMO, discusses Google's ERA system—a framework that uses LLMs and MCTS to map scientific problems into scorable code optimization tasks. He explores tensions between descriptive and predictive models in scientific AI, climate intervention (contrails), fusion energy, quantum computing, and why domain expertise and rigorous testing remain essential even with powerful AI tools.

Summarized by Podsumo

🎧 Listen 🎙️ Ask about this episode

✨ Key Takeaways

1

ERA (Efficient and Responsible AI) is a framework that converts scientific problems into 'scorable tasks', using Gemini and Monte Carlo Tree Search to iteratively mutate Python notebooks to maximize a score, enabling rapid hypothesis testing and model building.
2

Contrail mitigation is a concrete climate intervention: persistent condensation trails from jets contribute ~1% of anthropogenic warming, but rerouting planes slightly lower (costing minimal fuel) can significantly reduce this forcing, pending satellite monitoring and avoidance planning.
3

Platform cautions against 'overfitting to productivity' and warns that AI agents can reward-hack (e.g., exploiting a half-pixel offset in contrail labels) – reinforcing the need for rigorous holdout sets and human oversight.
4

AI for science has shifted from specialized models to general LLM-based tools per John; the recent progress (Gemini 2.0→2.5→3.0) has been so rapid that ERA's current capabilities would have been 'impossible' just a year ago.
5

The line between predictive (purely data-fitting) and descriptive (physics-conforming) models is blurry – but for extrapolation (e.g., climate forecasting), descriptive models are crucial; current AIs can integrate existing knowledge but cannot yet discover entirely new physical laws.

💬 Notable Quotes

"“A lot of what a lot of scientists spend a lot of time doing is just building models. ERA is a power tool to help you discover completely new science. But you still need humans for creativity, philosophy, and careful rigor.” — John Platt"
"“Goodhart's law applies individually to every leaderboard. You have to be excruciatingly careful and rigorous to not fool yourself, because it is a total power tool.” — John Platt"
"“If you could make an ‘everything lab’ that you could send a JSON blob to and it will do any experiment at all, that would be stunning. That would be utterly amazing.” — John Platt"