DataFramed

#367 Don't Build on Jell-O: How to Make Agentic AI Reliable with Dan Klein, CTO at Scaled Cognition

52 min

Dan Klein discusses the critical reliability challenges facing agentic AI, particularly the problem of hallucinations that are indistinguishable from correct answers. He argues that current LLMs are plausibility engines, not truth engines, and advocates for building models with reliability as a core design goal rather than retrofitting guardrails. Klein introduces Scale Cognition's approach of action models that operate over semantic objects to provide verifiable, trustworthy AI interactions.

Summarized by Podsumo

🎧 Listen 🎙️ Ask about this episode

✨ Key Takeaways

1

The worst thing about AI is lies that have no tells, making hallucinations indistinguishable from correct answers
2

Current LLMs are plausibility engines that generate fluent but potentially false outputs without metacognitive awareness
3

Scale Cognition's models use structured action-based architectures instead of token-based generation for verifiable outputs
4

Reliability-first design leads to smaller, faster, and cheaper models without needing second-model verification layers
5

Enterprises often underestimate hallucination rates by 5x or more because most subtle errors go unnoticed
6

Test-driven development with continuous monitoring is essential for deploying trustworthy AI systems

💬 Notable Quotes

"My mission personally is to build systems you can trust, systems you can control, systems that you can actually start to say things about what they will and, sometimes more importantly, will never do."

— Dan Klein
"If you think about like, okay, so I teach computer science classes, the single most important lesson we teach about how we build large systems ... is modularity."

— Dan Klein
"What happens to society when we have an oracle that always sounds true, but sometimes it is and sometimes it isn't, and isn't very good at crediting its sources?"

— Dan Klein