The Hidden Crisis in AI Development: No One’s Testing

25/04/2025 37 min Episodio 38
The Hidden Crisis in AI Development: No One’s Testing

Listen "The Hidden Crisis in AI Development: No One’s Testing"

Episode Synopsis

In this episode, we sit down with Alon Bochman, a seasoned AI leader and founder of RagMetrics, whose career spans leadership roles at Google, Microsoft, FactSet, and a successful startup exit to Thomson Reuters. Alon breaks down the urgent gaps in AI testing today, why bilateral evaluation between humans and AI is the future, and shares a chilling real-world example of autonomous agents evolving faster than expected.We dive into why true AGI might be forever elusive (because we keep redefining it), the moral and economic need for universal basic income, and why today's "chatbot phase" is just the first flicker of a much larger AI revolution—akin to the dawn of electricity.A must-listen for anyone thinking seriously about building with AI, investing in AI, or simply understanding where this tech tidal wave is headed.Chapters00:00 – Introduction to Alon Bochman: Ex-Google, Ex-Microsoft, now Founder of RagMetrics00:50 – Journey from fintech to leading AI teams at major tech giants01:50 – The problem no one talks about: Why AI apps often skip real testing03:50 – The three bad choices in AI evaluation today06:12 – How bilateral feedback loops improve AI testing and model reliability08:18 – Human bias in feedback: Lessons learned from FactSet tech support automation13:43 – The importance of read/write knowledge bases to adapt with change15:43 – RagMetrics’ mission: Making AI evaluation scalable and less painful16:08 – Redefining AGI: Why we move the goalposts every time computers get better21:10 – AI’s impact on jobs: Why universal income may become a necessity25:29 – The electricity analogy: How AI will transform industries like a silent revolution27:54 – The scariest AI demo Alon ever saw: Agents that build their own tools32:36 – Using AI personally: Learning, demo generation, and accelerating workflows34:49 – Final advice to entrepreneurs: Just start, but test before you shipNotable Quotes“If you're not testing your AI today, you're building a bridge without running trucks over it.”“AI will never reach AGI—because AGI keeps getting redefined.”“Progress is unstoppable. The only question is whether you're the Roomba—or the person programming it.”“Testing is the bridge from hobby projects to real-world value.”“The scariest thing I saw? An agent learning to create its own tools in 25 lines of code.”Links & ResourcesAlon Bochman on LinkedInRagMetrics WebsiteTaran Agarwal on LinkedInSimplify Tech Website

More episodes of the podcast Simple Tech Talk