Jeff Huber, Founder & CEO of Chroma, joins Jay to discuss why context engineering remains the core job of AI engineers, and how modern search infrastructure is evolving for AI-native applications. Jeff brings deep experience building Chroma into the leading AI application database, serving thousands of production AI agents with advanced search capabilities across code, dependencies, and unstructured data. The conversation centers around the reality of context rot in long-context models, why RegX vs semantic search debates miss the point, and how memory systems need to move beyond simple retrieval to enable true agent learning. Jeff explains why Chroma has indexed all major open source dependencies across NPM, PyPI, Cargo, and Go, enabling agents to search exact package versions instead of hallucinating APIs. Tune into the full episode to learn why context engineering remains the bottleneck for AI reliability and how search infrastructure will evolve beyond simple vector similarity!HIGHLIGHTS: 0:00 Intro2:24 AI databases vs traditional databases4:39 Context engineering as core AI developer job6:21 Context rot research - million-token degradation8:32 Long context performance vs marketing claims12:15 Prior failures boost agent performance, successes hurt it16:36 LLMs as query planners inside databases19:15 Why coding became first dominant AI use case21:06 RegX vs semantic search propaganda wars24:21 Language servers on crack for code search28:03 Multi-branch agent coordination30:09 Code Collections - searching NPM/PyPI packages31:21 Forkable collections enable 100ms Git indexing34:09 Deep research agents are "incredibly mid"38:17 Memory as context engineering vs weights40:36 Agent task learning vs user personalization43:18 Auditability problem with model-weight memory47:21 Agents need apprenticeship models for reliability49:54 Embarrassing lack of AI UX innovationConnect with Jeff - https://www.linkedin.com/in/jeffchuber/Connect with Jay - https://www.linkedin.com/in/jayhack/ or https://x.com/mathemagic1an Visit trychroma.com for AI application database infrastructure

Ver todos los episodios