SummaryIn this episode, Apurva Mehta, co-founder and CEO of Responsive, recounts his extensive journey in stream processing—from his early work at LinkedIn and Confluent to his current venture at Responsive. He explains how stream processing evolved from simple event ingestion and graph indexing to powering complex, stateful applications such as search indexing, inventory management, and trade settlement. Apurva clarifies the often-misunderstood concept of “real time,” arguing that low latency (often in the one- to two-second range) is more accurate for many applications than the instantaneous response many assume. He delves into the challenges of state management, discussing the limitations of embedded state stores like RocksDB and traditional databases (e.g., Postgres) when faced with high update rates and complex transactional requirements. The conversation also covers the trade-offs between SQL-based streaming interfaces and more flexible APIs, and how Responsive is innovating by decoupling state from compute—leveraging remote state solutions built on object stores (like S3) with specialized systems such as SlateDB—to improve elasticity, cost efficiency, and operational simplicity in mission-critical applications.Chapters00:00 Introduction to Apurva Mehta and Streaming Background08:50 Defining Real-Time in Streaming Contexts14:18 Challenges of Stateful Stream Processing19:50 Comparing Streaming Processing with Traditional Databases26:38 Product Perspectives on Streaming vs Analytical Systems31:10 Operational Rigor and Business Opportunities38:31 Developers' Needs: Beyond SQL45:53 Simplifying Infrastructure: The Cost of Complexity51:03 The Future of Streaming ApplicationsClick here to view the episode transcript.

Ver todos los episodios