● Apache Kafka, Flink and Pinot: Open Source Powering Uber's Real-Time Data Stack

15/10/2024 11 min Temporada 1 Episodio 14

Listen "● Apache Kafka, Flink and Pinot: Open Source Powering Uber's Real-Time Data Stack"

Episode Synopsis

Building and scaling a real-time data infrastructure is a complex undertaking, fraught with challenges and valuable lessons. This episode takes a deep dive into Uber's journey, exploring the hurdles they encountered while managing petabytes of real-time data. We'll discuss the need for data consistency, availability, and freshness, the complexities of handling diverse use cases and user groups, and the constant need for system evolution. Tune in to learn from Uber's experiences and gain insights into building robust and scalable real-time data infrastructures.

References: https://arxiv.org/pdf/2104.00087