Megastore: Providing Scalable, Highly Available Storage for Interactive Services

28/10/2024 18 min

Listen "Megastore: Providing Scalable, Highly Available Storage for Interactive Services"

Episode Synopsis

The provided text describes the architecture and design of Megastore, a Google-developed storage system designed to meet the needs of interactive online services. Megastore blends the scalability of NoSQL datastores with the convenience of traditional relational databases, offering high availability and strong consistency guarantees. It achieves this by utilizing Paxos, a fault-tolerant consensus algorithm, to synchronize data replication across geographically distributed data centers. Megastore also implements a partitioning strategy, dividing data into entity groups that are replicated separately, which allows for high throughput and localized outages. The authors detail the features of Megastore's data model, including ACID transactions, indexes, and queues, and explain how these features are implemented within the system. They also provide details about Megastore's replication algorithm and its performance characteristics, as well as insights into the development and operational experience with the system.

https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf