Overview
Period: April 2019
Languages: Go
Description:
This project was the capstone project of the distributed systems course I took at UCLA that consisted of implementing a key/value service that supports three RPCs: Put(k, v), Append(k, arg), and Get(k); and maintains a simple database of key/value pairs. The key/value service was implemented in three different ways with each handling different degrees of fault tolerance and performance:
- Primary/Backup replication, assisted by a view service that decides which machines are alive. The view service allows the P/B service to work correctly in the presence of network partitions. However, the view service itself is not replicated, and is a single point of failure.
- Paxos protocol to replicate the k/v database with no single point of failure, and handles network partitions correctly. This k/v service is slower than a non-replicated k/v server would be, but is fault tolerant.
- Shared k/v database, where each shard replicates its state using Paxos. This k/v service can perform put/get ops in parallel on different shards, allowing it to support applications such as MapReduce that can put a high load on a storage system. Additionally, this service has a replicated configuration service, which tells the shards the key range they are responsible for, and allows a change of the key-shard assignments; for example, in response to changing load. This final implementation has the core of a real-world design for thousands of servers.
