MapReduce banner

Overview

Period: April 2019
Languages: Go
Description:
This project consisted of two steps: (1) writing a simple MapReduce program that when handed as input a body of text, counts the number of occurrences of each word in alphabetical order, and (2) completing a version of MapReduce that splits the work up over a set of worker threads (in order to exploit multiple cores by having a master thread hand out work to the worker threads and waiting for them to finish). The master communicates with the workers via RPC, and the workers register with the master via a Register RPC. In addition, the master handles failed workers by re-assigning jobs given to failed workers to another worker. This project doesn't involve failures of the master (which is handled later in the next project).


GitHub Source Code