Project SherlockEst. Knowledge

Paper

2004

MapReduce: Simplified Data Processing on Large Clusters

Jeffrey Dean & Sanjay Ghemawat

Shows that a huge class of data-processing jobs can be expressed as two functions, letting the runtime handle parallelism, failure and data placement.

Before you start

FreeIntermediatelink checked 17 Sept 2026

Read these first

Not a difficulty warning — a route. These are in the library too.

The Google File SystemSanjay Ghemawat, Howard Gobioff & Shun-Tak Leung, 2003Argues that component failure should be treated as the normal case rather than the exception, and designs a file system around relaxed consistency and chunk-level replication instead of POSIX semantics.

Groundwork for

Works in the library that name this one as a prerequisite.

MillWheel: Fault-Tolerant Stream Processing at Internet ScaleTyler Akidau et al., 2013Shows exactly-once processing over out-of-order streams is achievable at scale by tracking a low watermark per record rather than relying on synchronised clocks.

Filed under Distributed Systems in Computer Science.