NOTE: This Wiki is obsolete as of November 2016 and is retained for reference only.

This section contains documentation on Spark's internals:

Spark's Cluster Mode Overview documentation has good descriptions of the various components involved in task scheduling and execution.

The Intro to Spark Internals Meetup talk (Video, PPT slides) is also a good introduction to the internals (the talk is from December 2012, so a few details might have changed since then, but the basics should be the same).

  • No labels