All current wiki content has been merged into pages at http://spark.apache.org as of November 2016. Each page links to the new location of its information on the Spark web site.
Obsolete wiki content is still hosted here, but carries a notice that it is no longer current.
In particular, release cadence information, formerly on this page, is now hosted at http://spark.apache.org/versioning-policy.html
Welcome to the Apache Spark wiki. This wiki is for pages that change frequently or benefit from being easier to edit than pages on the Spark website. Visit the Apache Spark website to find Spark documentation, introductory material, and more.
If you are interested in contributing to Spark, visit the Contributing to Spark page to learn more.
Current Release Window
In Spark 1.x, the project made a new feature release once every 3 months, i.e. Spark 1.6 was released 3 months after Spark 1.5. Starting Spark 2.x, the release cycle has been increased from 3 months to 4 months.
Spark 2.1 release window is the following:
| Date | Event |
|---|---|
| ~ Nov 1 2016 | Code freeze. Release branch cut. |
| Early Nov 2016 | QA period. Focus on bug fixes, tests, stability and docs. Generally, no new features merged. |
| Late Nov 2016 | Release candidates (RC), voting, etc. until final release passes |
Spark 2.2 release window is the following:
| Date | Event |
|---|---|
| ~ Mar 1 2017 | Code freeze. Release branch cut. |
| Early Mar 2017 | QA period. Focus on bug fixes, tests, stability and docs. Generally, no new features merged. |
| Late Mar 2017 | Release candidates (RC), voting, etc. until final release passes |
Spark Developer Documentation
- Contributing to Spark - Introduction to Spark development
- Useful Developer Tools - Tips and tricks for developers
- Spark Versioning Policy - An explanation of how Spark versioning works
- Spark Internals - Overview of internal architecture
- Profiling Spark Applications - Using YourKit to profile Spark
Spark Committer Documentation
- Preparing Spark Releases - Guide for release managers
Spark Related Projects
...