This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • PySpark Internals

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

NOTE: This Wiki is obsolete as of November 2016 and is retained for reference only.


Overview

PySpark is built on top of Spark's Java API. Data is processed in Python and cached / shuffled in the JVM:

...