Child pages
  • Configuration Properties

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

hive.spark.optimize.shuffle.serde
  • Default Value: false
  • Added In: Hive 3.0.0 with HIVE-15104

If this is set to true, Hive on Spark will register custom serializers for data types in shuffle. This should result in less shuffled data.

Remote Spark Driver

The remote Spark driver is the application launched in the Spark cluster, that submits the actual Spark job. It was introduced in HIVE-8528. It is a long-lived application initialized upon the first query of the current user, running until the user's session is closed. The following properties control the remote communication between the remote Spark driver and the Hive client that spawns it.

...