...
hive.spark.optimize.shuffle.serde
- Default Value:
false - Added In: Hive 3.0.0 with HIVE-15104
If this is set to true, Hive on Spark will register custom serializers for data types in shuffle. This should result in less shuffled data.
Remote Spark Driver
The remote Spark driver is the application launched in the Spark cluster, that submits the actual Spark job. It was introduced in HIVE-8528. It is a long-lived application initialized upon the first query of the current user, running until the user's session is closed. The following properties control the remote communication between the remote Spark driver and the Hive client that spawns it.
...