...
- As Hive on Spark is still in development, currently only a Hive assembly built from hive/spark development branch supports Spark execution. The development branch is located here: https://github.com/apache/hive/tree/spark. Checkout branch and build hive assembly as described in https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ.
Start hive with <spark-assembly-*.jar> on the hive auxpath:
Code Block hive --auxpath /location/to/spark<spark-assembly-spark_version-hadoop_version.jar*.jar>
Configure hive execution to Spark:
Code Block hive> set hive.execution.engine=spark;
Configure Spark-application configs for Hive. See: http://spark.apache.org/docs/latest/configuration.html. This can be done either by adding a file "spark-defaults.conf" with these properties to the hive classpath, or by setting them on hive configuration:
Code Block hive> set spark.master=<Spark Master URL> hive> set spark.eventLog.enabled=true; hive> set spark.executor.memory=512m; hive> set spark.serializer=org.apache.spark.serializer.KryoSerializer;
...