Hive is used for both interactive queries as well as part. The Hive variable substitution mechanism was designed to avoid some of the code that was getting baked into the scripting language on top of Hive. Examples such as:
are becoming commonplace. This is frustrating as Hive becomes closely coupled with scripting languages. The Hive startup time of a couple seconds is non-trivial when doing thousands of manipulations such as multiple
hive -e invocations.
Hive Variables combine the set capability you know and love with some limited yet powerful (evil laugh) substitution ability. For example:
For general information about Hive command line options, see Hive CLI.
There are three namespaces for variables – hiveconf, system, and env. The hiveconf variables are set as normal:
However they are retrieved using:
Annotated examples of usage from the test case ql/src/test/queries/clientpositive/set_processor_namespaces.q:
Disabling Variable Substitution
Variable substitution is on by default. If this causes an issue with an already existing script, disable it.