...
| No Format |
|---|
% bin/beeline Hive version 0.11.0-SNAPSHOT by Apache beeline> !connect jdbc:hive2://localhost:10000 scott tiger !connect jdbc:hive2://localhost:10000 scott tiger Connecting to jdbc:hive2://localhost:10000 Connected to: Hive (version 0.10.0) Driver: Hive (version 0.10.0-SNAPSHOT) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://localhost:10000> show tables; show tables; +-------------------+ | tab_name | +-------------------+ | primitives | | src | | src1 | | src_json | | src_sequencefile | | src_thrift | | srcbucket | | srcbucket2 | | srcpart | +-------------------+ 9 rows selected (1.079 seconds) |
You can also specify the connection parameters on command line. This means you can find the command with the connection string from your UNIX shell history.
...
Command | Description |
|---|---|
reset | Resets the configuration to the default values. |
| reset <key> | Resets the value of a particular configuration variable (key) to the default value. |
set <key>=<value> | Sets the value of a particular configuration variable (key). |
set | Prints a list of configuration variables that are overridden by the user or Hive. |
set -v | Prints all Hadoop and Hive configuration variables. |
add FILE[S] <filepath> <filepath>* | Adds one or more files, jars, or archives to the list of resources in the distributed cache. See Hive Resources for more information. |
| add FILE[S] <ivyurl> <ivyurl>* add JAR[S] <ivyurl> <ivyurl>* add ARCHIVE[S] <ivyurl> <ivyurl>* | As of Hive 1.2.0, adds one or more files, jars or archives to the list of resources in the distributed cache using an Ivy URL of the form ivy://group:module:version?query_string. See Hive Resources for more information. |
list FILE[S] | Lists the resources already added to the distributed cache. See Hive Resources for more information. (As of Hive 0.14.0: HIVE-7592). |
list FILE[S] <filepath>* | Checks whether the given resources are already added to the distributed cache or not. See Hive Resources for more information. |
delete FILE[S] <filepath>* | Removes the resource(s) from the distributed cache. |
| delete FILE[S] <ivyurl> <ivyurl>* delete JAR[S] <ivyurl> <ivyurl>* delete ARCHIVE[S] <ivyurl> <ivyurl>* | As of Hive 1.2.0, removes the resource(s) which were added using the <ivyurl> from the distributed cache. See Hive Resources for more information. |
| reload | As of Hive 0.14.0, makes HiveServer2 aware of any jar changes in the path specified by the configuration parameter hive.reloadable.aux.jars.path (without needing to restart HiveServer2). The changes can be adding, removing, or updating jar files. |
dfs <dfs command> | Executes a dfs command. |
<query string> | Executes a Hive query and prints results to standard output. |
...
Option | Description |
|---|---|
-u <database URL> | The JDBC URL to connect to. Special characters in parameter values should be encoded with URL encoding if needed. Usage: |
-r | Reconnect to last used URL (if a user has previously used Usage: Version: 2.1.0 (HIVE-13670) |
-n <username> | The username to connect as. Usage: |
-p <password> | The password to connect as. Usage: Optional password mode: Starting Hive 2.2.0 (HIVE-13589) the argument for -p option is optional. Usage : beeline -p [valid_password] If the password is not provided after -p BeeLine Beeline will prompt for the password while initiating the connection. When password is provided BeeLine Beeline uses it initiate the connection without prompting. |
-d <driver class> | The driver class to use. Usage: |
-e <query> | Query that should be executed. Double or single quotes enclose the query string. This option can be specified multiple times. Usage: Support to run multiple SQL statements separated by semicolons in a single query_string: 1.2.0 (HIVE-9877) |
| -f <file> | Script file that should be executed. Usage: Version: 0.12.0 (HIVE-4268) |
| -i (or) --init <file or files> | The init files for initialization Usage: Single file: Version: 0.14.0 (HIVE-6561) Multiple files: Version: 2.1.0 (HIVE-11336) |
| -w (or) --password-file <password file> | The password file to read password from. Version: 1.2.0 (HIVE-7175) |
| -a (or) --authType <auth type> | The authentication type passed to the jdbc as an auth property Version: 0.13.0 (HIVE-5155) |
| --property-file <file> | File to read configuration properties from Usage: Version: 2.2.0 (HIVE-13964) |
| --hiveconf property=value | Use value for the given configuration property. Properties that are listed in hive.conf.restricted.list cannot be reset with hiveconf (see Restricted List and Whitelist). Usage: Version: 0.13.0 (HIVE-6173) |
| --hivevar name=value | Hive variable name and value. This is a Hive-specific setting in which variables can be set at the session level and referenced in Hive commands or queries. Usage: |
| --color=[true/false] | Control whether color is used for display. Default is false. Usage: (Not supported for Separated-Value Output formats. See HIVE-9770) |
| --showHeader=[true/false] | Show column names in query results (true) or not (false). Default is true. Usage: |
| --headerInterval=ROWS | The interval for redisplaying column headers, in number of rows, when outputformat is table. Default is 100. Usage: (Not supported for Separated-Value Output formats. See HIVE-9770) |
| --fastConnect=[true/false] | When connecting, skip building a list of all tables and columns for tab-completion of HiveQL statements (true) or build the list (false). Default is true. Usage: |
| --autoCommit=[true/false] | Enable/disable automatic transaction commit. Default is false. Usage: |
| --verbose=[true/false] | Show verbose error messages and debug information (true) or do not show (false). Default is false. Usage: |
| --showWarnings=[true/false] | Display warnings that are reported on the connection after issuing any HiveQL commands. Default is false. Usage: |
| --showDbInPrompt=[true/false] | Display the current database name in prompt. Default is false. Usage: Version: 2.2.0 (HIVE-14123) |
| --showNestedErrs=[true/false] | Display nested errors. Default is false. Usage: |
| --numberFormat=[pattern] | Format numbers using a DecimalFormat pattern. Usage: |
| --force=[true/false] | Continue running script even after errors (true) or do not continue (false). Default is false. Usage: |
| --maxWidth=MAXWIDTH | The maximum width to display before truncating data, in characters, when outputformat is table. Default is to query the terminal for current width, then fall back to 80. Usage: |
| --maxColumnWidth=MAXCOLWIDTH | The maximum column width, in characters, when outputformat is table. Default is 50 in Hive version 2.2.0+ (see HIVE-14135) or 15 in earlier versions. Usage: |
| --silent=[true/false] | Reduce the amount of informational messages displayed (true) or not (false). It also stops displaying the log messages for the query from HiveServer2 (Hive 0.14 and later) and the HiveQL commands (Hive 1.2.0 and later). Default is false. Usage: |
| --autosave=[true/false] | Automatically save preferences (true) or do not autosave (false). Default is false. Usage: |
| --outputformat=[table/vertical/csv/tsv/dsv/csv2/tsv2] | Format mode for result display. Default is table. See Separated-Value Output Formats below for description of recommended sv options. Usage: Version: dsv/csv2/tsv2 added in 0.14.0 (HIVE-8615) |
| --truncateTable=[true/false] | If true, truncates table column in the console when it exceeds console length. Version: 0.14.0 (HIVE-6928) |
| --delimiterForDSV= DELIMITER | The delimiter for delimiter-separated values output format. Default is '|' character. Version: 0.14.0 (HIVE-7390) |
| --isolation=LEVEL | Set the transaction isolation level to TRANSACTION_READ_COMMITTED Usage: |
| --nullemptystring=[true/false] | Use historic behavior of printing null as empty string (true) or use current behavior of printing null as NULL (false). Default is false. Usage: Version: 0.13.0 (HIVE-4485) |
--incremental=[true/false] | Defaults to |
| --incrementalBufferRows=NUMROWS | The number of rows to buffer when printing rows on stdout, defaults to 1000; only applicable if Usage: Version: 2.3.0 (HIVE-14170) |
| --maxHistoryRows=NUMROWS | The maximum number of rows to store Beeline history. Version: 2.3.0 (HIVE-15166) |
| --delimiter=; | Set the delimiter for queries written in Beeline. Multi-char delimiters are allowed, but quotation marks, slashes, and -- are not allowed. Defaults to ; Usage: Version: 3.0.0 (HIVE-10865) |
| --convertBinaryArrayToString=[true/false] | Display binary column data as string or as byte array. Usage: Version: 3.0.0 (HIVE-14786) |
--help | Display a usage message. Usage: |
...
<host1>:<port1>,<host2>:<port2>is a server instance or a comma separated list of server instances to connect to (if dynamic service discovery is enabled). If empty, the embedded server will be used.dbNameis the name of the initial database.- <file> is the path of init script file (Hive 2.2.0 and later). This script file is written with SQL statements which will be executed automatically after connection. This option can be empty.
sess_var_listis a semicolon separated list of key=value pairs of session variables (e.g.,user=foo;password=bar).hive_conf_listis a semicolon separated list of key=value pairs of Hive configuration variables for this sessionhive_var_listis a semicolon separated list of key=value pairs of Hive variables for this session.
Special characters in sess_var_list, hive_conf_list, hive_var_list parameter values should be encoded with URL encoding if needed.
Connection URL for Remote or Embedded Mode
...
As of Hive 2.2.0 (HIVE-14063), BeeLineBeeline adds support to use the hive-site.xml present in the classpath to automatically generate a connection url URL based on the configuration properties in hive-site.xml and an additional user configuration file. Not all the url URL properties can be derived from hive-site.xml and hence in order to use this feature user must create a configuration file called “beeline-hs2-connection.xml” which is a Hadoop xml XML format file. This file is used to provide user-specific connection properties for the connection URL. BeeLine Beeline looks for this configuration file in ${user.home}/.beeline/ (Unix based OS) or ${user.home}\beeline\ directory (in case of Windows). If the file is not found in the above locations BeeLine Beeline looks for it in ${HIVE_CONF_DIR} location and /etc/hive/conf (Check check HIVE-16335 which fixes this location from /etc/conf/hive in Hive 2.2.0) in that order. Once the file is found, BeeLine Beeline uses beeline-hs2-connection.xml in conjunction with the hive-site.xml in the class path to determine the connection URL.
The url URL connection properties in beeline-hs2-connection.xml must have the prefix “beeline.hs2.connection.” followed by the url URL property name. For example in order to provide the property ssl the property key in the beeline-hs2-connection.xml should be “beeline.hs2.connection.ssl”. The sample beeline.hs2.connection.xml below provides the value of user and password for the beeline Beeline connection urlURL. In this case the rest of the properties like HS2 hostname and port information, kerberos Kerberos configuration properties, SSL properties, transport mode, etc., are picked up using the hive-site.xml in the class path. If the password is empty beeline.hs2.connection.password property should be removed. In most cases the below configuration values in beeline-hs2-connection.xml and the correct hive-site.xml in classpath should be sufficient to make the connection to the HiveServer2.
| Code Block |
|---|
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>beeline.hs2.connection.user</name> <value>hive</value> </property> <property> <name>beeline.hs2.connection.password</name> <value>hive</value> </property> </configuration> |
In case of properties which are present in both beeline-hs2-connection.xml and hive-site.xml, the property value derived from beeline-hs2-connection.xml takes precedence. For example in the below beeline-hs2-connection.xml file provides the value of principal for BeeLine Beeline connection in a kerberos Kerberos enabled environment. In this case the property value for beeline.hs2.connection.principal overrides the value of HiveConf.ConfVars.HIVE_SERVER2_KERBEROS_PRINCIPAL from hive-site.xml as far as connection URL is concerned.
...
In case of properties beeline.hs2.connection.hosts, beeline.hs2.connection.hiveconf and beeline.hs2.connection.hivevar the property value is a comma-separated list of values. For example the following beeline-hs2-connection.xml provides the hiveconf and hivevar values in a comma separated format.
...
When the beeline-hs2-connection.xml is present and when no other arguments are provided BeeLine , Beeline automatically connects to the URL generated using configuration files. When connection arguments (-u, -n or -p) are provided BeeLine uses them and does not use beeline-hs2-connection.xml to automatically connect. Removing or renaming the beeline-hs2-connection.xml disables this feature.connection arguments (-u, -n or -p) are provided, Beeline uses them and does not use beeline-hs2-connection.xml to automatically connect. Removing or renaming the beeline-hs2-connection.xml disables this feature.
Using beeline-site.xml to automatically connect to HiveServer2
In addition to the above method of using hive-site.xml and beeline-hs2-connection.xml for deriving the JDBC connection URL to use when connecting to HiveServer2 from Beeline, a user can optionally add beeline-site.xml to their classpath, and within beeline-site.xml, she can specify complete JDBC URLs. A user can also specify multiple named URLs and use beeline -c <named_url> to connect to a specific URL. This is particularly useful when the same cluster has multiple HiveServer2 instances running with different configurations. One of the named URLs is treated as default (which is the URL that gets used when the user simply types beeline). An example beeline-site.xml is shown below:
| Code Block |
|---|
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>beeline.hs2.jdbc.url.tcpUrl</name>
<value>jdbc:hive2://localhost:10000/default;user=hive;password=hive</value>
</property>
<property>
<name>beeline.hs2.jdbc.url.httpUrl</name>
<value>jdbc:hive2://localhost:10000/default;user=hive;password=hive;transportMode=http;httpPath=cliservice</value>
</property>
<property>
<name>beeline.hs2.jdbc.url.default</name>
<value>tcpUrl</value>
</property>
</configuration> |
In the above example, simply typing beeline opens a new JDBC connection to jdbc:hive2://localhost:10000/default;user=hive;password=hive. If both beeline-site.xml and beeline-hs2-connection.xml are present in the classpath, the final URL is created by applying the properties specified in beeline-hs2-connection.xml on top of the URL properties derived from beeline-site.xml. As an example consider the following beeline-hs2-connection.xml:
| Code Block |
|---|
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>beeline.hs2.connection.user</name>
<value>hive</value>
</property>
<property>
<name>beeline.hs2.connection.password</name>
<value>hive</value>
</property>
</configuration> |
Consider the following beeline-site.xml:
| Code Block |
|---|
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>beeline.hs2.jdbc.url.tcpUrl</name>
<value>jdbc:hive2://localhost:10000/default</value>
</property>
<property>
<name>beeline.hs2.jdbc.url.httpUrl</name>
<value>jdbc:hive2://localhost:10000/default;transportMode=http;httpPath=cliservice</value>
</property>
<property>
<name>beeline.hs2.jdbc.url.default</name>
<value>tcpUrl</value>
</property>
</configuration> |
In the above example, simply typing beeline opens a new JDBC connection to jdbc:hive2://localhost:10000/default;user=hive;password=hive. When the user types beeline -c httpUrl, a connection is opened to jdbc:hive2://localhost:10000/default;transportMode=http;httpPath=cliservice;user=hive;password=hive.
Using JDBC
You can use JDBC to access data stored in a relational database or other tabular format.
...
Hive Type | Java Type | Specification | |
|---|---|---|---|
TINYINT | byte | signed or unsigned 1-byte integer | |
SMALLINT | short | signed 2-byte integer | |
INT | int | signed 4-byte integer | |
BIGINT | long | signed 8-byte integer | |
FLOAT | double | single-precision number (approximately 7 digits) | |
DOUBLE | double | double-precision number (approximately 15 digits) | |
DECIMAL | java.math.BigDecimal | fixed-precision decimal value | |
BOOLEAN | boolean | a single bit (0 or 1) | |
STRING | String | character string or variable-length character string | |
TIMESTAMP | java.sql.Timestamp | date and time value | |
BINARY | String | binary data | |
Complex Types |
| ||
ARRAY | String – json encoded | values of one data type | |
MAP | String – json encoded | key-value pairs | |
STRUCT | String – json encoded | structured values |
...
When the above URL is specified, Beeline will call underlying requests to add HTTP cookie in the request header, and will set it to <name1>=<value1> and <name2>=<value2>.