GROUP BY; SORT/ORDER/CLUSTER/DISTRIBUTE BY; JOIN (Hive Joins, Join Optimization, Outer Join Behavior); UNION; TABLESAMPLE; Subqueries; Virtual Columns; Operators and UDFs; LATERAL VIEW; Windowing, OVER, and Analytics; Common Table Expressions
- A SELECT statement can be part of a union query or a subquery of another query.
table_referenceindicates the input to the query. It can be a regular table, a view, a join construct or a subquery.
- Table names and column names are case insensitive.
- In Hive 0.12 and earlier, only alphanumeric and underscore characters are allowed in table and column names.
- In Hive 0.13 and later, column names can contain any Unicode character (see HIVE-6013). Any column name that is specified within backticks (
`) is treated literally. Within a backtick string, use double backticks (
``) to represent a backtick character.
- To revert to pre-0.13.0 behavior and restrict column names to alphanumeric and underscore characters, set the configuration property
none. In this configuration, backticked names are interpreted as regular expressions. For details, see Supporting Quoted Identifiers in Column Names (attached to HIVE-6013). Also see REGEX Column Specification below.
Simple query. For example, the following query retrieves all columns and all rows from table t1.
As of Hive 0.13.0, FROM is optional (for example,
db_name.table_name" allows a query to access tables in different databases.
USE sets the database for all subsequent HiveQL statements. Reissue it with the keyword "
default" to reset to the default database.
The WHERE condition is a boolean expression. For example, the following query returns only those sales records which have an amount greater than 10 from the US region. Hive supports a number of operators and UDFs in the WHERE clause:
As of Hive 0.13 some types of subqueries are supported in the WHERE clause.
ALL and DISTINCT Clauses
The ALL and DISTINCT options specify whether duplicate rows should be returned. If none of these options are given, the default is ALL (all matching rows are returned). DISTINCT specifies removal of duplicate rows from the result set. Note, Hive supports SELECT DISTINCT * starting in release 1.1.0 (HIVE-9194).
ALL and DISTINCT can also be used in a UNION clause – see Union Syntax for more information.
Partition Based Queries
In general, a SELECT query scans the entire table (other than for sampling). If a table created using the PARTITIONED BY clause, a query can do partition pruning and scan only a fraction of the table relevant to the partitions specified by the query. Hive currently does partition pruning if the partition predicates are specified in the WHERE clause or the ON clause in a JOIN. For example, if table page_views is partitioned on column date, the following query retrieves rows for just days between 2008-03-01 and 2008-03-31.
If a table page_views is joined with another table dim_users, you can specify a range of partitions in the ON clause as follows:
Hive added support for the HAVING clause in version 0.7.0. In older versions of Hive it is possible to achieve the same effect by using a subquery, e.g:
can also be expressed as
Limit indicates the number of rows to be returned. The rows returned are chosen at random. The following query returns 5 rows from t1 at random.
Top k queries. The following query returns the top 5 sales records wrt amount.
REGEX Column Specification
A SELECT statement can take regex-based column specification in Hive releases prior to 0.13.0, or in 0.13.0 and later releases if the configuration property
hive.support.quoted.identifiers is set to
- We use Java regex syntax. Try http://www.fileformat.info/tool/regex.htm for testing purposes.
- The following query selects all columns except ds and hr.
More Select Syntax
See the following documents for additional syntax and features of SELECT statements:
- GROUP BY
- SORT/ORDER/CLUSTER/DISTRIBUTE BY
- Virtual Columns
- Operators and UDFs
- LATERAL VIEW
- Windowing, OVER, and Analytics
- Common Table Expressions