Hive Data Types

Overview

This lists all supported data types in Hive. See Type System in the Tutorial for additional information.

For data types supported by HCatalog, see:

• HCatLoader Data Types
• HCatStorer Data Types
• HCatRecord Data Types

Numeric Types

• TINYINT (1-byte signed integer, from -128 to 127)
• SMALLINT (2-byte signed integer, from -32,768 to 32,767)

• INT/INTEGER (4-byte signed integer, from -2,147,483,648 to 2,147,483,647)

• BIGINT (8-byte signed integer, from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807)
• FLOAT (4-byte single precision floating point number)
• DOUBLE (8-byte double precision floating point number)

• DOUBLE PRECISION (alias for DOUBLE, only available starting with Hive 2.2.0)

• DECIMAL
  • Introduced in Hive 0.11.0 with a precision of 38 digits
  • Hive 0.13.0 introduced user-definable precision and scale
• NUMERIC (same as DECIMAL, starting with Hive 3.0.0DECIMAL (Note: Only available starting with Hive 0.11.0)

Date/Time Types

• TIMESTAMP (Note: Only available starting with Hive 0.8.0)
• DATE (Note: Only available starting with Hive 0.12.0)
• INTERVAL (Note: Only available starting with Hive 1.2.0)

String Types

• STRING
• VARCHAR (Note: Only available starting with Hive 0.12.0)
• CHAR (Note: Only available starting with Hive 0.13.0)

Misc Types

• BOOLEANSTRING
• BINARY (Note: Only available starting with Hive 0.8.0)

Complex Types

• arrays: ARRAY<data_type> (Note: negative values and non-constant expressions are allowed as of Hive 0.14.)
• maps: MAP<primitive_type, data_type>unmigrated-wiki-markup (Note: negative values and non-constant expressions are allowed as of Hive 0.14.)
• structs: {{STRUCT<col_name : data_type \ [COMMENT col_comment], ...>}}
• union: UNIONTYPE<data_type, data_type, ...> (Note: Only available starting with Hive 0.7.0.)

Column Types

Integral Types (TINYINT, SMALLINT, INT/INTEGER, BIGINT)

Anchor
Integral Types (TINYINT, SMALLINT, INT/INTEGER, BIGINT) Integral Types (TINYINT, SMALLINT, INT/INTEGER, BIGINT)

Anchor
tinyint tinyint

Anchor
smallint smallint

Anchor
int int

Anchor
bigint bigint

Integral literals are assumed to be INT by default, unless the number exceeds the range of INT in which case it is interpreted as a BIGINT, or if one of the following postfixes is present on the number.

Strings

Anchor
string string

Anchor
Strings Strings

String literals can be expressed with either single quotes (') or double quotes ("). Hive uses C-style escaping within the strings.

Varchar

Anchor
varchar varchar

Anchor
Varchar Varchar

Varchar types are created with a length specifier (between 1 and 65535), which defines the maximum number of characters allowed in the character string. If a string value being converted/assigned to a varchar value exceeds the length specifier, the string is silently truncated. Character length is determined by the number of code points contained by the character string.

Like string, trailing whitespace is significant in varchar and will affect comparison results.

Char

Anchor
char char

Anchor
Char Char

Char types are similar to Varchar but they are fixed-length meaning that values shorter than the specified length value are padded with spaces but trailing spaces are not important during comparisons. The maximum length is fixed at 255.

CREATE TABLE foo (bar CHAR(10))

Timestamps

Anchor
timestamp timestamp

Anchor
Timestamps Timestamps

Supports traditional UNIX timestamp with optional nanosecond precision.

...

Timestamps are interpreted to be timezoneless and stored as an offset from the UNIX epoch. Convenience UDFs for conversion to and from timezones are provided (to_utc_timestamp, from_utc_timestamp).
All existing datetime UDFs (month, day, year, hour, etc.) work with the TIMESTAMP data type.

Timestamps in text files have to use the format yyyy-mm-dd hh:mm:ss[.f...]. If they are in another format, declare them as the appropriate type (INT, FLOAT, STRING, etc.) and use a UDF to convert them to timestamps.

On the table level, alternative timestamp formats can be supported by providing the format to the SerDe property "timestamp.formats" (as of release 1.2.0 with HIVE-9298). For example, yyyy-MM-dd'T'HH:mm:ss.SSS,yyyy-MM-dd'T'HH:mm:ss.

...

Dates

DATE values describe a particular year/month/day, in the form YYYY-­MM-­DD. For example, DATE '2013-­01-­01'. Date types do not have a time of day component. The range of values supported for the Date type is 0000-­01-­01 to 9999-­12-­31, dependent on support by the primitive Java Date type.

Casting Dates

Date types can only be converted to/from Date, Timestamp, or String types. Casting with user-specified formats is documented here.

Intervals

Decimals 

Anchor
decimal decimal

Anchor
Decimals Decimals

The DECIMAL type in Hive is based on Java's BigDecimal which is used for representing immutable arbitrary precision decimal numbers in Java. All regular number operations (e.g. +, -, *, /) and relevant UDFs (e.g. Floor, Ceil, Round, and many more) handle decimal types. You can cast to/from decimal types like you would do with other numeric types. The persistence format of the decimal type supports both scientific and non-scientific notation. Therefore, regardless of whether your dataset contains data like 1E+44 like 4.004E+3 (scientific notation) or 4004 (non-scientific notation) or a combination of both, DECIMAL can be used for it.With the present Hive implementation, the Decimal type has precision limited to 38 digits. While this has minimal impact on usability, it's worth noting for performance reasons. HIVE-3796 is tracking the progress of allowing users to specify scale and precision when creating tables with decimal datatype

• Hive 0.11 and 0.12 have the precision of the DECIMAL type fixed and limited to 38 digits.
• As of Hive 0.13 users can specify scale and precision when creating tables with the DECIMAL datatype using a DECIMAL(precision, scale) syntax.  If scale is not specified, it defaults to 0 (no fractional digits). If no precision is specified, it defaults to 10.

CREATE TABLE foo (
  a DECIMAL, -- Defaults to decimal(10,0)
  b DECIMAL(9, 7)
)

For usage, see Floating Point Types in the Literals section below.

Decimal Literals

Anchor
Decimal Literals Decimal Literals

Integral literals larger than BIGINT must be handled with Decimal(38,0). The Postfix BD is required. Example:

select CAST(18446744073709001000BD AS DECIMAL(38,0)) from my_table limit 1;

Decimal Type Incompatibilities between Hive 0.12.0 and 0.13.0

With the changes in the Decimal data type in Hive 0.13.0, the pre-Hive 0.13.0 columns (of type "decimal") will be treated as being of type decimal(10,0).  What this means is that existing data being read from these tables will be treated as 10-digit integer values, and data being written to these tables will be converted to 10-digit integer values before being written. To avoid these issues, Hive users on 0.12 or earlier with tables containing Decimal columns will be required to migrate their tables, after upgrading to Hive 0.13.0 or later.

Upgrading Pre-Hive 0.13.0 Decimal Columns

If the user was on Hive 0.12.0 or earlier and created tables with decimal columns, they should perform the following steps on these tables after upgrading to Hive 0.13.0 or later.

1. Determine what precision/scale you would like to set for the decimal column in the table.

2. For each decimal column in the table, update the column definition to the desired precision/scale using the ALTER TABLE command:

   Code Block
   sql sql
   ALTER TABLE foo CHANGE COLUMN dec_column_name dec_column_name DECIMAL(38,18);

   If the table is not a partitioned table, then you are done.  If the table has partitions, then go on to step 3.

3. If the table is a partitioned table, then find the list of partitions for the table:

   Code Block
   sql sql
   SHOW PARTITIONS foo;   ds=2008-04-08/hr=11 ds=2008-04-08/hr=12 ...

   
   

4. Each existing partition in the table must also have its DECIMAL column changed to add the desired precision/scale.

   This can be done with a single ALTER TABLE CHANGE COLUMN by using dynamic partitioning (available for ALTER TABLE CHANGE COLUMN in Hive 0.14 or later, with HIVE-8411):

   Code Block
   sql sql
   SET hive.exec.dynamic.partition = true;   -- hive.exec.dynamic.partition needs to be set to true to enable dynamic partitioning with ALTER PARTITION -- This will alter all existing partitions of the table - be sure you know what you are doing! ALTER TABLE foo PARTITION (ds, hr) CHANGE COLUMN dec_column_name dec_column_name DECIMAL(38,18);

   
   

   Alternatively, this can be done one partition at a time using ALTER TABLE CHANGE COLUMN, by specifying one partition per statement (This is available in Hive 0.14 or later, with HIVE-7971.):

   Code Block
   sql sql
   ALTER TABLE foo PARTITION (ds='2008-04-08', hr=11) CHANGE COLUMN dec_column_name dec_column_name DECIMAL(38,18); ALTER TABLE foo PARTITION (ds='2008-04-08', hr=12) CHANGE COLUMN dec_column_name dec_column_name DECIMAL(38,18); ...

   
   

The Decimal datatype is discussed further in Floating Point Types below.

Union Types

Anchor
union union

Anchor
Union Types Union Types

...

Union Types

Union types can at any one point hold exactly one of their specified data types. You can create an instance of this type using the create_union UDF:

CREATE TABLE union_test(foo UNIONTYPE<int, double, array<string>, struct<a:int,b:string>>);
SELECT foo FROM union_test;

{0:1}
{1:2.0}
{2:["three","four"]}
{3:{"a":5,"b":"five"}}
{2:["six","seven"]}
{3:{"a":8,"b":"eight"}}
{0:9}
{1:10.0}

...

To create a union you have to provide this tag to the create_union UDF:

SELECT create_union(0, key), create_union(if(key<100, 0, 1), 2.0, value), create_union(1, "a", struct(2, "b")) FROM src LIMIT 2;

{0:"238"}	{1:"val_238"}	{1:{"col1":2,"col2":"b"}}
{0:"86"}	{0:2.0}	{1:{"col1":2,"col2":"b"}}

...

Decimal literals provide precise values and greater range for floating point numbers than the DOUBLE type. Decimal data types store exact representations of numeric values, while DOUBLE data types store very close approximations of numeric values.

...

You can create a table in Hive that uses the Decimal type with the following syntax:

create table decimal_1 (t decimal);

...

You can read and write values in such a table using either the LazySimpleSerDe or the LazyBinarySerDe. For example:

alter table decimal_1 set serde 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe';

or:

alter table decimal_1 set serde 'org.apache.hadoop.hive.serde2.lazy.LazyBinarySerDe';

You can use a cast to convert a Decimal value to any other primitive type such as a BOOLEAN. For example:

select cast(t as boolean) from decimal_2;

...

Decimal also supports many arithmetic operators, mathematical UDFs and UDAFs with the same syntax as used in the case of DOUBLE.

...

• Positive
• Negative
• Addition
• Subtraction
• Multiplication
• Division
• Average (avg)
• Sum
• Count
• Modulus (pmod)
• Sign – Hive 0.13.0 and later
• Exp – Hive 0.13.0 and later
• Ln – Hive 0.13.0 and later
• Log2 – Hive 0.13.0 and later
• Log10 – Hive 0.13.0 and later
• Log(base) – Hive 0.13.0 and later
• Sqrt – Hive 0.13.0 and later
• Sin – Hive 0.13.0 and later
• Asin – Hive 0.13.0 and later
• Cos – Hive 0.13.0 and later
• Acos – Hive 0.13.0 and later
• Tan – Hive 0.13.0 and later
• Atan – Hive 0.13.0 and later
• Radians – Hive 0.13.0 and later
• Degrees – Hive 0.13.0 and later

These rounding functions can also take decimal types:

• Floor
• Ceiling
• Round

But these mathematical UDFs are not currently supported:

• Exp
• Log (ln)
• Log2
• Log10
• Sqrt

Power(decimal, n) only supports positive integer values for the exponent n.

...

Missing values are represented by the special value NULL. To import data with NULL fields, check documentation of the SerDe used by the table. (The default Text Format uses LazySimpleSerDe which interprets the string \N as NULL when importing.)

Change Types

When hive.metastore.disallow.incompatible.col.type.changes is set to false, the types of columns in Metastore can be changed from any type to any other type. After such a type change, if the data can be shown correctly with the new type, the data will be displayed. Otherwise, the data will be displayed as NULL.

Allowed Implicit Conversions

Save