Child pages
  • LanguageManual UDF

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: edit 2 timestamp descriptions in Date Functions, change signature of 2nd to match 1st (ts, not *)

...

Return Type

Name(Signature)

Description

string

from_unixtime(bigint unixtime[, string format])

Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the format of "1970-01-01 00:00:00".

bigint

unix_timestamp()

Gets current Unix timestamp in seconds. This function is not deterministic and its value is not fixed for the scope of a query execution, therefore prevents proper optimization of queries - this has been deprecated since 2.0 in favour of CURRENT_TIMESTAMP constant.

bigint

unix_timestamp(string date)

Converts time string in format yyyy-MM-dd HH:mm:ss to Unix timestamp (in seconds), using the default timezone and the default locale, return 0 if fail: unix_timestamp('2009-03-20 11:30:01') = 1237573801

bigint

unix_timestamp(string date, string pattern)

Convert time string with given pattern (see [http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html]) to Unix time stamp (in seconds), return 0 if fail: unix_timestamp('2009-03-20', 'yyyy-MM-dd') = 1237532400.

pre 2.1.0: string

2.1.0 on: date

to_date(string timestamp)

Returns the date part of a timestamp string (pre-Hive 2.1.0): to_date("1970-01-01 00:00:00") = "1970-01-01". As of Hive 2.1.0, returns a date object.

Prior to Hive 2.1.0 (HIVE-13248) the return type was a String because no Date type existed when the method was created.

int

year(string date)

Returns the year part of a date or a timestamp string: year("1970-01-01 00:00:00") = 1970, year("1970-01-01") = 1970.

int

quarter(date/timestamp/string)Returns the quarter of the year for a date, timestamp, or string in the range 1 to 4 (as of Hive 1.3.0). Example: quarter('2015-04-08') = 2.

int

month(string date)

Returns the month part of a date or a timestamp string: month("1970-11-01 00:00:00") = 11, month("1970-11-01") = 11.

int

day(string date) dayofmonth(date)

Returns the day part of a date or a timestamp string: day("1970-11-01 00:00:00") = 1, day("1970-11-01") = 1.

int

hour(string date)

Returns the hour of the timestamp: hour('2009-07-30 12:58:59') = 12, hour('12:58:59') = 12.

int

minute(string date)

Returns the minute of the timestamp.

int

second(string date)

Returns the second of the timestamp.

int

weekofyear(string date)

Returns the week number of a timestamp string: weekofyear("1970-11-01 00:00:00") = 44, weekofyear("1970-11-01") = 44.

int

extract(field FROM source)

Retrieve fields such as days or hours from source (as of Hive 2.2.0). Source must be a date, timestamp, interval or a string that can be converted into either a date or timestamp. Supported fields include: day, dayofweek, hour, minute, month, quarter, second, week and year.

Examples:

  1. select extract(month from "2016-10-20") results in 10.

  2. select extract(hour from "2016-10-20 05:06:07") results in 5.

  3. select extract(dayofweek from "2016-10-20 05:06:07") results in 5.

  4. select extract(month from interval '1-3' year to month) results in 3.

  5. select extract(minute from interval '3 12:20:30' day to second) results in 20.

int

datediff(string enddate, string startdate)

Returns the number of days from startdate to enddate: datediff('2009-03-01', '2009-02-27') = 2.

pre 2.1.0: string

2.1.0 on: date

date_add(date/timestamp/string startdate, tinyint/smallint/int days)

Adds a number of days to startdate: date_add('2008-12-31', 1) = '2009-01-01'.

Prior to Hive 2.1.0 (HIVE-13248) the return type was a String because no Date type existed when the method was created.

pre 2.1.0: string

2.1.0 on: date

date_sub(date/timestamp/string startdate, tinyint/smallint/int days)

Subtracts a number of days to startdate: date_sub('2008-12-31', 1) = '2008-12-30'.

Prior to Hive 2.1.0 (HIVE-13248) the return type was a String because no Date type existed when the method was created.

timestamp

from_utc_timestamp({any primitive type} *ts, string timezone)

Coverts Converts a timestamp* in UTC to a given timezone (as of Hive 0.8.0).

* timestamp is a primitive type, including timestamp/date, tinyint/smallint/int/bigint, float/double and decimal.

Fractional values are considered as seconds. Integer values are considered as milliseconds. . E.g For example, from_utc_timestamp(2592000.0,'PST'), from_utc_timestamp(2592000000,'PST') and from_utc_timestamp(timestamp '1970-01-30 16:00:00','PST') all return the timestamp 1970-01-30 08:00:00.

timestamp

to_utc_timestamp({any primitive type} ts, string timezone)

Coverts Converts a timestamp* in a given timezone to UTC (as of Hive 0.8.0).

* timestamp is a primitive type, including timestamp/date, tinyint/smallint/int/bigint, float/double and decimal.

Fractional values are considered as seconds. Integer values are considered as milliseconds. . E.g For example, to_utc_timestamp(2592000.0,'PST'), to_utc_timestamp(2592000000,'PST') and to_utc_timestamp(timestamp '1970-01-30 16:00:00','PST') all return the timestamp 1970-01-31 00:00:00.

datecurrent_date

Returns the current date at the start of query evaluation (as of Hive 1.2.0). All calls of current_date within the same query return the same value.

timestampcurrent_timestamp

Returns the current timestamp at the start of query evaluation (as of Hive 1.2.0). All calls of current_timestamp within the same query return the same value.

stringadd_months(string start_date, int num_months)

Returns the date that is num_months after start_date (as of Hive 1.1.0). start_date is a string, date or timestamp. num_months is an integer. The time part of start_date is ignored. If start_date is the last day of the month or if the resulting month has fewer days than the day component of start_date, then the result is the last day of the resulting month. Otherwise, the result has the same day component as start_date.

stringlast_day(string date)Returns the last day of the month which the date belongs to (as of Hive 1.1.0). date is a string in the format 'yyyy-MM-dd HH:mm:ss' or 'yyyy-MM-dd'. The time part of date is ignored.
stringnext_day(string start_date, string day_of_week)Returns the first date which is later than start_date and named as day_of_week (as of Hive 1.2.0). start_date is a string/date/timestamp. day_of_week is 2 letters, 3 letters or full name of the day of the week (e.g. Mo, tue, FRIDAY). The time part of start_date is ignored. Example: next_day('2015-01-14', 'TU') = 2015-01-20.
stringtrunc(string date, string format)Returns date truncated to the unit specified by the format (as of Hive 1.2.0). Supported formats: MONTH/MON/MM, YEAR/YYYY/YY. Example: trunc('2015-03-17', 'MM') = 2015-03-01.
doublemonths_between(date1, date2)Returns number of months between dates date1 and date2 (as of Hive 1.2.0). If date1 is later than date2, then the result is positive. If date1 is earlier than date2, then the result is negative. If date1 and date2 are either the same days of the month or both last days of months, then the result is always an integer. Otherwise the UDF calculates the fractional portion of the result based on a 31-day month and considers the difference in time components date1 and date2. date1 and date2 type can be date, timestamp or string in the format 'yyyy-MM-dd' or 'yyyy-MM-dd HH:mm:ss'. The result is rounded to 8 decimal places. Example: months_between('1997-02-28 10:30:00', '1996-10-30') = 3.94959677
stringdate_format(date/timestamp/string ts, string fmt)

Converts a date/timestamp/string to a value of string in the format specified by the date format fmt (as of Hive 1.2.0). Supported formats are Java SimpleDateFormat formats https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html. The second argument fmt should be constant. Example: date_format('2015-04-08', 'y') = '2015'.

date_format can be used to implement other UDFs, e.g.:

  • dayname(date) is date_format(date, 'EEEE')
  • dayofyear(date) is date_format(date, 'D')

...

Code Block
languagesql
select explode(array('A','B','C'));
select explode(array('A','B','C')) as col;
select tf.* from (select 0) t lateral view explode(array('A','B','C')) tf;
select tf.* from (select 0) t lateral view explode(array('A','B','C')) tf as col;



HTML
outputxhtml
<table><tbody><tr><th>col</th></tr><tr><td>A</td></tr><tr><td>B</td></tr><tr><td>C</td></tr></tbody></table>




explode (map)

 

Code Block
languagesql
select explode(map('A',10,'B',20,'C',30));
select explode(map('A',10,'B',20,'C',30)) as (key,value);
select tf.* from (select 0) t lateral view explode(map('A',10,'B',20,'C',30)) tf;
select tf.* from (select 0) t lateral view explode(map('A',10,'B',20,'C',30)) tf as key,value;



HTML
outputxhtml
<table><tbody><tr><th>key</th><th>value</th></tr><tr><td>A</td><td>10</td></tr><tr><td>B</td><td>20</td></tr><tr><td>C</td><td>30</td></tr></tbody></table>

 


 

posexplode (array)

 

Code Block
languagesql
select posexplode(array('A','B','C'));
select posexplode(array('A','B','C')) as (pos,val);
select tf.* from (select 0) t lateral view posexplode(array('A','B','C')) tf;
select tf.* from (select 0) t lateral view posexplode(array('A','B','C')) tf as pos,val;


HTML
outputxhtml
<table><tbody><tr><th>pos</th><th>val</th></tr><tr><td>0</td><td>A</td></tr><tr><td>1</td><td>B</td></tr><tr><td>2</td><td>C</td></tr></tbody></table>
 


 

inline (array of structs)

Code Block
languagesql
select inline(array(struct('A',10,date '2015-01-01'),struct('B',20,date '2016-02-02')));
select inline(array(struct('A',10,date '2015-01-01'),struct('B',20,date '2016-02-02'))) as (col1,col2,col3);
select tf.* from (select 0) t lateral view inline(array(struct('A',10,date '2015-01-01'),struct('B',20,date '2016-02-02'))) tf;
select tf.* from (select 0) t lateral view inline(array(struct('A',10,date '2015-01-01'),struct('B',20,date '2016-02-02'))) tf as col1,col2,col3;



HTML
outputxhtml
<table><tbody><tr><th>col1</th><th>col2</th><th>col3</th></tr><tr><td>A</td><td>10</td><td>2015-01-01</td></tr><tr><td>B</td><td>20</td><td>2016-02-02</td></tr></tbody></table>




stack (values)

 

Code Block
languagesql
select stack(2,'A',10,date '2015-01-01','B',20,date '2016-01-01');
select stack(2,'A',10,date '2015-01-01','B',20,date '2016-01-01') as (col0,col1,col2);
select tf.* from (select 0) t lateral view stack(2,'A',10,date '2015-01-01','B',20,date '2016-01-01') tf;
select tf.* from (select 0) t lateral view stack(2,'A',10,date '2015-01-01','B',20,date '2016-01-01') tf as col0,col1,col2;


HTML
outputxhtml
<table><tbody><tr><th>col0</th><th>col1</th><th>col2</th></tr><tr><td>A</td><td>10</td><td>2015-01-01</td></tr><tr><td>B</td><td>20</td><td>2016-01-01</td></tr></tbody></table>
 


 

 

Using the syntax "SELECT udtf(col) AS colAlias..." has a few limitations:

...