Index

Authorization modes

This is the design document for the original Hive authorization mode. See Authorization for an overview of authorization modes, which include storage based authorization and SQL standards based authorization.

1. Privilege

1.1 Access Privilege

Admin privilege, DB privilege, Table level privilege, column level privilege

1.1.1 Admin privileges are global privileges, and are used to perform administration.

1.1.2 DB privileges are database specific, and apply to all objects inside that database.

1.1.3 Table privileges apply to table/view/index in a given database

1.1.4 Column privileges apply to column level.

All DB/Table/Column privilege differentiate read and write privileges even though now hive does not support column level overwrite. And there is no partition level privilege.

2. Hive Operations

create index/drop index

create database/drop database

create table/drop table

create view/drop view

alter table

show databases

lock table/unlock table/show lock

add partition

3. Metadata

Store the privilege information in the new metastore tables 'user', 'db', 'tables_priv', 'columns_priv'.

The user table indicates user's global privileges, which apply to all databases.
The db table determine database level access privileges, which apply to all objects inside that database.

3.1 user, group, and roles

User can belong to some groups. The group information is provided by authenticator.

And each user or group can have some privileges and roles. A role can be a member of another role, but not in a circular manner.

So hive metadata needs to store:

1) roles -> privileges, roles mapping

2) Hive user/group -> privileges, role mapping

3.1.1 Role management

create role

drop role

grant a role to a user

revoke a role from a user

3.1.2 role metadata

role_name - string

create_time - int

3.1.3 hive role user membership table

role_name - string

user_name - string

is_group – is the user name a group name

is_role – is the user name a role name

3.2 Privileges to be supported by Hive

3.2.1 metadata

The below shows how we store the grant information in metastore. The deny information is stored in a same matter (just in different tables).

So for each grant table, there will also be a deny table. The metastore tables are

user, deny_user, db, deny_db, tables_priv, deny_tables_priv, columns_priv, deny_columns_priv

Another way to do it is to add a column in the grant table to record this row is grant or deny.

We store privileges in one column, and use comma to separate different privileges.

hive> desc user;

Field

- - -

User

isRole

isGroup

isSuper

db_priv – set (Select_priv, Insert_priv, Create_priv, Drop_priv, Reload_priv,

Grant_priv, Index_priv, Alter_priv, Show_db_priv,

Lock_tables_priv, Create_view_priv, Show_view_priv)

hive> desc db;

Field

- - -

Db

User

isRole

isGroup

Table_priv – set (Select_priv, Insert_priv, Create_priv, Drop_priv, Grant_priv,

Index_priv, Reload_priv, Alter_priv, Create_tmp_table_priv,

Lock_tables_priv, Create_view_priv, Show_view_priv)

hive> desc tables_priv;

Field

- - -

Db

User

isRole

isGroup

Table_name

Grantor

Timestamp

Table_priv – set('Select','Insert',,'Create','Drop','Grant','Index','Alter','Create View','Show view')

Column_priv – set('Select','Insert',)

mysql> desc columns_priv;

Field

- - -

Db

User

isRole

isGroup

Table_name

Column_name

Timestamp

Column_priv – set('Select','Insert','Update')

4. grant/revoke access privilege

4.1 Privilege names/types:

ALL Privileges

ALTER

Create

Create view

Delete

Drop

Index

Insert

Lock Tables

Select

Show databases

Super

4.2 show grant

4.3 grant/revoke statement

GRANT
    priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    ON [object_type] priv_level
    TO user [, user] ...
WITH ADMIN OPTION

object_type:
    TABLE

priv_level:
    *
  | *.*
  | db_name.*
  | db_name.tbl_name
  | tbl_name

REVOKE
    priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    ON [object_type] priv_level
    FROM user [, user] ...

REVOKE ALL PRIVILEGES, GRANT OPTION
    FROM user [, user] ...


DENY  
	priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    ON [object_type] priv_level
    FROM user [, user] ...

5. Authorization verification

5.1 USER/GROUP/ROLE

USER

GROUP

ROLE

GROUP is very similar to a role. And we support Group is because we may need to pass the group information to HDFS/Map-reduce.

A role can also contain other roles and privileges - and they can be granted to users and groups.

Role can be nested but not circular.

5.2 The verification steps

When a user logins to the system, he has a user name, one or few groups that he belongs to.
So it is

[

username, 

list of group names, 

list of privileges and roles that has been directly granted, 

list of privileges and roles that been directly granted to groups that users belongs to

].

Steps to authorize one access: *

First try user name:

 # If there is an entry in 'user' that accept this access, return ACCEPT

2. If there is an entry in 'db'  that accept this access, return ACCEPT

3. If there is an entry in 'table'  that accept this access, return ACCEPT

4. If there is an entry in 'column'  that accept this access, return ACCEPT

Second try the user's group/role names one by one until we get an ACCEPT. 

For each role/group, we do the same routine as we did for user name.

5.3 Examples

5.3.1 I want to grant everyone (new people may join at anytime) to
db_name.*, and then later i want to protect one table db_name.T from ALL
users but a few

1) Add all users to a group 'users'. (assumption: new users will
automatically join this group). And grant 'users' ALL privileges to db_name.*

2) Add those few users to a new group 'users2'. AND REMOVE them from 'users'

3) DENY 'users' to db_name.T

4) Grant ALL on db_name.T to users2

5.3.2 I want to protect one table db_name.T from one/few users, but all
other people can access it

1) Add all users to a group 'users'. (assumption: new users will automatically
join this group). And grant 'users' ALL privileges to db_name.*.

2) Add those few users to a new group 'users2'. (Note: those few users will now
belong to 2 groups: users and user2)

3) DENY 'users2' to db_name.T

6. Where to add authorization in Hive

CliDriver and HiveServer. Basically they share the same code. If HiveServer invokes CliDriver, we can just add it into CliDriver. And we also need to make HiveServer be able to support multiple user/connections.
This still does not solve the problem if someone accesses the metastore directly (without going through Hive).

7. Implementation

7.1 Authenticator interface

We only get the user's user name, group names from the authenticator. The authenticator implementations need to provide these information. This is the only interface between authenticator and authorization.

7.2 Authorization

Authorization decision manager manages a set of authorization provider, and each provider can decide to accept or deny. And it is the decision manager to do the final decision. Can be vote based, or one -1 then deny, or one +1 then accept. Authorization provider decides whether to accept or deny an access based on his own information.

8. Metastore upgrade script for mysql

--
-- Table structure for table {{ROLES}}
--

DROP TABLE IF EXISTS {{ROLES}};
CREATE TABLE {{ROLES}} (
  {{ROLE_ID}} bigint(20) NOT NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{OWNER_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{ROLE_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  PRIMARY KEY  ({{ROLE_ID}}),
  UNIQUE KEY {{ROLEENTITYINDEX}} ({{ROLE_NAME}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;


--
-- Table structure for table {{ROLE_MAP}}
--

DROP TABLE IF EXISTS {{ROLE_MAP}};
CREATE TABLE {{ROLE_MAP}} (
  {{ROLE_GRANT_ID}} bigint(20) NOT NULL,
  {{ADD_TIME}} int(11) NOT NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{ROLE_ID}} bigint(20) default NULL,
  PRIMARY KEY  ({{ROLE_GRANT_ID}}),
  UNIQUE KEY {{USERROLEMAPINDEX}} ({{PRINCIPAL_NAME}},{{ROLE_ID}},{{GRANTOR}},{{GRANTOR_TYPE}}),
  KEY {{ROLE_MAP_N49}} ({{ROLE_ID}}),
  CONSTRAINT {{ROLE_MAP_FK1}} FOREIGN KEY ({{ROLE_ID}}) REFERENCES {{ROLES}} ({{ROLE_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

--
-- Table structure for table {{GLOBAL_PRIVS}}
--

DROP TABLE IF EXISTS {{GLOBAL_PRIVS}};
CREATE TABLE {{GLOBAL_PRIVS}} (
  {{USER_GRANT_ID}} bigint(20) NOT NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{USER_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
  PRIMARY KEY  ({{USER_GRANT_ID}}),
  UNIQUE KEY {{GLOBALPRIVILEGEINDEX}} ({{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{USER_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;


--
-- Table structure for table {{DB_PRIVS}}
--

DROP TABLE IF EXISTS {{DB_PRIVS}};
CREATE TABLE {{DB_PRIVS}} (
  {{DB_GRANT_ID}} bigint(20) NOT NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{DB_ID}} bigint(20) default NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{DB_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
  PRIMARY KEY  ({{DB_GRANT_ID}}),
  UNIQUE KEY {{DBPRIVILEGEINDEX}} ({{DB_ID}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{DB_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
  KEY {{DB_PRIVS_N49}} ({{DB_ID}}),
  CONSTRAINT {{DB_PRIVS_FK1}} FOREIGN KEY ({{DB_ID}}) REFERENCES {{DBS}} ({{DB_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

--
-- Table structure for table {{TBL_PRIVS}}
--

DROP TABLE IF EXISTS {{TBL_PRIVS}};

CREATE TABLE {{TBL_PRIVS}} (
  {{TBL_GRANT_ID}} bigint(20) NOT NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{TBL_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{TBL_ID}} bigint(20) default NULL,
  PRIMARY KEY  ({{TBL_GRANT_ID}}),
  KEY {{TBL_PRIVS_N49}} ({{TBL_ID}}),
  KEY {{TABLEPRIVILEGEINDEX}} ({{TBL_ID}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{TBL_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
  CONSTRAINT {{TBL_PRIVS_FK1}} FOREIGN KEY ({{TBL_ID}}) REFERENCES {{TBLS}} ({{TBL_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

--
-- Table structure for table {{PART_PRIVS}}
--

DROP TABLE IF EXISTS {{PART_PRIVS}};

CREATE TABLE {{PART_PRIVS}} (
  {{PART_GRANT_ID}} bigint(20) NOT NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PART_ID}} bigint(20) default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PART_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
  PRIMARY KEY  ({{PART_GRANT_ID}}),
  KEY {{PARTPRIVILEGEINDEX}} ({{PART_ID}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{PART_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
  KEY {{PART_PRIVS_N49}} ({{PART_ID}}),
  CONSTRAINT {{PART_PRIVS_FK1}} FOREIGN KEY ({{PART_ID}}) REFERENCES {{PARTITIONS}} ({{PART_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

--
-- Table structure for table {{TBL_COL_PRIVS}}
--

DROP TABLE IF EXISTS {{TBL_COL_PRIVS}};
CREATE TABLE {{TBL_COL_PRIVS}} (
  {{TBL_COLUMN_GRANT_ID}} bigint(20) NOT NULL,
  {{COLUMN_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{TBL_COL_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{TBL_ID}} bigint(20) default NULL,
  PRIMARY KEY  ({{TBL_COLUMN_GRANT_ID}}),
  KEY {{TABLECOLUMNPRIVILEGEINDEX}} ({{TBL_ID}},{{COLUMN_NAME}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{TBL_COL_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
  KEY {{TBL_COL_PRIVS_N49}} ({{TBL_ID}}),
  CONSTRAINT {{TBL_COL_PRIVS_FK1}} FOREIGN KEY ({{TBL_ID}}) REFERENCES {{TBLS}} ({{TBL_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

DROP TABLE IF EXISTS {{PART_COL_PRIVS}};
CREATE TABLE {{PART_COL_PRIVS}} (
  {{PART_COLUMN_GRANT_ID}} bigint(20) NOT NULL,
  {{COLUMN_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{CREATE_TIME}} int(11) NOT NULL,
  {{GRANT_OPTION}} smallint(6) NOT NULL,
  {{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PART_ID}} bigint(20) default NULL,
  {{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
  {{PART_COL_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
  PRIMARY KEY  ({{PART_COLUMN_GRANT_ID}}),
  KEY {{PART_COL_PRIVS_N49}} ({{PART_ID}}),
  KEY {{PARTITIONCOLUMNPRIVILEGEINDEX}} ({{PART_ID}},{{COLUMN_NAME}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{PART_COL_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
  CONSTRAINT {{PART_COL_PRIVS_FK1}} FOREIGN KEY ({{PART_ID}}) REFERENCES {{PARTITIONS}} ({{PART_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

HDFS Permission

The above has a STRONG assumption on the file layer security. Users can easily by-pass the security if the hdfs file permission is open to him. We hope we can easily plug in external authorizations (like HDFS permission/Howl permission) to alter the authorization result or even the rule.

Space shortcuts

Child pages

1. Privilege

1.1 Access Privilege

2. Hive Operations

3. Metadata

3.1 user, group, and roles

3.1.1 Role management

3.1.2 role metadata

3.1.3 hive role user membership table

3.2 Privileges to be supported by Hive

3.2.1 metadata

4. grant/revoke access privilege

4.1 Privilege names/types:

4.2 show grant

4.3 grant/revoke statement

5. Authorization verification

5.1 USER/GROUP/ROLE

5.2 The verification steps

5.3 Examples

6. Where to add authorization in Hive

7. Implementation

7.1 Authenticator interface

7.2 Authorization

8. Metastore upgrade script for mysql

HDFS Permission

Space shortcuts

Child pages

AuthDev

1. Privilege

1.1 Access Privilege

2. Hive Operations

3. Metadata

3.1 user, group, and roles

3.1.1 Role management

3.1.2 role metadata

3.1.3 hive role user membership table

3.2 Privileges to be supported by Hive

3.2.1 metadata

4. grant/revoke access privilege

4.1 Privilege names/types:

4.2 show grant

4.3 grant/revoke statement

5. Authorization verification

5.1 USER/GROUP/ROLE

5.2 The verification steps

5.3 Examples

6. Where to add authorization in Hive

7. Implementation

7.1 Authenticator interface

7.2 Authorization

8. Metastore upgrade script for mysql

HDFS Permission