Introduction of Sentry-Hive binding with V2
Currently Hive-Sentry binding with v1 grant/revoke privileges via hook DDLTask, and do authorization via HiveSemanticAnalyzerHook. Now hive has a pluggable authorization framework via exposing some interfaces HiveAccessController and HiveAuthorizationValidator. HiveAccessController is used to grant/revoke roles and privileges. HiveAuthorizationValidator is used to do fine-grained authorization.We add a new authorization V2 via implement Hive authorization framework.
Architecture diagram of Sentry-Hive binding with V2
Related configuration
Properties required on Hive to talk to Sentry policy store service (hive-site.xml):
| Configuration Key | Configuration Value |
|---|---|
| hive.security.authorization.task.factory | org.apache.sentry.binding.hive.v2.SentryHiveAuthorizationTaskFactoryImplV2 |
| hive.server2.session.hook | org.apache.sentry.binding.hive.v2.HiveAuthzBindingSessionHookV2 |
| hive.server2.enable.doAs | false |
| hive.security.authorization.enabled | true |
| hive.security.authorization.manager | org.apache.sentry.binding.hive.v2.SentryAuthorizerFactory |
| hive.security.authenticator.manager | org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator |
Properties required on Metastore to talk to Sentry policy store service: (hive-site.xml)：
| Configuration Key | Configuration Value |
|---|---|
| hive.metastore.rawstore.impl | org.apache.sentry.binding.hive.v2.metastore.AuthorizingObjectStoreV2 |
| hive.metastore.pre.event.listeners | org.apache.sentry.binding.hive.v2.metastore.MetastoreAuthzBindingV2 |
Support Hive version
While we
Q: Which Hive version could enable Sentry with Hive Authorization V2
(1)Here is the release date of Hive, since 0.13.0 is released at 2014/04/15, it may not a good backport for 1.1.0
hive-0.13.0 2014-04-15
...
hive-1.1.0 2015-03-09
hive-1.1.1 2015-05-21
hive-1.2.0 2015-05-15
hive-1.2.1 2015-06-26
hive-2.0.0 2016-02-15
...
have some fixes at hive side, they are HIVE-11780, HIVE-11498, HIVE-11190, HIVE-11179. Especially HIVE-11179 which blocks the authorization of URI type privilege in Sentry. Hive 1.3.0 and Hive 2.0.0 can be adapted in theory, since 1.3.0 is not released, our E2E test works for Hive 2.0.0.
Q: Is there any backward incompatible changes? Upgrade concerns?
Here is the configuration need to be updated.
| Configuration Key | Configuration Value |
|---|---|
| hive.security.authorization.task.factory | org.apache.sentry.binding.hive.v2.SentryHiveAuthorizationTaskFactoryImplV2 |
| hive.server2.session.hook | org.apache.sentry.binding.hive.v2.HiveAuthzBindingSessionHookV2 |
| hive.metastore.rawstore.impl | org.apache.sentry.binding.hive.v2.metastore.AuthorizingObjectStoreV2 |
| hive.metastore.pre.event.listeners | org.apache.sentry.binding.hive.v2.metastore.MetastoreAuthzBindingV2 |
| hive.server2.enable.doAs | false |
| hive.security.authorization.enabled | true |
| hive.security.authorization.manager | org.apache.sentry.binding.hive.v2.SentryAuthorizerFactory |
| hive.security.authenticator.manager | org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator |
...
Testing
All the unit tests are passed after fixed some test failures which are caused by the difference output format of some certain commands between Hive 1.1.0 and Hive 2.0.0, such as SHOW INDEX ON [table] in “TestMetadataObjectRetrieval.java”, SHOW GRANT [type] [object], column type privilege are put into “[]”.
We can run independent unit test by
mvn test -Dtest=className#methodName \
-DfailIfNoTests=false \
-P-hive-authz1,hive-authz2,-datanucleus3,datanucleus4
We have done e2e tests in development environment and v2 nightly build. Due to lack of real cluster environment, we hope other committers could help verify it on real cluster.
Continuous integration
The nightly build in Apache Jenkins is
https://builds.apache.org/job/Sentry-jdk-1.7-v2
The pre-commit build in Apache Jenkins is ("Hive V2" component should be added at jira)
https://builds.apache.org/job/PreCommit-SENTRY-Build/
Plan to deprecate V1