Title: High Level Design of Role Based Access Controller in SQOOP 2
JIRA : SQOOP-1834 and its sub tickets
Problem
Sqoop 2 needs a pluggable role based access controller (RBAC), which is responsible for the authorization to Sqoop 2 resources, such as connector, link, job, submission etc.
Basic Idea
- The access controller is pluggable
- Set controller class in sqoop.properties
org.apache.sqoop.accessController.class=org.apache.sqoop.accessController.DefaultSqoopAuthorizerImpl
- The default implement in Sqoop 2 could be a FAKE controller (always return true)
- The access controller class could be implemented by other controller framework, such as Sentry
- Connector
Resource, actions and rules
Resource | Global Namespace | Instance |
---|---|---|
Connector |
|
|
Link |
|
|
Job |
|
|
Action | Privilege needed |
---|---|
show connector |
|
show link |
|
create link |
|
update link |
|
delete link |
|
enable link |
|
disable link |
|
show job |
|
create job |
|
update job |
|
delete job |
|
enable job |
|
disable job |
|
start job |
|
stop job |
|
show submission |
|
Authorization framework
- Config in sqoop.properties
#org.apache.sqoop.authorization.handler=org.apache.sqoop.security.DefaultAuthorizationHandler #org.apache.sqoop.authorization.controller=org.apache.sqoop.security.DefaultAccessController #org.apache.sqoop.authorization.validator=org.apache.sqoop.security.DefaultAuthorizationValidator
- Four metadata classes.
- Role
- principal
- This class defines user or group.
- Type: user, group, role.
- principal could be granted a role. i.e. if we want to grant a admin role to user hadoop, then grantRole (principal (name=hadoop, type=user), role (name=admin)).
- Resource
- This class defines four resources in Sqoop 2.
- Type: connector, link, job.
- Privilege
- Action: create, view, update, delete, use, enable, disable.
- with_grant_option: boolean, defines whether the role could grant this privilege to other role.
- Five classes will be added into Sqoop-core as org.apache.sqoop.security package.
- AuthorizationManager
- Similar with other Sqoop Manager, ie. ConnectorManager, RepositoryManager, etc., the AuthorizationManager handles two singleton instances, AuthorizationManager and AuthorizationHandler.
- The initialize function is run when starting the Sqoop server
- The initialize function will initial AuthorizationHandler, according to the handler name (DefaultAuthorizationhandler or SentryAuthorizationHandler) from configuration file (sqoop.properties).
- AuthorizationHandlerFactory
- It is a factory design mode.
- It is to use ClassUtils.loadClass to refact the real AuthorizationHandler in getAuthorizationHandler function.
- AuthorizationHandler
- It is an abstract class.
- There is a default implementation (DefaultAuthorizationHandler) in Sqoop-security component.
- It handles two singleton instances, AccessController and AuthorizationValidator.
- All function will be delegated to these two instances to handle. AccessController to handle grantRole, revokeRole, grantPrivilege and revokePrivilege. AuthorizationValidator to handle checkPrivilege.
- AccessController
- It is an abstract class.
- There is a default implementation (DefaultAccessController) in Sqoop-security component.
- This class is responsible to manage roles, privileges.
- AuthorzationValidator
- It is an abstract class.
- There is a default implementation (DefaultAuthorizationValidator) in Sqoop-security component.
- This class is responsible to check privileges.
- AuthorizationManager
- Three classes will be added into Sqoop-security as org.apache.sqoop.security package.
- DefaultAuthorizationHandler
- This class extends abstract AuthorizationHandler.
- It handles two singleton instances, DefaultAccessController and DefaultAuthorizationValidator.
- DefaultAccessController
- This class extends abstract AccessController.
- Default AuthorzationValidator
- This class extends abstract AuthorizationValidator.
- As default/simple implementation, it always returns true and will not check the privilege actually.
- DefaultAuthorizationHandler
- All functions in RequestHandler, which handles all requests, ie. create link, will be added privilege validation check.
/** * Create or Update link in repository. * * @param ctx Context object * @return Validation bean object */ private JsonBean createUpdateLink(RequestContext ctx, boolean create) { AuthorizationEngine.createLinkPrivilige(); ...... }
- Privilege check request will be analyzed by AuthorizationEngine.
Override public void createLinkPrivilige() throws SqoopAccessControlException { List<Privilege> privileges; privileges.add(new Privilege(new Resource("Link", "1"), "Create", null)); privileges.add(new Privilege(new Resource("Connector", "1"), "Use", null)); AuthorizationManager.getAuthenticationHandler.checkPrivileges(privileges); }
- Privilege check will be passed to real AccessController from AuthorizationHandler.
@Override public void checkPrivileges(List<principal> principals) throws SqoopAccessControlException { authValidator.checkPrivileges(principals); }
Command line tool
- The grant/revoke privilege should be run in command line in Sqoop client
- The commands are showed below
show role show role -role_name admin show role -principal_name sqoop -principal_type user show role -privilege_name my_privilege add role –role_name admin update role -old_role_name admin -new_role_name developer remove role -role_name admin show principal show principal -principal_name sqoop show principal -principal_type user show principal -principal_name sqoop -principal_type user show principal -role_name admin add principal –principal_name sqoop -principal_type user update principal -old_principal_name sqoop -old_principal_type user -new_principal_name hadoop -new_principal_type group remove principal -principal_name sqoop remove principal -principal_type user remove principal -principal_name sqoop -principal_type user grant role –role_name admin -principal_name sqoop -principal_type user revoke role –role_name admin -principal_name sqoop -principal_type user show resource show resource -resource_type link show resource -resource_name 1 -resource_type link add resource -resource_name 1 -resource_type link update resource -old_resource_name 1 -old_resource_type link -new_resource_name 2 -new_resource_type link remove resource -resource_type link remove resource -resource_name 1 -resource_type link show privilege show privilege -privilege_name my_privilege show privilege -resource_name 1 -resource_type link show privilege -role_name admin add privilege -privilege_name my_privilege -resource_name 1 -resource_type link -action create -with_grant_option true update privilege -old_privilege_name my_privilege -old_resource_name 1 -old_resource_type link -old_action create -old_with_grant_option true -new_privilege_name others_privilege -new_resource_name 1 -new_resource_type link -new_action create -new_with_grant_option true remove privilege -privilege_name my_privilege remove privilege -resource_name 1 -resource_type link grant privilege –role_name admin -privilege_name my_privilege revoke privilege –role_name admin -privilege_name my_privilege
- Restful call API is handled by org.apache.sqoop.handler.AuthorizationEngine.java in sqoop-server
- GET /v1/role/{role_name}
- Return details about one particular role with role_name
- Return all of them if role_name is null
- GET /v1/role/principal/name/{name}/type/{type}
- Return all roles the particular principal has with principal (name, type)
- GET /v1/role/privilege/{privilege_name}
- Return all roles with the particular privilege (privilege_name)
- PUT /v1/role
- Create new role with role_name. Put data of JsonObject role (role_name)
- Update existing role with old_role_name and new_role_name. Put data of JsonObject old_role (role_name) and new_role (role_name)
- DELETE /v1/role/{role_name}
- GET /v1/principal/name/{name}/type/{type}
- Return details about one particular principal with name and type
- Return all of them if name is null or type is null or both are null
- GET /v1/principal/role/{role_name}
- Return all principals with the particular role (role_name)
- PUT /v1/principal
- Create new principal with name and type. Put data of JsonObject principal (name, type)
- Update existing principal with old_name, old_type, new_name, new_type. Put data of JsonObject old_principal (name, type) and new_principal (name, type)
- DELETE /v1/principal/name/{name}/type/{type}
- Delete all of them if name is null or type is null
- PUT /v1/grant_role
- Grant a role to a user/group/role
- PUT data of JsonObject role(role_name) and principal (name, type)
- PUT /v1/revoke_role
- Revoke a role to a user/group/role
- PUT data of JsonObject role(role_name) and principal (name, type)
- GET /v1/resource/name/{name}/type/{type}
- Return details about one particular resource with name and type
- Return all of them if name is null or type is null or both are null
- PUT /v1/resource
- Create new resource with name and type. Put data of JsonObject resource (name, type)
- Update existing resource with old_name, old_type, new_name, new_type. Put data of JsonObject old_resource (name, type) and new_resource (name, type)
- DELETE /v1/resource/name/{name}/type/{type}
- Delete all of them if name is null or type is null
- GET /v1/privilege/{privilege_name}
- Return details about one particular privilege with privilege_name
- Return all of them if name is null or type is null or both are null
- GET /v1/privilege/resource/name/{name}/type/{type}
- Return all privilege with the particular principal (name, type)
- GET /v1/privilege/role/{role_name}
- Return all privilege with the particular role (role_name)
- PUT /v1/privilege
- Create new privilege with resource, action and with_grant_option. Put data of JsonObject resource (name, type), action and with_grant_option
- Update existing privilege with old_privilege_name, old_resource, old_action, old_with_grant_option, new_privilege_name, new_resource, new_action and new_with_grant_option. Put data of JsonObject old_privilege (name, resource (name, type), action, with_grant_option) and new_privilege (name, resource (name, type), action, with_grant_option)
- DELETE /v1/privilege/{privilege_name}
- DELETE /v1/resource/name/{name}/type/{type}
- PUT /v1/grant_privilege
- Grant a privilege to a role
- PUT data of JsonObject role(role_name) and privilege (name)
- PUT /v1/revoke_privilege
- Revoke a privilege to a role
- PUT data of JsonObject role(role_name) and privilege (name)
- GET /v1/role/{role_name}
Sentry implementation
- Sentry could be used as an alternative access controller
- Config in sqoop.properties
#org.apache.sqoop.authorization.handler=org.apache.sqoop.security.SentryAuthorizationHandler #org.apache.sqoop.authorization.controller=org.apache.sqoop.security.SentryAccessController #org.apache.sqoop.authorization.validator=org.apache.sqoop.security.SentryAuthorizationValidator
- Use Sentry to check access privilege
- Set access privilege using hue (optional)
Database design
- Role table
- Id
- Name
- Comment
- Role name could be admin, developer, user, etc.
- Role_User_Group table
- Id
- Role_id
- User_name
- Group_name
- Comment
- The information of user and group comes from Linux or LDAP etc.
- Only one of user name and group name is set. If user name is set and leave group name empty, it means that this user has this rule. If group name is set and leave user name empty, it means that all users in this group has this rule.
- One user/group could have one or multiple roles.
- Privilege table
- Id
- Role_id
- Resource_id
- Resource_type
- Action_type
- Comment
- Resource type could be the existing resource table, such as connector, link, job, submission, etc.
- Resource type could be added in the future, say config etc.
- If resource_id is 0, it means all resource of this type, ie. resource_id=0 and resource_type=link means all links.
- Use resource id and resource type to identify the resource, ie. resource_id=1 and resource_type=link means the resource of “select * from link where id =1”.
- Action type could be read, create, update, delete, use etc.
- Accordingly, MRole, MRoleUserGroup and MPrivilege classes are added into package org.apache.sqoop.model.