Ranger 0.6 supports authorization of access based on tags associated with the resources, in addition to resource-based access authorization. Tag-based policy model offers many advantages over resource-based authorization model. One of the important advantages of this model is that it supports separation of resource-classification from access-authorization. This, in turn, allows security administrators to conceptualize and author access policies across multiple components (such as HDFS, Hive) in terms of type/class of data (which is a higher-level abstraction) as against in terms of component-specific resources as required by resource-based model.
Ranger tag policy model provides structures and abstractions to express and enforce tag-based policies. However, the tags (which embody a class of data contained by resources) are provisioned by an external system that maintains meta-data about the resources. This necessitates a tool to keep the external tag-source and Ranger Admin in synchronization.
Ranger suite includes a module, called Tag-Sync – short for Tag Synchronization Module - that enables synchronization of tagging information (such as entity-tag association as well value of tag-attributes if any) between a tag-source (usually, Atlas in the Apache-DGI ecosystem) and Ranger Admin.
Tag-sync module does not concern itself with how the entities are associated with tags and how the tag-attributes are initialized with values. It only ensures that any notifications regarding change to any entity-tag association and to tag-attribute values, are received and Ranger Admin is updated accordingly.
Tag-Sync module is implemented within Ranger as a stand-alone, daemon process named ranger-tagsync.
Installation
Tag-Sync is part of Apache-Ranger project distribution.
To install Tag-Sync:
Log in to the installation machine as a super-user (root)
Locate and download ranger-tagsync package and install it.
This will unpack and create a directory and needed sub-directories and files in “ranger-tagsync” directory under Apache installation root directory. This is ranger-tagsync-install-directory.
Make ranger-tagsync-install-directory as your current working directory.
% cd ranger-tagsync-install-directory
Set up the JAVA_HOME environment variable to point to Java distribution on the installation machine.
Edit “install.properties” file in ranger-tagsync-install-directory to suit the operational environment.
Contents of this file are described in Configuration section in this document.
Run setup script to create run-time directory structure.
% ./setup.sh
The run-time directory structure copies scripts into appropriate directories, creates files required for Atlas integration, creates configuration files which are used by ranger-tagsync process to configure itself, and sets up symbolic links wherever necessary.
Configuration
Tag-Sync configuration consists of providing property values for controlling following aspects of the module.
Configuration of the source system (such as Atlas)
Configuration of target of tags (Ranger Admin, also referred to as TagAdmin)
Ranger-tagsync process control
The property values are provided by the Tag-Sync installer, in a simple file, named “install.properties”, in “name=value” format.
“install.properties” file contains the following properties.
PROPERTY
DESCRIPTION
PROPERTY_NAME in ranger-tagsync-site.xml
COMMENTS
TAG_DEST_RANGER_ENDPOINT
URL of the destination of tags
ranger.tagsync.dest.ranger.endpoint
Please customize the value to suit your deployment. (Default value: ‘http://localhost:6080’
)
TAGSYNC_KEYSTORE_FILENAME
File containing encrypted Ranger Admin password
ranger.tagsync.keystore.filename
Please customize the value to suit your deployment. (Default value: ‘/etc/ranger/tagsync/conf/rangertagsync.jceks’)
TAG_DEST_RANGER_SSL_CONFIG_FILENAME
File containing SSL Configuration
ranger.tagsync.dest.ranger.ssl.config.filename
Please customize the value to point to SSL configuration specific to your deployment. The value will be ignored in the scheme in TAGADMIN_ENPOINT URL is ‘http’. (Default value: ‘’)
TAG_SOURCE_ATLAS_ENABLED
Source of the tags
ranger.tagsync.source.atlas
True if source of tags is ‘atlas’. This is used if Atlas events are the source of tags. (Default value: ‘True’)
TAG_SOURCE_ATLAS_KAFKA_BOOTSTRAP_SERVERS
URL of the Kafka endpoint to which Atlas sends its notifications.
atlas.kafka.bootstrap.servers (in atlas-application.properties)
Please customize the value to suit your deployment. (Default value: ‘localhost:6667‘)
TAG_SOURCE_ATLAS_KAFKA_ZOOKEEPER_CONNECT
URL of the zookeeper endpoint needed for Atlas.
atlas.kafka.zookeeper.connect (in atlas-application.properties)
Please customize the value to suit your deployment. (Default value: ‘localhost:2181‘)
TAG_SOURCE_ATLAS_KAFKA_ENTITIES_GROUP_ID
String representing Kafka Consumer Group id used by Tag-Sync.
atlas.kafka.entities.group.id (in atlas-application.properties)
Please customize the value to suit your deployment. (Default value: ‘ranger_entities_consumer‘)
TAG_SOURCE_ATLAS_KAFKA_SERVICE_NAME
Kafka service name used by Atlas
atlas.kafka.sasl.kerberos.service.name (in atlas-application.properties)
Please customize the value to suit your deployment. (Default value: ‘kafka‘)
TAG_SOURCE_ATLAS_KAFKA_SECURITY_PROTOCOL
Kafka security protocol used in kerberized cluster
atlas.kafka.sasl.kerberos.security.protocol (in atlas-application.properties)
Please customize the value to suit your deployment. (Default value: ‘SASL_PLAINTEXT‘)
TAG_SOURCE_ATLAS_KAFKA_KERBEROS_PRINCIPAL
Kerberos principal presented to Kafka
atlas.jaas.kafkaClient.option.principal (in atlas-application.properties)
Please customize the value to suit your deployment.
TAG_SOURCE_ATLAS_KAFKA_KERBEROS_KEYTAB
Kerberos keytab presented to Kafka
atlas.jaas.kafkaClient.option.keyTab (in atlas-application.properties)
Please customize the value to suit your deployment.
TAG_SOURCE_ATLASREST_ENABLED
Source of tags
ranger.tagsync.source.atlasrest
True if source of tags is ‘atlasrest’ – This is used if tags are downloaded from Atlas.
(Default value: ‘False’)
TAG_SOURCE_ATLASREST_ENDPOINT
URL of the Atlas Endpoint. If TAG_SOURCE_ATLASREST_ENABLED is true, then this needs to be set.
ranger.tagsync.source.atlasrest.endpoint
Please customize the value to suit your deployment. (Default value: ‘http://localhost:21000’)
TAG_SOURCE_ATLASREST_DOWNLOAD_INTERVAL_IN_MILLIS
Number of milliseconds between successive downloads of tags from Atlas when TAG_SOURCE_ATLASREST_ENABLED is true.
Please customize the value to suit your deployment. (Default value: ‘’)
TAGSYNC_ATLAS_CUSTOM_RESOURCE_MAPPERS
Used to extend ranger-tagsync to support tags for components other than Hive.
ranger.tagsync.atlas.custom.resource.mappers
For future use. Currently, the value is empty. (Default value: ‘’)
TAGSYNC_KEYSTORE_FILENAME
File to store encrypted password for Ranger Admin user ‘rangertagsync’ that is used for communicating with Ranger
ranger.tagsync.keystore.filename
Please customize to suit your deployment. (Default value: ‘/etc/ranger/tagsync/conf/rangertagsync.jceks’)
unix_user
ranger-tagsync process is run as this Unix user.
unix_user
There is no need to change this value (default is ‘ranger’).
unix_group
ranger-tagsync process is run with this Unix group-id.
unix_group
There is no need to change this value (default is ‘ranger’).
logdir
Directory where logs are stored.
ranger.tagsync.logdir
Please customize the value to suit your deployment. (default is ‘log’)
is_secure
Indicates if it is secure or simple cluster; 'True' is secure; 'False' if simple.
Please customize the value to suit your deployment. (default is ‘False’)
tagsync_principal
Kerberos Principal
ranger.tagsync.kerberos.principal
Please customize the value to suit your deployment
tagsync_keytab
Kerberos keytab
ranger.tagsync.kerberos.keytab
Please customize the value to suit your deployment.
hadoop_conf
Hadoop configuration directory.
hadoop_conf
Please customize the value to suit your deployment. (Default value: ‘/etc/hadoop/conf
‘)
Updating password for Ranger Admin user ‘rangertagsync’ and/or for Atlas user
If, after installation, it is desired to change password of ‘rangertagsync’ user, then
Change the password using Ranger Admin, AND,
If password is stored in encrypted form, run updatetagadminpassword.sh in ranger-tagsync-install-directory and provide 'Ranger' as the destination to update the password in the TAGSYNC_KEYSTORE_FILENAME keystore file; else update /etc/ranger/tagsync/conf/ranger-tagsync-site.xml file manually to add/update ‘ranger.tagsync.tagadmin.password’ property appropriately.
Similarly, to change username and password for Atlas user (used if tag-source is 'atlasrest')
If the password is stored in encrypted form, run updatetagadminpassword.sh script in ranger-tagsync-install-directory and provide 'Atlas' as the destination to update the password in TAG_SOURCE_ATLASREST_KEYSTORE_FILENAME keystore file; else update /etc/ranger/tagsync/conf/ranger-tagsync-site.xml file manually to add/update 'ranger.tagsync.source.atlasrest.password' property appropriately.
Run-time structure
A run-time directory structure is created after configuring property values in install.properties, and then executing setup.sh script.
Directory
File
Linked to
Contains
/etc/ranger/tagsync/conf
ranger-tagsync-install-directory/conf
/etc/ranger/tagsync/conf
atlas-application.properties
Properties required for Atlas interface
/etc/ranger/tagsync/conf
java_home.sh
Shell command to set JAVA_HOME environment variable
/etc/ranger/tagsync/conf
log4j.xml
Log4j configuration properties
/etc/ranger/tagsync/conf
ranger-tagsync-site.xml
Ranger-tagsync process configuration properties
/etc/ranger/tagsync/conf
rangertagsync.jceks
Encrypted password for Ranger Admin user ‘rangertagsync’
/etc/init.d
ranger-tagsync
Shell script to start/stop/query ranger-tagsync service
Directory where logs (ranger-tagsync.log*) generated by ranger-tagsync are stored.
/var/run/ranger
tagsync.pid
File containing process-id of the ranger-tagsync process, if it is running
Usage
ranger-tagsync service may be started after Tag-Sync module is installed and configured.
To check if ranger-tagsync is running, log in as a super-user and execute
% service ranger-tagsync status
To start ranger-tagsync process, log in as a super-user and execute
% service ranger-tagsync start
To stop ranger-tagsync process, log in as a super-user and execute
% service ranger-tagsync stop
Ambari integration
At present, Tag-Sync module is not integrated with Ambari. It needs to be installed, configured and executed separately from rest of the modules in the Ranger suite using the command described in this document.
For integrating Tag-Sync with Ambari, a Ambari-interface file containing the properties in ranger-tagsync-site.xml and ranger-tagsync-default.xml (which is in the .jar file for ranger-tagsync) files (for configuring ranger-tagsync process) and properties in application.properties (for configuring Atlas interface) needs to set up.
2 Comments
Abhay Kulkarni
Overview
Ranger 0.6 supports authorization of access based on tags associated with the resources, in addition to resource-based access authorization. Tag-based policy model offers many advantages over resource-based authorization model. One of the important advantages of this model is that it supports separation of resource-classification from access-authorization. This, in turn, allows security administrators to conceptualize and author access policies across multiple components (such as HDFS, Hive) in terms of type/class of data (which is a higher-level abstraction) as against in terms of component-specific resources as required by resource-based model.
Ranger tag policy model provides structures and abstractions to express and enforce tag-based policies. However, the tags (which embody a class of data contained by resources) are provisioned by an external system that maintains meta-data about the resources. This necessitates a tool to keep the external tag-source and Ranger Admin in synchronization.
Ranger suite includes a module, called Tag-Sync – short for Tag Synchronization Module - that enables synchronization of tagging information (such as entity-tag association as well value of tag-attributes if any) between a tag-source (usually, Atlas in the Apache-DGI ecosystem) and Ranger Admin.
Tag-sync module does not concern itself with how the entities are associated with tags and how the tag-attributes are initialized with values. It only ensures that any notifications regarding change to any entity-tag association and to tag-attribute values, are received and Ranger Admin is updated accordingly.
Tag-Sync module is implemented within Ranger as a stand-alone, daemon process named ranger-tagsync.
Installation
Tag-Sync is part of Apache-Ranger project distribution.
To install Tag-Sync:
This will unpack and create a directory and needed sub-directories and files in “ranger-tagsync” directory under Apache installation root directory. This is ranger-tagsync-install-directory.
% cd ranger-tagsync-install-directory
% export JAVA_HOME=location-of-java-home-on-the-machine
Contents of this file are described in Configuration section in this document.
% ./setup.sh
The run-time directory structure copies scripts into appropriate directories, creates files required for Atlas integration, creates configuration files which are used by ranger-tagsync process to configure itself, and sets up symbolic links wherever necessary.
Configuration
Tag-Sync configuration consists of providing property values for controlling following aspects of the module.
The property values are provided by the Tag-Sync installer, in a simple file, named “install.properties”, in “name=value” format.
“install.properties” file contains the following properties.
PROPERTY
DESCRIPTION
PROPERTY_NAME in ranger-tagsync-site.xml
COMMENTS
TAG_DEST_RANGER_ENDPOINT
URL of the destination of tags
ranger.tagsync.dest.ranger.endpoint
)
TAG_DEST_RANGER_SSL_CONFIG_FILENAME
File containing SSL Configuration
Please customize the value to point to SSL configuration specific to your deployment. The value will be ignored in the scheme in TAGADMIN_ENPOINT URL is ‘http’. (Default value: ‘’)
TAG_SOURCE_ATLAS_ENABLED
Source of the tags
True if source of tags is ‘atlas’. This is used if Atlas events are the source of tags. (Default value: ‘True’)
URL of the Kafka endpoint to which Atlas sends its notifications.
URL of the zookeeper endpoint needed for Atlas.
String representing Kafka Consumer Group id used by Tag-Sync.
TAG_SOURCE_ATLASREST_ENABLED
Source of tags
True if source of tags is ‘atlasrest’ – This is used if tags are downloaded from Atlas.
(Default value: ‘False’)
URL of the Atlas Endpoint. If TAG_SOURCE_ATLASREST_ENABLED is true, then this needs to be set.
Number of milliseconds between successive downloads of tags from Atlas when TAG_SOURCE_ATLASREST_ENABLED is true.
Please customize the value to suit your deployment. (Default value: ‘90000’)
TAG_SOURCE_FILE_ENABLED
Source of tags
True if source of tags is ‘file’. A sample file format is available at /etc/ranger/tagsync/conf/etc/ranger/data/tags.json. (Default value: ‘False’)
File name containing tags if TAG_SOURCE_FILE_ENABLED is true.
Number of milliseconds between checks for changes to TAGSYNC_SOURCE_FILE_FILENAME if TAG_SOURCE_FILE_ENABLED is true.
Please customize the value to suit your deployment. (Default value: ‘60000’)
TAGSYNC_ATLAS_TO_RANGER_SERVICE_MAPPING
Mapping between Atlas cluster-name, component-type and Ranger service-name
This mapping is provided as a string value in the following format.
clusterName,componentType,serviceName;clusterName2,componentType2,serviceName;..,
Please customize the value to suit your deployment. (Default value: ‘’)
TAGSYNC_ATLAS_CUSTOM_RESOURCE_MAPPERS
Used to extend ranger-tagsync to support tags for components other than Hive.
For future use. Currently, the value is empty. (Default value: ‘’)
TAGSYNC_KEYSTORE_FILENAME
File to store encrypted password for Ranger Admin user ‘rangertagsync’ that is used for communicating with Ranger
unix_user
ranger-tagsync process is run as this Unix user.
There is no need to change this value (default is ‘ranger’).
unix_group
ranger-tagsync process is run with this Unix group-id.
There is no need to change this value (default is ‘ranger’).
logdir
Directory where logs are stored.
Please customize the value to suit your deployment. (default is ‘log’)
Kerberos Principal
Please customize the value to suit your deployment
Kerberos keytab
Please customize the value to suit your deployment.
Hadoop configuration directory.
‘)
Updating password for Ranger Admin user ‘rangertagsync’ and/or for Atlas user
If, after installation, it is desired to change password of ‘rangertagsync’ user, then
Similarly, to change username and password for Atlas user (used if tag-source is 'atlasrest')
Run-time structure
A run-time directory structure is created after configuring property values in install.properties, and then executing setup.sh script.
Directory
File
Linked to
Contains
/etc/ranger/tagsync/conf
ranger-tagsync-install-directory/conf
/etc/ranger/tagsync/conf
atlas-application.properties
Properties required for Atlas interface
/etc/ranger/tagsync/conf
java_home.sh
Shell command to set JAVA_HOME environment variable
/etc/ranger/tagsync/conf
log4j.xml
Log4j configuration properties
/etc/ranger/tagsync/conf
ranger-tagsync-site.xml
Ranger-tagsync process configuration properties
/etc/ranger/tagsync/conf
rangertagsync.jceks
Encrypted password for Ranger Admin user ‘rangertagsync’
/etc/init.d
ranger-tagsync
Shell script to start/stop/query ranger-tagsync service
/usr/bin
ranger-tagsync-services.sh
ranger-tagsync-install-directory/ranger-tagsync.sh
Shell script to start/stop ranger-tagsync process
/var/log/ranger/tagsync
ranger-tagsync-install-directory/log
Directory where logs (ranger-tagsync.log*) generated by ranger-tagsync are stored.
/var/run/ranger
tagsync.pid
File containing process-id of the ranger-tagsync process, if it is running
Usage
ranger-tagsync service may be started after Tag-Sync module is installed and configured.
To check if ranger-tagsync is running, log in as a super-user and execute
% service ranger-tagsync status
To start ranger-tagsync process, log in as a super-user and execute
% service ranger-tagsync start
To stop ranger-tagsync process, log in as a super-user and execute
% service ranger-tagsync stop
Ambari integration
At present, Tag-Sync module is not integrated with Ambari. It needs to be installed, configured and executed separately from rest of the modules in the Ranger suite using the command described in this document.
For integrating Tag-Sync with Ambari, a Ambari-interface file containing the properties in ranger-tagsync-site.xml and ranger-tagsync-default.xml (which is in the .jar file for ranger-tagsync) files (for configuring ranger-tagsync process) and properties in application.properties (for configuring Atlas interface) needs to set up.
Shivaji Dutta
Locate and download ranger-tagsync package and install it.