Page tree
Skip to end of metadata
Go to start of metadata
  • No labels

2 Comments

  1. Overview

     

    Ranger 0.6 supports authorization of access based on tags associated with the resources, in addition to resource-based access authorization. Tag-based policy model offers many advantages over resource-based authorization model. One of the important advantages of this model is that it supports separation of resource-classification from access-authorization. This, in turn, allows security administrators to conceptualize and author access policies across multiple components (such as HDFS, Hive) in terms of type/class of data (which is a higher-level abstraction) as against in terms of component-specific resources as required by resource-based model.

     

    Ranger tag policy model provides structures and abstractions to express and enforce tag-based policies. However, the tags (which embody a class of data contained by resources) are provisioned by an external system that maintains meta-data about the resources. This necessitates a tool to keep the external tag-source and Ranger Admin in synchronization.

     

    Ranger suite includes a module, called Tag-Sync – short for Tag Synchronization Module - that enables synchronization of tagging information (such as entity-tag association as well value of tag-attributes if any) between a tag-source (usually, Atlas in the Apache-DGI ecosystem) and Ranger Admin.

     

    Tag-sync module does not concern itself with how the entities are associated with tags and how the tag-attributes are initialized with values. It only ensures that any notifications regarding change to any entity-tag association and to tag-attribute values, are received and Ranger Admin is updated accordingly.

     

    Tag-Sync module is implemented within Ranger as a stand-alone, daemon process named ranger-tagsync.


    Installation

     

    Tag-Sync is part of Apache-Ranger project distribution.

     

    To install Tag-Sync:

    • Log in to the installation machine as a super-user (root)
    • Locate and download ranger-tagsync package and install it.

     

    This will unpack and create a directory and needed sub-directories and files in “ranger-tagsync” directory under Apache installation root directory.  This is ranger-tagsync-install-directory

    • Make ranger-tagsync-install-directory as your current working directory.

    % cd ranger-tagsync-install-directory

     

    • Set up the JAVA_HOME environment variable to point to Java distribution on the installation machine.

     

    % export JAVA_HOME=location-of-java-home-on-the-machine

     

    • Edit “install.properties” file in ranger-tagsync-install-directory to suit the operational environment.

     

    Contents of this file are described in Configuration section in this document.

     

    • Run setup script to create run-time directory structure.

     

    % ./setup.sh

     

    The run-time directory structure copies scripts into appropriate directories, creates files required for Atlas integration, creates configuration files which are used by ranger-tagsync process to configure itself, and sets up symbolic links wherever necessary. 

     

    Configuration

     

    Tag-Sync configuration consists of providing property values for controlling following aspects of the module.

     

    • Configuration of the source system (such as Atlas)
    • Configuration of target of tags (Ranger Admin, also referred to as TagAdmin)
    • Ranger-tagsync process control

     

     

    The property values are provided by the Tag-Sync installer, in a simple file, named “install.properties”, in “name=value” format.

     

    “install.properties” file contains the following properties.

    PROPERTY

    DESCRIPTION

    PROPERTY_NAME in ranger-tagsync-site.xml

    COMMENTS

    TAG_DEST_RANGER_ENDPOINT

    URL of the destination of tags

       ranger.tagsync.dest.ranger.endpoint

     

    Please customize the value to suit your deployment. (Default value: http://localhost:6080

    )

    TAGSYNC_KEYSTORE_FILENAME
    File containing encrypted Ranger Admin password ranger.tagsync.keystore.filenamePlease customize the value to suit your deployment. (Default value: /etc/ranger/tagsync/conf/rangertagsync.jceks)

    TAG_DEST_RANGER_SSL_CONFIG_FILENAME

     

    File containing SSL Configuration

       ranger.tagsync.dest.ranger.ssl.config.filename

     

    Please customize the value to point to SSL configuration specific to your deployment. The value will be ignored in the scheme in TAGADMIN_ENPOINT URL is ‘http’. (Default value: ‘’)

    TAG_SOURCE_ATLAS_ENABLED

     

    Source of the tags

       ranger.tagsync.source.atlas

     

    True if source of tags is ‘atlas’. This is used if Atlas events are the source of tags. (Default value: ‘True’)

    TAG_SOURCE_ATLAS_KAFKA_BOOTSTRAP_SERVERS

     

    URL of the Kafka endpoint to which Atlas sends its notifications.

       atlas.kafka.bootstrap.servers (in atlas-application.properties)

     

    Please customize the value to suit your deployment. (Default value: ‘localhost:6667‘)
    TAG_SOURCE_ATLAS_KAFKA_ZOOKEEPER_CONNECT

     

    URL of the zookeeper endpoint needed for Atlas.

       atlas.kafka.zookeeper.connect (in atlas-application.properties)

     

    Please customize the value to suit your deployment. (Default value: localhost:2181‘)
    TAG_SOURCE_ATLAS_KAFKA_ENTITIES_GROUP_ID

                                                                           

    String representing Kafka Consumer Group id used by Tag-Sync.

       atlas.kafka.entities.group.id (in atlas-application.properties)

     

    Please customize the value to suit your deployment. (Default value: ‘ranger_entities_consumer‘)
    TAG_SOURCE_ATLAS_KAFKA_SERVICE_NAMEKafka service name used by Atlas
     atlas.kafka.sasl.kerberos.service.name (in atlas-application.properties)
     Please customize the value to suit your deployment. (Default value: ‘kafka‘)
    TAG_SOURCE_ATLAS_KAFKA_SECURITY_PROTOCOLKafka security protocol used in kerberized cluster
     atlas.kafka.sasl.kerberos.security.protocol (in atlas-application.properties)
    Please customize the value to suit your deployment. (Default value: ‘SASL_PLAINTEXT‘)
    TAG_SOURCE_ATLAS_KAFKA_KERBEROS_PRINCIPALKerberos principal presented to Kafka
     atlas.jaas.kafkaClient.option.principal (in atlas-application.properties)
    Please customize the value to suit your deployment.
    TAG_SOURCE_ATLAS_KAFKA_KERBEROS_KEYTABKerberos keytab presented to Kafka
     atlas.jaas.kafkaClient.option.keyTab (in atlas-application.properties)
    Please customize the value to suit your deployment.

    TAG_SOURCE_ATLASREST_ENABLED

     

    Source of tags

       ranger.tagsync.source.atlasrest

    True if source of tags is ‘atlasrest’ – This is used if tags are downloaded from Atlas.

     (Default value: ‘False’)

    TAG_SOURCE_ATLASREST_ENDPOINT

     

    URL of the Atlas Endpoint. If TAG_SOURCE_ATLASREST_ENABLED is true, then this needs to be set.

       ranger.tagsync.source.atlasrest.endpoint

     

    Please customize the value to suit your deployment. (Default value: ‘http://localhost:21000)
    TAG_SOURCE_ATLASREST_DOWNLOAD_INTERVAL_IN_MILLIS

     

    Number of milliseconds between successive downloads of tags from Atlas when TAG_SOURCE_ATLASREST_ENABLED is true.

       ranger.tagsync.source.atlasrest.download.interval.millis

     

    Please customize the value to suit your deployment. (Default value: ‘90000’)

    TAG_SOURCE_ATLASREST_USERNAME
    Username of Atlas user for downloading tag information ranger.tagsync.source.atlasrest.usernamePlease customize the value to suit your deployment. (Default value: ‘admin’)
    TAG_SOURCE_ATLASREST_PASSWORD
    Password of Atlas user ranger.tagsync.source.atlasrest.password 
    TAG_SOURCE_ATLASREST_KEYSTORE_FILENAME
    File containing encrypted Atlas password ranger.tagsync.source.atlasrest.keystore.filenamePlease customize the value to suit your deployment. (Default value: /etc/ranger/tagsync/conf/atlasuser.jceks)
    TAG_SOURCE_ATLASREST_SSL_CONFIG_FILENAME
    File containing SSL Configuration ranger.tagsync.source.atlasrest.ssl.config.filenamePlease customize the value to suit your deployment. (Default value: ‘’)

    TAG_SOURCE_FILE_ENABLED

     

    Source of tags

       ranger.tagsync.source.file
     

    True if source of tags is ‘file’. A sample file format is available at /etc/ranger/tagsync/conf/etc/ranger/data/tags.json. (Default value: ‘False’)

    TAG_SOURCE_FILE_FILENAME

     

    File name containing tags if TAG_SOURCE_FILE_ENABLED is true.

       ranger.tagsync.source.file.filename

     

    Please customize the value to suit your deployment. (Default value: /etc/ranger/data/tags.json)

     

    TAG_SOURCE_FILE_CHECK_INTERVAL_IN_MILLIS

     

    Number of milliseconds between checks for changes to TAGSYNC_SOURCE_FILE_FILENAME  if TAG_SOURCE_FILE_ENABLED is true.

       ranger.tagsync.source.file.check.interval.millis

     

    Please customize the value to suit your deployment. (Default value: ‘60000’)

    TAGSYNC_ATLAS_TO_RANGER_SERVICE_MAPPING

     

    Mapping between Atlas cluster-name, component-type and Ranger service-name

      The property name is generated dynamically based on value of this property
     

    This mapping is provided as a string value in the following format.

    clusterName,componentType,serviceName;clusterName2,componentType2,serviceName;..,

    Please customize the value to suit your deployment. (Default value: ‘’)

    TAGSYNC_ATLAS_CUSTOM_RESOURCE_MAPPERS

     

    Used to extend ranger-tagsync to support tags for components other than Hive.

      ranger.tagsync.atlas.custom.resource.mappers
     

    For future use. Currently, the value is empty. (Default value: ‘’)

    TAGSYNC_KEYSTORE_FILENAME

     

    File to store encrypted password for  Ranger Admin user ‘rangertagsync’ that is used for communicating with Ranger

      ranger.tagsync.keystore.filename
     
    Please customize to suit your deployment. (Default value: /etc/ranger/tagsync/conf/rangertagsync.jceks)

    unix_user

     

    ranger-tagsync process is run as this Unix user.

       unix_user
     

    There is no need to change this value (default is ‘ranger’).

    unix_group

     

    ranger-tagsync process is run with this Unix group-id.

       unix_group
     

    There is no need to change this value (default is ‘ranger’).

    logdir

    Directory where logs are stored.

       ranger.tagsync.logdir
     

    Please customize the value to suit your deployment. (default is ‘log’)

    is_secure
    Indicates if it is secure or simple cluster; 'True' is secure; 'False' if simple. Please customize the value to suit your deployment. (default is ‘False’)
    tagsync_principal

     

    Kerberos Principal

       ranger.tagsync.kerberos.principal
     

    Please customize the value to suit your deployment

    tagsync_keytab

     

    Kerberos keytab

       ranger.tagsync.kerberos.keytab
     

    Please customize the value to suit your deployment.

    hadoop_conf

     

    Hadoop configuration directory.

       hadoop_conf
     
    Please customize the value to suit your deployment. (Default value: /etc/hadoop/conf

    ‘)

     

    Updating password for Ranger Admin user ‘rangertagsync’ and/or for Atlas user

     

    If, after installation, it is desired to change password of ‘rangertagsync’ user, then

    • Change the password using Ranger Admin, AND,
    • If password is stored in encrypted form, run updatetagadminpassword.sh in ranger-tagsync-install-directory and provide 'Ranger' as the destination to update the password in the TAGSYNC_KEYSTORE_FILENAME  keystore file; else update /etc/ranger/tagsync/conf/ranger-tagsync-site.xml file manually to add/update ‘ranger.tagsync.tagadmin.password’ property appropriately.

    Similarly, to change username and password for Atlas user (used if tag-source is 'atlasrest')

    • If the password is stored in encrypted form, run updatetagadminpassword.sh script in ranger-tagsync-install-directory and provide 'Atlas' as the destination to update the password in TAG_SOURCE_ATLASREST_KEYSTORE_FILENAME keystore file; else update /etc/ranger/tagsync/conf/ranger-tagsync-site.xml file manually to add/update 'ranger.tagsync.source.atlasrest.password' property appropriately.


    Run-time structure

    A run-time directory structure is created after configuring property values in install.properties, and then executing setup.sh script.

    Directory

    File

    Linked to

    Contains

    /etc/ranger/tagsync/conf

    ranger-tagsync-install-directory/conf

    /etc/ranger/tagsync/conf

    atlas-application.properties

    Properties required for Atlas interface

    /etc/ranger/tagsync/conf

    java_home.sh

    Shell command to set JAVA_HOME environment variable

    /etc/ranger/tagsync/conf

    log4j.xml

    Log4j configuration properties

    /etc/ranger/tagsync/conf

    ranger-tagsync-site.xml

    Ranger-tagsync process configuration properties

    /etc/ranger/tagsync/conf

    rangertagsync.jceks

    Encrypted password for Ranger Admin user ‘rangertagsync’

    /etc/init.d

    ranger-tagsync

    Shell script to start/stop/query ranger-tagsync service

    /usr/bin

    ranger-tagsync-services.sh

    ranger-tagsync-install-directory/ranger-tagsync.sh

    Shell script to start/stop ranger-tagsync process

    /var/log/ranger/tagsync

    ranger-tagsync-install-directory/log

    Directory where logs (ranger-tagsync.log*) generated by ranger-tagsync are stored.

    /var/run/ranger

    tagsync.pid

    File containing process-id of the ranger-tagsync process, if it is running

     

    Usage

    ranger-tagsync service may be started after Tag-Sync module is installed and configured.

    To check if ranger-tagsync is running, log in as a super-user and execute

    % service ranger-tagsync status

    To start ranger-tagsync process, log in as a super-user and execute

    % service ranger-tagsync start

    To stop ranger-tagsync process, log in as a super-user and execute

    % service ranger-tagsync stop

     

    Ambari integration

    At present, Tag-Sync module is not integrated with Ambari. It needs to be installed, configured and executed separately from rest of the modules in the Ranger suite using the command described in this document.

    For integrating Tag-Sync with Ambari, a Ambari-interface file containing the properties in ranger-tagsync-site.xml and ranger-tagsync-default.xml (which is in the .jar file for ranger-tagsync)  files (for configuring ranger-tagsync process) and properties in application.properties (for configuring Atlas interface) needs to set up. 

  2.  

    Need the following change
    • Locate and download ranger-tagsync package and install it.
    • yum install ranger-tagsync