Page tree
Skip to end of metadata
Go to start of metadata

There is a newer release available

 


 

There is a newer release available

Summary

This document details the steps involved in installing and configuring Apache Ranger. 

These instructions are for installing Ranger on CentOS/RHEL (release 6).

Prerequisites

  • JDK 7+ needs to be installed. Both OpenJDK and Oracle JDK are supported.

  • MySQL (5.6+) or ORACLE DB (11g+) for Policy/Audit DB. <TBD: Link on database settings for Ranger>
    • DB server can be installed on the same host. Or Ranger services need to have access to DB server host
    • For production, ensure appropriate capacity planning is done for the database size
  • Maven. If not installed, please follow below steps
    • wget http://mirrors.gigenet.com/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz (Use different mirror if needed)

    • su -c "tar -zxvf apache-maven-3.0.5-bin.tar.gz -C /opt/"

    • su -c "vi /etc/profile.d/maven.sh" # Add the following lines to maven.sh 

      • export M2_HOME=/opt/apache-maven-3.0.5 

      • export M2=$M2_HOME/bin 

      • export PATH=$M2:$PATH

    • Now test your install of Maven. Logout of the system and then log back into it. Enter the following command:

    • mvn -version
  • Ranger Admin process requires approximately 1.5GB of RAM

Building Ranger from source

  1. Get the ranger source 

  2. Now build the source

    • cd ranger

    • export MAVEN_OPTS="-Xmx512M"

    • yum search gcc and Install gcc (yum install gcc.x86_64)

    • mvn clean compile package assembly:assembly

  3. Verify all the tar files under target dir

    ls -ltr *tar.gz
    -rw-r--r-- 1 root root 15068844 Dec 1 04:30 ranger-0.4.0-hdfs-plugin.tar.gz
    -rw-r--r-- 1 root root 14480716 Dec 1 04:30 ranger-0.4.0-hive-plugin.tar.gz
    -rw-r--r-- 1 root root 14349626 Dec 1 04:30 ranger-0.4.0-hbase-plugin.tar.gz
    -rw-r--r-- 1 root root 17763192 Dec 1 04:30 ranger-0.4.0-knox-plugin.tar.gz
    -rw-r--r-- 1 root root 21243470 Dec 1 04:31 ranger-0.4.0-storm-plugin.tar.gz
    -rw-r--r-- 1 root root 126143540 Dec 1 04:31 ranger-0.4.0-admin.tar.gz
    -rw-r--r-- 1 root root 7677999 Dec 1 04:31 ranger-0.4.0-usersync.tar.gz

Install/Configure Ranger Admin

What is done here?

  • Ranger Admin bits are laid out

  • Database and database users for Ranger Admin policy store and Ranger Audit database are created.

  • Ranger Admin properties are configured

  • Ranger Admin startup service files are created and updated

Steps

  1. Lay down the binaries into appropriate places.  

    • cd /usr/local

    • sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-admin.tar.gz

    • sudo ln -s ranger-0.4.0-admin ranger-admin

  2. cd ranger-admin and open install.properties using your text editor (e.g. vi install.properties)

  3. Verify the root password that you had picked while installing mysql.  I had chosen root so the relevant section in my install.properties file looks as follows 

    • db_root_user=root
    • db_root_password=root
    • db_host=localhost

  4. The install process would create a couple of users in the database for storing administration and audit information, pick passwords for those too.  With my choices here’s how the relevant sections in the install.properties file look now.Ranger allows you to get different authentication modes but for now let’s just leave rest of the things in install.properties file as they are. 

    • # DB information Ranger Policy Store schema
      #
    • db_name=ranger
    • db_user=rangeradmin
    • db_password=rangeradmin
    • # DB UserId for storing auditlog infromation
    • #
    • audit_db_name=ranger (Note the database details used here. This is needed while configuring plugins in later steps)
    • audit_db_user=rangerlogger  
    • audit_db_password=rangerlogger
  5. Once all the required properties are updated, execute the below scripts to install ranger admin service.

        Execute : ./setup.sh (This will configure the properties)
        Execute : ./set_globals.sh (This will create the soft links and other folders for logs, etc)

  6. Create a valid symlink in /usr/bin/ for start/stop of ranger admin
        cd /usr/bin
        ln -sf /usr/local/ranger-admin/ews/start-ranger-admin.sh ranger-admin-start
        ln -sf /usr/local/ranger-admin/ews/stop-ranger-admin.sh ranger-admin-stop

  7. Update ranger-admin service file to link to the start and stop scripts
        vim /etc/init.d/ranger-admin ( Update the Start and Stop commands to point to the created symlinks )

  8. Start the Ranger Admin
         service ranger-admin start

  9. You can verify by visiting the external URL of the server using browser, for example :
        http://<Host Address>:6080/
        Defaut user and password is admin/admin. Please change it post update.

  10. Logs are in ews/logs folder. The path is relative to where you have installed ranger-admin. Check xa_portal.log and catalina.out files for ERROR and WARN log messages

Next Steps

  • Ranger Admin authentication can be configured to use LDAP or Linux system. Consider configuring one of them in production environment. TBD: Provide link to configure LDAP or Linux for authentication

  • Review database capacity for Audit database. It can grow dramatically in HBase or high volume environment. TBD: Provide link DB capacity planning

Install/Configure Ranger User Sync

What is done here?

  • Ranger UserSync bits are laid out

  • Configure Ranger UserSync to synchronize with LDAP/AD or Unix system.

  • Configure Ranger UserSync to be the authentication server for Linux users (optional)

  • Other Ranger UserSync properties are configured

  • Ranger UserSync startup service files are created and updated

Steps

  1. Start by extracting out binaries at the appropriate place. 

        cd /usr/local

        sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-usersync.tar.gz

        sudo ln -s ranger-0.4.0-usersync ranger-usersync

        sudo mkdir -p /var/log/ranger-usersync

        sudo chown ranger /var/log/ranger-usersync; sudo chgrp ranger /var/log/ranger-usersync

        cd ranger-usersync

  2. Now let’s edit the install.properties file. Here are the relevant lines that you should edit:

         POLICY_MGR_URL=http://localhost:6080 (This should point to the server where Ranger Admin has been installed)

         SYNC_SOURCE=unix (This assumes that the user from the linux/unix system will be synchronized). If you want to use LDAP users, you can find here how to Configure Ranger UserSync for LDAP 

         logdir=/var/log/ranger/usersync

  3. Now install the usersync by running the setup command

         export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64 (Change this to your Java install folder)

         ./setup.sh

  4. create a valid symlink in /usr/bin/ for start/stop of ranger usersync
         cd /usr/bin
         ln -sf /usr/local/ranger-usersync/start.sh ranger-usersync-start
         ln -sf /usr/local/ranger-admin/ews/stop.sh ranger-usersync-stop

  5. update ranger-usersync service file to link to the start and stop scripts 

         vim /etc/init.d/ranger-usersync ( Update the Start and Stop commands to point to the created symlinks )

  6. Start the Ranger Usersync
          service ranger-usersync start

  7. You can verify by looking at the users tab in Ranger Admin. Unix host users should be sync'ed to ranger.

  8. Logs are in logs folder. It is relative to the location where ranger-usersync was installed. Look for usersync.log file for User Sync related errors and auth.log for remote login errors.

 

Next Steps

  • If you are using LDAP or AD, then you will have to configure using to synchronize with LDAP/AD

Install/Configure Ranger HDFS Plugin

Ranger HDFS plugin helps to centralize HDFS authorization policies. To verify that, first Apache Hadoop needs to be installed. If Hadoop is not already installed, follow below steps.

  • These instructions were written for Hadoop 2.5.2. So grab that tar (hadoop-2.5.2.tar.gz) and checksum file (hadoop-2.5.2.tar.gz.md5).

  • Follow the instructions available on the hadoop site http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html.

  • Follow steps given in pseudo distributed mode.

  • Make note of the location where you installed hadoop. Here I assume that you have installed it in /usr/local/hadoop.

  • Create a user under which we could install and ultimately run the various hadoop processes. And login as that user.

                      sudo useradd --home-dir /var/hadoop --create-home --shell /bin/bash --user-group hadoop
                      sudo tar xzf hadoop-2.5.2.tar.gz -C /usr/local
                      cd /usr/local
                      sudo ln -s hadoop-2.5.2 hadoop
                      sudo chown hadoop -R hadoop hadoop-2.5.2
                      sudo chgrp hadoop -R hadoop hadoop-2.5.2
                      sudo su - hadoop

Now let's follow the below steps to install/configure Ranger HDFS plugin.

    • Create a repository in Ranger Policy Manager. E.g. "local_hdfs". The same name needs to be configured during plugin setup

    • Extract binaries at the appropriate place (/usr/local).

                 cd /usr/local

                 sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-hdfs-plugin.tar.gz

                 sudo ln -s ranger-0.4.0-hdfs-plugin ranger-hdfs-plugin

                 cd ranger-hdfs-plugin

    • Now let’s edit the install.properties file. Here are the relevant lines that you should edit:

                POLICY_MGR_URL=http://localhost:6080

                REPOSITORY_NAME=local_hdfs

                XAAUDIT.DB.HOSTNAME=localhost

                XAAUDIT.DB.DATABASE_NAME=ranger

                XAAUDIT.DB.USER_NAME=rangerlogger

                XAAUDIT.DB.PASSWORD=rangerlogger (Change password as it was set during Ranger Admin install)

    • Now enable the hdfs-plugin by running the enable-hdfs-plugin.sh command (Remember to set JAVA_HOME)

      • Create a symlink as conf dir of hadoop linking to hadoop conf dir
        • cd /usr/local/hadoop
        • ln -s /usr/local/hadoop/etc/hadoop conf
      • Enable Ranger HDFS plugin
        • export HADOOP_HOME=/usr/local/hadoop
        • cd /usr/local/ranger-hdfs-plugin
        • ./enable-hdfs-plugin.sh
      • Copy all the jar files from ${hadoop_home}/lib
        • cp /usr/local/hadoop/lib/* /usr/local/hadoop/share/hadoop/hdfs/lib/
    • Now edit the xasecure-audit.xml file. 

      • cd /usr/local/hadoop/conf
      • change the xasecure-audit.xml file to look like the below. Make sure the JDBC properties are correct.

                   <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
                   <value>jdbc:mysql://localhost/ranger</value>
                   </property>
                   <property>
                   <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
                   <value>rangerlogger</value>
                   </property>
                   <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
                   <value>rangerlogger</value>
                   </property>

  • Once these changes are done Restart Hadoop namenode. This should start the association of ranger-hdfs-plugin with hadoop.

  • You can verify by logging into the Ranger Admin Web interface -> Audit -> Agents

  • Now HDFS resources will be authorized via Ranger policies.

Install/Configure Ranger Hive Plugin


Ranger Hive plugin integrates with Hive to enforce authorization policies. To verify that, first Apache Hive needs to be installed. If not already installed, follow below steps. Version 0.14 is required.

  • Download and install Hive by following the instructions available on the apache hive website. Hive Install instructions

            cd /usr/local

            sudo tar zxf ~/dev/apache-hive-0.14.0-bin.tar.gz

            sudo ln -s ~/dev/apache-hive-0.14.0-bin.tar.gz hive

            cd hive

  • Note: Hive Server 2 doesn’t start unless HADOOP_VERSION is exported to bashrc 

Now let's follow the below steps to install/configure Ranger Hive plugin.

  • Create a repository in Ranger Policy Manager. E.g. "hivedev". The same name needs to be configured during plugin setup

  • Extract binaries at the appropriate place.

              cd /usr/local

              sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-hive-plugin.tar.gz

              sudo ln -s ranger-0.4.0-hive-plugin ranger-hive-plugin

              cd ranger-hive-plugin

  • Now let’s edit the install.properties file. Here are the relevant lines that you should edit:

              POLICY_MGR_URL=http://localhost:6080

              REPOSITORY_NAME=hivedev

              XAAUDIT.DB.HOSTNAME=localhost

              XAAUDIT.DB.DATABASE_NAME=ranger

              XAAUDIT.DB.USER_NAME=rangerlogger

              XAAUDIT.DB.PASSWORD=rangerlogger

  • Now enable the hive-plugin by running the enable-hive-plugin.sh command (Remember to set JAVA_HOME)

              cd /usr/local/ranger-hive-plugin

              ./enable-hive-plugin.sh

  • Now edit the xasecure-audit.xml file.Once these changes are done Restart hive. This should start the association of ranger-hive-plugin with hive.

    • cd /usr/local/hive/conf
    • Change the xasecure-audit.xml file. Make sure the JDBC properties are correct.
                     <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
                     <value>jdbc:mysql://localhost/ranger</value>
                     </property>
                     <property>
                     <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
                     <value>rangerlogger</value>
                     </property>
                     <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
                     <value>rangerlogger</value>
                     </property>
  • You can verify by logging into the Ranger Admin Web interface -> Audit Tab -> Agents

Install/Configure Ranger HBase Plugin

 

Ranger HBase plugin integrates with HBase to enforce authorization policies. To verify that, first Apache HBase needs to be installed. If not already installed, follow below steps. Ranger will work only with HBase version 0.99.2 or above.

  • Build Hbase > 0.99.2 (0.99.2RC0 is used for this document)
  • Untar the build Hbase assembly to /usr/local directory
    • cd /usr/local && cp /root/dev/hbase-trunk/hbase/hbase-assembly/target/hbase-0.99.2-bin.tar.gz .
    • tar xzf hbase-0.99.2-bin.tar.gz
    • sudo ln -s hbase-0.99.2 hbase
    • cd hbase
  • Now follow the instructions in Apache HBase site to configure in Pseudo mode. http://hbase.apache.org/book/getting_started.html

Now let's follow the below steps to install/configure Ranger HBase plugin.

  • Create a repository in Ranger Policy Manager. E.g. "hbasedev". The same name needs to be configured during plugin setup

  • Extract binaries at the appropriate place.

              cd /usr/local

              sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-hbase-plugin.tar.gz

              sudo ln -s ranger-0.4.0-hbase-plugin ranger-hbase-plugin

              cd ranger-hbase-plugin

  • Now let’s edit the install.properties file. Here are the relevant lines that you should edit:

              POLICY_MGR_URL=http://localhost:6080

              REPOSITORY_NAME=hbasedev

              XAAUDIT.DB.HOSTNAME=localhost

              XAAUDIT.DB.DATABASE_NAME=ranger

              XAAUDIT.DB.USER_NAME=rangerlogger

              XAAUDIT.DB.PASSWORD=rangerlogger

  • Now enable the hbase-plugin by running the enable-hbase-plugin.sh command (Remember to set JAVA_HOME)

              cd /usr/local/ranger-hbase-plugin

              ./enable-hbase-plugin.sh

  • Now edit the xasecure-audit.xml file.Once these changes are done Restart HBase. This should start the association of ranger-hbase-plugin with HBase.

    • cd /usr/local/hbase/conf
    • Change the xasecure-audit.xml file. Make sure the JDBC properties are correct.
                     <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
                     <value>jdbc:mysql://localhost/ranger</value>
                     </property>
                     <property>
                     <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
                     <value>rangerlogger</value>
                     </property>
                     <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
                     <value>rangerlogger</value>
                     </property>
  • You can verify by logging into the Ranger Admin Web interface -> Audit Tab -> Agents

Install/Configure Ranger Knox Plugin


Ranger Knox plugin integrates with Knox to enforce authorization policies. To verify that, first Apache Knox needs to be installed. If not already installed, follow below steps.
  • Let’s download and install apache Knox from Apache Mirrors. http://mirrors.koehn.com/apache/knox/0.5.0/
                cd /usr/local
                sudo tar zxf ~/dev/knox-0.5.0.tar.gz
                sudo ln -s knox-0.5.0 knox
                cd knox

  • Now follow the instructions in Apache Knox site to configure Knox Gateway/LDAP. Knox Install Instructions 

Now let's follow the below steps to install/configure Ranger Knox plugin.

  • Create a repository in Ranger Policy Manager. E.g. "knoxdev". The same name needs to be configured during plugin setup
  • Extract binaries at the appropriate place.
              cd /usr/local
              sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-knox-plugin.tar.gz
              sudo ln -s ranger-0.4.0-knox-plugin ranger-knox-plugin
              cd ranger-knox-plugin
  • Now let’s edit the install.properties file. Here are the relevant lines that you should edit:
              POLICY_MGR_URL=http://localhost:6080
              REPOSITORY_NAME=knoxdev
              KNOX_HOME=/usr/local/knox
              XAAUDIT.DB.HOSTNAME=localhost
              XAAUDIT.DB.DATABASE_NAME=ranger
              XAAUDIT.DB.USER_NAME=rangerlogger
              XAAUDIT.DB.PASSWORD=rangerlogger
  • Now enable the knox-plugin by running the enable-knox-plugin.sh command (Remember to set JAVA_HOME)
              cd /usr/local/ranger-knox-plugin
              ./enable-knox-plugin.sh
  • Now edit the xasecure-audit.xml file.
    • cd /usr/local/knox/conf
    • Put in /usr/<hadoop-installs>/<your version>/knox/ext/ a symbolic link to mysql-connector-java.jar
    • Change the xasecure-audit.xml file. Make sure the JDBC properties are correct.
                     <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
                     <value>jdbc:mysql://localhost/ranger</value>
                     </property>
                     <property>
                     <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
                     <value>rangerlogger</value>
                     </property>
                     <property> <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
                     <value>rangerlogger</value>
                     </property>

     

  • Once these changes are done Restart Knox (Gateway/LDAP). This should start the association of ranger-knox-plugin with Knox.
  • You can verify by logging into the Ranger Admin Web interface -> Audit Tab -> Agents

Install/Configure Ranger Storm Plugin

Ranger Storm plugin integrates with Storm to enforce authorization policies. To verify that, first Apache Storm needs to be installed. If not already installed, follow below steps.

Let’s download and install apache storm from Apache Mirrors.  

Now let's follow the below steps to install/configure Ranger Storm Plugin.

    • Create a repository in Ranger Policy Manager. E.g. "stormdev". The same name needs to be configured during plugin setup

    • cd /usr/local

    • sudo tar zxf ~/dev/ranger/target/ranger-0.4.0-storm-plugin.tar.gz

    • sudo ln -s ranger-0.4.0-storm-plugin ranger-storm-plugin

    • cd ranger-storm-plugin

Now let’s edit the install.properties file.  Here are the relevant lines that you should edit:

    • Change the insall.properties file

    • POLICY_MGR_URL=http://localhost:6080

    • REPOSITORY_NAME=stormdev

    • KNOX_HOME=/usr/local/storm

    • XAAUDIT.DB.IS_ENABLED=true

    • XAAUDIT.DB.HOSTNAME=localhost

    • XAAUDIT.DB.DATABASE_NAME=ranger

    • XAAUDIT.DB.USER_NAME=rangerlogger

    • XAAUDIT.DB.PASSWORD=rangerlogger

 Now enable the storm-plugin by running the enable-storm-plugin.sh command (Remember to set JAVA_HOME)

    • cd /usr/local/ranger-storm-plugin

    • ./enable-storm-plugin.sh

Now edit the xasecure-audit.xml file. Here are the relevant lines that you should edit :

  • cd /usr/local/storm/conf

  • change the xasecure-audit.xml file.

<property>
<name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
<value>jdbc:mysql://localhost/ranger</value>
<
/property>
<property>
<name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
<value>rangerlogger</value>
</property>
<property>
<name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
<value>rangerlogger</value>
</property>

  • Once these changes are done Restart Storm

  • This should start the association of ranger-storm-plugin  with storm.

  • You can verify by logging into the Ranger Admin Web interface ->  Audit -> Agents