Overview

As part of RANGER-2331, ranger has enabled support for saving master key of Ranger KMS in Gemalto's KeySecure HSM ( https://safenet.gemalto.com/ ).  Similar to Luna HSM support, this feature provides end users to use another HSM service to save Ranger KMS's master key. In order to get started, first of all user needs to setup SafeNet KeySecure Management Console and then configure Ranger KMS to communicate with the KeySecure instance.

Once the communication between Ranger KMS and KeySecure is successful, the master key of Ranger KMS on start, will be created and saved in HSM. After that, all Encryption Keys will be created using the master key saved in HSM. 

This patch covers,

  • Support of Ranger KMS with NAE-XML protocol for using KeySecure HSM. 
  • Migration Utility to migrate master key from Ranger KMS DB to KeySecure HSM.
  • Migration utility to migrate master key from KeySecure HSM to Ranger KMS DB.

In order to get it working. There are few steps that needs to be done on KeySecure end first, 

Add a Device on SafeNet KeySecure

  • Login to KeySecure instance.
  • Navigate to Device tab, under section Cryptographic Key Server Configuration,   click on Add button.
  • Protocol NAE-XML (At present we only support NAE-XML),
    IP [All]
    Port 9000
    Use SSL (Key uncheck for simple and check when you want key secure SSL) 
    Server Certificate (None for simple and select certificate  when you key secure SSL)
  • Click on save

  • Click on NAE-XML protocol and check the properties. Sharing screenshot of working properties: 

Add a user on SafeNet KeySecure

  • Click on Security tab on left side panel we have the section with name User & Groups. Under that click on Local Authentication
  • Add user with username, password, check User Administration Permission and Change Password Permission
    • E.g username : user1
    • Password : <passwordValue>

Note : This user's credentials will be needed in Ranger KMS configs.

Fresh Installation Of Ranger KMS with SafeNet KeySecure (NAE-XML) (Simple Env)

  1. Create some directory and copy below mentioned files at this location (These files needs to be obtained from Gemalto SafeNet KeySecure).
    1. IngrianNAE.properties
    2. libIngPKCS11.so
    3. sunpkcs11.cfg
      1. For example : mkdir -p /opt/safenetConf/64/8.3.1 (and then copy files under this location)
    4. Note : Make sure you have provided read-write access (chmod 775)  to Ranger KMS service user (chown kms:kms)  to all the 3 files as shown above.

  2. Update the IngrianNAE.properties file (vi IngrianNAE.properties) and initialize the below mentioned properties.

    1. NAE_IP = Your Safenet KeySecure IP address (If it’s on different n/w please provide public IP)

    2. NAE_Port=9000 (should be the same port provided on KeySecure instance under Device tab)

    3. Protocol=tcp (valid values should be either ssl or tcp, should be the same protocol provided on key secure instance under device tab)

    4. (Initialize below prop only when key secure is SSL)

      1. CA_File=/etc/security/serverKeys/KSCAN.crt (location of SSL certificate created from KeySecure and copied in Ranger KMS host)

  3. Update profile file on Ranger KMS Host,  and enter below 3 export command at the end.

       vi /etc/profile

    export IngrianNAE_Properties_Conf_Slot_ID_Max=100

    export IngrianNAE_Properties_Conf_SessionID_Max=100

    export NAE_Properties_Conf_Filename=/opt/safenetConf/64/8.3.1/IngrianNAE.properties

    (directory created in step 1)

  4. Add Ranger KMS from Ambari UI  and add the below custom properties under Advanced tab in section Custom dbks-site.

    ranger.kms.keysecure.enabled=true

    ranger.kms.keysecure.UserPassword.Authentication=true

    ranger.kms.keysecure.masterkey.name=keysecureMasterKeyName

    ranger.kms.keysecure.login.username=user1

    ranger.kms.keysecure.login.password=<passwordValue>

    (user details created on KeySecure while adding user)

    ranger.kms.keysecure.hostname=<Keysecure Hostname>

    ranger.kms.keysecure.masterkey.size=256

    ranger.kms.keysecure.sunpkcs11.cfg.filepath=/opt/safenetConf/64/8.3.1/sunpkcs11.cfg

    (directory created in step 1)

    ranger.kms.keysecure.login.password.alias=ranger.ks.login.password

    Note : Add ranger.kms.keysecure.login.password config as a password type in custom section of dbks-site. To avoid writing password in plaintext add the value as “_” from Ambari and then add the actual password in jceks using below command,

    Ranger CredentialAPI

    /usr/jdk64/jdk1.8.0_112/bin/java -cp '/usr/hdp/current/ranger-kms/cred/lib/*' org.apache.ranger.credentialapi.buildks create ranger.ks.login.password -value <passwordValue> -provider jceks://file/etc/ranger/kms/rangerkms.jceks

    Sample Screenshot :

  5. Save Configs, and start the Ranger KMS service. During start of service, Ranger KMS will try and create master key and using above configs, it will save master key in Safenet KeySecure HSM. 
  6. In order to verify, successful creation of master key in HSM, login to KeySecure portal to check if master key is created or not. 
  7. Key with above mentioned name (keySecureMasterKeyName) will be created under Security tab.
    Sample screenshot :


    On KeySecure, 2 keys will be created, one with the mentioned name and other with some random hex code. We need not worry about another key as KeySecure internally uses it.


  8. User can now proceed for creating encryption zone keys on Ranger UI.

Manual Steps for configuring Gemalto SafeNet KeySecure in SSL Mode : 

In order to get started on SSL enabled environment. First of all, adding few simple steps to create certificate on KeySecure end. After that, steps to integrate SSL certificate in ranger  kms will be explained. 

Creating a Local CA

1. Log on to the Management Console as an administrator with Certificate Authorities access control.

2. Navigate to the Create Local Certificate Authority section on the Certificate and CA Configuration page

(Security, CAs & SSL Certificates, Local CAs).

3. Modify the fields as needed or refer below image for testing purpose.

4. Select Self-signed Root CA  as the Certificate Authority Type.

5. Click Create.

6. After creating Local CA its visible as shown below


Creating a Server Certificate Request on the Management Console

1. Log on to the Management Console as an administrator with Certificates access control.

2. Click on Security tab and on left side panel navigate to section Device CAs & SSL Certificates , click on SSL certificates link and modify the fields as needed.


3. Click Create Certificate Request. This creates the certificate request and places it in the Certificate List section of the Certificate and CA Configuration page. The new entry shows that the Certificate Purpose is Certificate Request and that the Certificate Status is Request Pending.


Signing a Server Certificate Request with a Local CA

To sign a server certificate request with a local CA:

1. Log on to the Management Console as an administrator with Certificates and Certificate Authorities access

controls.

2. Navigate to the Certificate List section on the Certificate and CA Configuration page (Security, CAs & SSL

Certificates, SSL Certificates).

3. Select the certificate request and click Properties.


4. Copy the text of the certificate request. The copied text must include the header (-----BEGIN CERTIFICATE

REQUEST-----) and footer (-----END CERTIFICATE REQUEST-----).

5. On left side panel navigate to section Device CAs & SSL Certificates , click on Local CAs link and click Sign Request to access the Sign Certificate Request section.

6. Modify the fields as shown:

7. Sign with Certificate Authority - Select the CA that signs the request.

8. Certificate Purpose - Select Server.

9. Certificate Duration (days) - Enter the life-span of the certificate.

10. Certificate Request - Paste all text from the certificate request, including the header and footer.


11. Click Sign Request. This will take you to the CA Certificate Information section.

12. Copy the actual certificate. The copied text must include the header (-----BEGIN CERTIFICATE-----) and footer (-----END CERTIFICATE-----).

13. Navigate back to the Certificate List section (Security, CAs & SSL Certificates, SSL Certificates). Select your certificate request and click Properties.

14. Click Install Certificate.

15. Paste the actual certificate in the Certificate Response text box. Click Save. The Management Console returns you to the Certificate List section. The section will now show that the Certificate Purpose is Server and that the Certificate Status is Active.

Above steps completes manual tasks to create and activate SSL certificate on Gemalto's SafeNet KeySecure.

Downloading the Local CA Certificate

To download a local CA certificate from the SafeNet KeySecure appliance:

1. Log on to the Management Console as an administrator with Certificate Authorities access control.

2. Navigate to the Local Certificate Authority List section of the Certificates and CA Configuration page (Security, CAs & SSL Certificates, Local CAs).

3. Select the Local CA and click Download to download the file to your client. You should place the CA certificate in a secure location and modify access appropriately.


Now, in order to use KeySecure HSM on Ranger KMS end, you need to follow similar set of  steps as that of simple environment.

Fresh Installation Of Ranger KMS with SafeNet KeySecure (NAE-XML) (SSL Env)

For SSL enabled environment, mainly step 2 and 4 will differ and all other steps remains same. Listing changes in steps to be followed.

  1. Step 1 will be same as that of simple environment's step.
  2. Download certificate file (e.g. KSCAN.crt) created on key secure cluster
    (Log in to key secure => Security tab (on top ) => Local CAs (on left )

    and copy on your Ranger KMS host at location  “/etc/security/serverKeys/”.
    Make sure Ranger KMS user has read-write access to the file.
       chown  kms:kms KSCAN.crt
       chmod  755 KSCAN.crt

  3. Update the IngrianNAE.properties file (vi IngrianNAE.properties) and initialize the below mentioned properties.

    1. NAE_IP = Your SafeNet KeySecure IP address (If it’s on different n/w please provide public IP)

    2. NAE_Port=9000 (should be the same port provided on KeySecure instance under Device tab)

    3. Protocol=ssl (valid values should be either ssl or tcp, should be the same protocol provided on key secure instance under device tab)

    4. (Initialize below prop only when key secure is SSL)
      CA_File=/etc/security/serverKeys/KSCAN.crt (location of SSL certificate created from KeySecure and copied in Ranger KMS host)

  4. Step 4, 5 and 6 will be same as that of simple environment.


Migration Utilities 

Migration of master key from DB to KeySecure NAE-XML :

  1. Once Ranger KMS is installed and running.

  2. Stop the ranger kms from ambari.

  3. Note : before proceeding, confirm if Ranger KMS is stopped. 

  4. Perform this step only when KeySecure is SSL enabled.
    Download certificate file (e.g. KSCAN.crt) created on KeySecure cluster
    (Log in to key secure => Security tab (on top ) => Local CAs (on left )

    and copy on your Ranger KMS instance at location  “/etc/security/serverKeys/” .
    Make sure Ranger KMS user has read-write access to the file.
       chown  kms:kms KSCAN.crt
       chmod  755 KSCAN.crt

  5. Create some directory and copy below mentioned files at this location (These files needs to be obtained from Gemalto SafeNet KeySecure).
    1. IngrianNAE.properties
    2. libIngPKCS11.so
    3. sunpkcs11.cfg
      1. For example : mkdir -p /opt/safenetConf/64/8.3.1 (and then copy files under this location)
    4. Note : Make sure you have provided read-write access (chmod 775)  to Ranger KMS service user (chown kms:kms)  to all the 3 files as shown above.

  6. Update profile file on Ranger KMS Host,  and enter below 3 export command at the end.

       vim /etc/profile

    export IngrianNAE_Properties_Conf_Slot_ID_Max=100

    export IngrianNAE_Properties_Conf_SessionID_Max=100

    export NAE_Properties_Conf_Filename=/opt/safenetConf/64/8.3.1/IngrianNAE.properties

    (directory created in step 5)

  7. Update the IngrianNAE.properties file (vi IngrianNAE.properties) and initialize the below mentioned properties.

    1. NAE_IP = Your SafeNet KeySecure IP address (If it’s on different n/w please provide public IP)

    2. NAE_Port=9000 (should be the same port provided on KeySecure instance under Device tab)

    3. Protocol=tcp (valid values should be either ssl or tcp, should be the same protocol provided on key secure instance under device tab)

    4. (Initialize below prop only when key secure is SSL)
      CA_File=/etc/security/serverKeys/KSCAN.crt (location of SSL certificate created from KeySecure and copied in Ranger KMS host)

  8. Export the below environment variables related to key secure as shown below on location

    cd /grid/0/hdp/current/ranger-kms/

    export IngrianNAE_Properties_Conf_Slot_ID_Max=100

    export IngrianNAE_Properties_Conf_SessionID_Max=100

    export NAE_Properties_Conf_Filename=/opt/safenetConf/64/8.3.1/IngrianNAE.properties

    (directory created in step 5)

    Ensure that variables successfully exported using below command.

    env | grep NAE

  9. On the location of ranger-kms we have the migration utility script name DBMKTOKEYSECURE.sh

    USAGE: ./ DBMKTOKEYSECURE.sh <keySecureMasterKeyName> <keySecureUsername> <keySecurePassword> <sunpkcs11CfgFilePath>

    For example :
     ./ DBMKTOKEYSECURE.sh masterkeyname user1 user1's_password /opt/safenetConf/64/8.3.1/sunpkcs11.cfg


    <keySecureMasterKeyName>     : Name of the key which needs to be created on KeySecure

    <keySecureUsername>  : User created on KeySecure cluster 

    <keySecurePassword> : Password of the user 

    <sunpkcs11CfgFilePath> : location created in step 5

    Note :  Export JAVA_HOME if required.

    Add/Update below property in java.security file at location JAVA_HOME/jre/lib/security

    jceks.key.serialFilter = java.lang.Enum;java.security.KeyRep;\java.security.KeyRep$Type;javax.crypto.spec.SecretKeySpec;org.apache.hadoop.crypto.key.**;!*

  10. Add the below custom properties under Advanced tab in section Custom dbks-site for  Ranger KMS.

    Please make sure you provide the same values as used in step 9.

    ranger.kms.keysecure.enabled=true

    ranger.kms.keysecure.UserPassword.Authentication=true

    ranger.kms.keysecure.masterkey.name=masterkeyname

    ranger.kms.keysecure.login.username=user1

    ranger.kms.keysecure.login.password=user1's_password

    (user details created on KeySecure while adding user)

    ranger.kms.keysecure.hostname=<Keysecure Hostname>

    ranger.kms.keysecure.masterkey.size=256

    ranger.kms.keysecure.sunpkcs11.cfg.filepath=/opt/safenetConf/64/8.3.1/sunpkcs11.cfg

    (directory created in step 5)

    ranger.kms.keysecure.login.password.alias=ranger.ks.login.password

  11. Start the Ranger KMS service from ambari.


Migration of master key from KeySecure HSM to DB :

  1. Make sure your ranger kms is up and running fine.

  2. Stop the ranger kms from ambari.

  3. Note : before proceeding, confirm if Ranger KMS is stopped. 
  4. On the location of ranger-kms we have the migration utility script name KEYSECUREMKTOKMSDB.sh , execute it
    E.g. ./KEYSECUREMKTOKMSDB.sh myPassword

    Note :  Export JAVA_HOME if required.

    Add/Update below property in java.security file at location JAVA_HOME/jre/lib/security

    jceks.key.serialFilter = java.lang.Enum;java.security.KeyRep;\java.security.KeyRep$Type;javax.crypto.spec.SecretKeySpec;org.apache.hadoop.crypto.key.**;!*

  5. After successful execution of script switch back to ambari and update the below prop.
    ranger.kms.keysecure.enabled=false
    KMS_MASTER_KEY_PASSWD = myPassword (same as provided in step 4)

  6. Start the ranger-kms service from ambari or manually. 













  • No labels