Apache Solr Documentation

6.4 Ref Guide (PDF Download)
Solr Tutorial
Solr Community Wiki

Older Versions of this Guide (PDF)

6.5 Draft Ref Guide Topics

Meta-Documentation

This Unreleased Guide Will Cover Apache Solr 6.5

Skip to end of metadata
Go to start of metadata

If you are using Kerberos to secure your network environment, the Kerberos authentication plugin can be used to secure a Solr cluster. This allows Solr to use a Kerberos service principal and keytab file to authenticate with ZooKeeper and between nodes of the Solr cluster. Users of the Admin UI and alll clients (such as SolrJ) would also need to have a valid ticket before being able to use the UI or send requests to Solr.

Support for the Kerberos authentication plugin is only available in SolrCloud mode.

If you are using Solr with a Hadoop cluster secured with Kerberos and intend to store your Solr indexes in HDFS, also see the section Running Solr on HDFS for additional steps to configure Solr for that purpose. The instructions on this page apply only to scenarios where Solr will be secured with Kerberos. If you only need to store your indexes in a Kerberized HDFS system, please see the other section referenced above.

How Solr Works With Kerberos

When setting up Solr to use Kerberos, configurations are put in place for Solr to use a service principal, or a Kerberos username, which is registered with the Key Distribution Center (KDC) to authenticate requests. The configurations define the service principal name and the location of the keytab file that contains the credentials.

security.json

The Solr authentication model uses a file called /security.json which is stored in ZooKeeper. A description of this file and how it is created and maintained is covered in the section Authentication and Authorization Plugins, and can only be used when Solr is running in SolrCloud mode. If this file is created after an initial startup of Solr, a restart of the system on each node is required. 

Alternatively, the authentication plugin implementation can be specified during node startup using the system parameter: -DauthenticationPlugin=org.apache.solr.security.KerberosPlugin. This parameter can be used with either SolrCloud mode or standalone mode. However, if you are using Solr in standalone mode, this system parameter is the only way to enable Kerberos.

If you are using SolrCloud mode, the approach to use security.json is the best practice.

Service Principals and Keytab Files

Each Solr node must have a service principal registered with the Key Distribution Center (KDC). The Kerberos plugin uses SPNego to negotiate authentication.

Using HTTP/host1@YOUR-DOMAIN.ORG, as an example of a service principal:

  • HTTP indicates the type of requests which this service principal will be used to authenticate. The HTTP/ in the service principal is a must for SPNego to work with requests to Solr over HTTP.
  • host1 is the host name of the machine hosting the Solr node.
  • YOUR-DOMAIN.ORG is the organization wide Kerberos realm. 

Multiple Solr nodes on the same host may have the same service principal, since the host name is common to them all.

Along with the service principal, each Solr node needs a keytab file which should contain the credentials of the service principal used. A keytab file contains encrypted credentials to support passwordless logins while obtaining Kerberos tickets from the KDC. For each Solr node, the keytab file should be kept in a secure location and not shared with users of the cluster.

Since a Solr cluster requires internode communication, each node must also be able to make Kerberos enabled requests to other nodes. By default, Solr uses the same service principal and keytab as a 'client principal' for internode communication. You may configure a distinct client principal explicitly, but doing so is not recommended and is not covered in the examples below.

Kerberized ZooKeeper

When setting up a kerberized Solr cluster, it is recommended to enable Kerberos security for Zookeeper as well. In such a setup, the client principal used to authenticate requests with Zookeeper can be shared for internode communication as well. This has the benefit of not needing to renew the ticket granting tickets (TGTs) separately, since the Zookeeper client used by Solr takes care of this. To achieve this, a single JAAS configuration (with the app name as Client) can be used for the Kerberos plugin as well as for the Zookeeper client. See the configuration section below for an example of starting Zookeeper in Kerberos mode.

Browser Configuration

In order for your browser to access the Solr Admin UI after enabling Kerberos authentication, it must be able to negotiate with the Kerberos authenticator service to allow you access. Each browser supports this differently, and some (like Chrome) do not support it at all. If you see 401 errors when trying to access the Solr Admin UI after enabling Kerberos authentication, it's likely your browser has not been configured properly to know how or where to negotiate the authentication request.

Detailed information on how to set up your browser is beyond the scope of this documentation; please see your system administrators for Kerberos for details on how to configure your browser. 

Plugin Configuration

Consult Your Kerberos Admins!

Before attempting to configure Solr to use Kerberos authentication, please review each step outlined below and consult with your local Kerberos administrators on each detail to be sure you know the correct values for each parameter. Small errors can cause Solr to not start or not function properly, and are notoriously difficult to diagnose.

Configuration of the Kerberos plugin has several parts:

  • Create service principals and keytab files
  • ZooKeeper configuration
  • Create or update /security.json
  • Define jaas-client.conf
  • Solr startup parameters

We'll walk through each of these steps below.

Using Hostnames

To use host names instead of IP addresses, use the SOLR_HOST config in bin/solr.in.sh or pass a -Dhost=<hostname> during Solr startup. This guide uses IP addresses . If you specify a hostname replace all the IP addresses in the guide with the solr hostname

Get Service Principals and Keytabs

Before configuring Solr, make sure you have a Kerberos service principal for each Solr host and ZooKeeper (if ZooKeeper has not already been configured) available in the KDC server, and generate a keytab file as shown below.

This example assumes the hostname is 192.168.0.107 and your home directory is /home/foo/. This example should be modified for your own environment.

Copy the keytab file from the KDC server’s /tmp/107.keytab location to the Solr host at /keytabs/107.keytab. Repeat this step for each Solr node.

You might need to take similar steps to create a Zookeeper service principal and keytab if it has not already been set up. In that case, the example below shows a different service principal for ZooKeeper, so the above might be repeated with zookeeper/host1 as the service principal for one of the nodes

ZooKeeper Configuration

If you are using a ZooKeeper that has already been configured to use Kerberos, you can skip the ZooKeeper-related steps shown here.

Since ZooKeeper manages the communication between nodes in a SolrCloud cluster, it must also be able to authenticate with each node of the cluster. Configuration requires setting up a service principal for ZooKeeper, defining a JAAS configuration file and instructing ZooKeeper to use both of those items. 

The first step is to create a file java.env in ZooKeeper's conf directory and add the following to it, as in this example:

The JAAS configuration file should contain the following parameters. Be sure to change the principal and keyTab path as appropriate. The file must be located in the path defined in the step above, with the filename specified.

Finally, add the following lines to the ZooKeeper configuration file zoo.cfg:

Once all of the pieces are in place, start ZooKeeper with the following parameter pointing to the JAAS configuration file:

Create /security.json

Set up Solr to use the Kerberos plugin by uploading the security.json as follows:

More details on how to use a /security.json file in Solr are available in the section Authentication and Authorization Plugins.

If you already have a /security.json file in Zookeeper, download the file, add or modify the authentication section and upload it back to ZooKeeper using the Command Line Utilities available in Solr.

Define a JAAS Configuration File

The JAAS configuration file defines the properties to use for authentication, such as the service principal and the location of the keytab file. Other properties can also be set to ensure ticket caching and other features.

The following example can be copied and modified slightly for your environment. The location of the file can be anywhere on the server, but it will be referenced when starting Solr so it must be readable on the filesystem. The JAAS file may contain multiple sections for different users, but each section must have a unique name so it can be uniquely referenced in each application.

In the below example, we have created a JAAS configuration file with the name and path of /home/foo/jaas-client.conf. We will use this name and path when we define the Solr start parameters in the next section. Note that the client principal here is the same as the service principal. This will be used to authenticate internode requests and requests to Zookeeper.  Make sure to use the correct principal hostname and the keyTab file path.

The first line of this file defines the section name, which will be used with the solr.kerberos.jaas.appname parameter, defined below.

The main properties we are concerned with are the keyTab and principal properties, but there are others which may be required for your environment. The javadocs for the Krb5LoginModule (the class that's being used and is called in the second line above) provide a good outline of the available properties, but for reference the ones in use in the above example are explained here:

  • useKeyTab: this boolean property defines if we should use a keytab file (true, in this case). 
  • keyTab: the location and name of the keytab file for the principal this section of the JAAS configuration file is for. The path should be enclosed in double-quotes.
  • storeKey: this boolean property allows the key to be stored in the private credentials of the user.
  • useTicketCache: this boolean property allows the ticket to be obtained from the ticket cache.
  • debug: this boolean property will output debug messages for help in troubleshooting.
  • principal: the name of the service principal to be used.

Solr Startup Parameters

While starting up Solr, the following host-specific parameters need to be passed. These parameters can be passed at the command line with the bin/solr start command (see Solr Control Script Reference for details on how to pass system parameters) or defined in bin/solr.in.sh or bin/solr.in.cmd as appropriate for your operating system.

Parameter NameRequiredDescription

solr.kerberos.name.rules

No

Used to map Kerberos principals to short names. Default value is DEFAULT. Example of a name rule: RULE:[1:$1@$0](.*EXAMPLE.COM)s/@.*//  

solr.kerberos.cookie.domain

Yes

Used to issue cookies and should have the hostname of the Solr node.

solr.kerberos.cookie.portawareNoWhen set to true, cookies are differentiated based on host and port, as opposed to standard cookies which are not port aware. This should be set if more than one Solr node is hosted on the same host. The default is false.

solr.kerberos.principal

Yes

The service principal.

solr.kerberos.keytab

Yes

Keytab file path containing service principal credentials.

solr.kerberos.jaas.appname

No

The app name (section name) within the JAAS configuration file which is required for internode communication. Default is Client, which is used for Zookeeper authentication as well. If different users are used for ZooKeeper and Solr, they will need to have separate sections in the JAAS configuration file.

java.security.auth.login.config

Yes

Path to the JAAS configuration file for configuring a Solr client for internode communication.

Here is an example that could be added to bin/solr.in.sh. Make sure to change this example to use the right hostname and the keytab file path.

KDC with AES-256 encryption

If your KDC uses AES-256 encryption, you need to add the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files to your JRE before a kerberized Solr can interact with the KDC.

You will know this when you see an error like this in your Solr logs : "KrbException: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled"

For Java 1.8, this is available here: http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html.

Replace the local_policy.jar present in JAVA_HOME/jre/lib/security/ with the new local_policy.jar from the downloaded package and restart the Solr node.

Using Delegation Tokens

The Kerberos plugin can be configured to use delegation tokens, which allow an application to reuse the authentication of an end-user or another application.

There are a few use cases for Solr where this might be helpful:

  • Using distributed clients (such as MapReduce) where each client may not have access to the user's credentials.
  • When load on the Kerberos server is high. Delegation tokens can reduce the load because they do not access the server after the first request.
  • If requests or permissions need to be delegated to another user.

To enable delegation tokens, several parameters must be defined. These parameters can be passed at the command line with the bin/solr start command (see Solr Control Script Reference for details on how to pass system parameters) or defined in bin/solr.in.sh or bin/solr.in.cmd as appropriate for your operating system.

Parameter NameRequiredDescription

solr.kerberos.delegation.token.enabled

Yes, to enable tokensFalse by default, set to true to enable delegation tokens.

solr.kerberos.delegation.token.kind

NoType of delegation tokens. By default this is solr-dt. Likely this does not need to change. No other option is available at this time.

solr.kerberos.delegation.token.validity

NoTime, in seconds, for which delegation tokens are valid. The default is 36000 seconds.

solr.kerberos.delegation.token.signer.secret.provider

NoWhere delegation token information is stored internally. The default is zookeeper which must be the location for delegation tokens to work across Solr servers (when running in SolrCloud mode). No other option is available at this time.

solr.kerberos.delegation.token.signer.secret.provider.zookeper.path

NoThe ZooKeeper path where the secret provider information is stored. This is in the form of the path + /security/token. The path can include the chroot or the chroot can be omitted if you are not using it. This example includes the chroot: server1:9983,server2:9983,server3:9983/solr/security/token.

solr.kerberos.delegation.token.secret.manager.znode.working.path

NoThe ZooKeeper path where token information is stored. This is in the form of the path + /security/zkdtsm. The path can include the chroot or the chroot can be omitted if you are not using it. This example includes the chroot: server1:9983,server2:9983,server3:9983/solr/security/zkdtsm.

Start Solr

Once the configuration is complete, you can start Solr with the bin/solr script, as in the example below. This example assumes you modified bin/solr.in.sh or bin/solr.in.cmd, with the proper values, but if you did not, you would pas the system parameters along with the start command. Note you also need to customize the -z property as appropriate for the location of your ZooKeeper nodes.

Test the Configuration

  1. Do a kinit with your username. For example, "kinit user@EXAMPLE.COM". 
  2. Try to access Solr using curl. You should get a successful response.

Using SolrJ with a Kerberized Solr

To use Kerberos authentication in a SolrJ application, you need the following two lines before you create a SolrClient:

You need to specify a Kerberos service principal for the client and a corresponding keytab in the JAAS client configuration file above. This principal should be different from the service principal we created for Solr .

Here’s an example:

Delegation Tokens with SolrJ

Delegation tokens are also supported with SolrJ, in the following ways:

  • DelegationTokenRequest and DelegationTokenResponse can be used to get, cancel, and renew delegation tokens.
  • HttpSolrClient.Builder includes a withDelegationToken function for creating an HttpSolrClient that uses a delegation token to authenticate.

Sample code to get a delegation token:

To create a HttpSolrClient that uses delegation tokens:

To create a CloudSolrClient that uses delegation tokens:

Hadoop's delegation token responses are in JSON map format. A response parser for that is available in DelegationTokenResponse. Other response parsers may not work well with Hadoop responses.

  • No labels

13 Comments

  1. how does kerberos authentication work in solr standalone mode.

    1. You would use the startup parameter instead of security.json: "Alternatively, the authentication plugin implementation can be specified during node startup using the system parameter: -DauthenticationPlugin=org.apache.solr.security.KerberosPlugin."

      Page edited to make using that for standalone more clear.

  2. thank you cassandra for your great job.

  3. even if solr is running in standalone mode i have to create security.json in file zookepper.?

  4. Kostalii, in standalone mode, there is no need for ZooKeeper itself. So, you don't need the security.json at all. Just start with the start up parameter as Cassandra mentioned. Also, use the parameters as specified in the document to put in the solr.in.sh file. You should be good to go. If you have questions, you could ask on the solr-users mailing list. Thanks!

  5. SOLR-8373: solr.kerberos.cookie.portaware (Reqd=no, default=false). When set to true, the cookies are differentiated based on host and port, as opposed to standard cookies which are not port aware. This should be set for the Solr nodes if more than one node is hosted on the same host.

  6. Hi folks, minor problem here: zookeeper won't load the JAAS configuration file if the keytabs line is NOT in double quotes. It also throws a strange error, 

    No JAAS configuration section named 'Server' was foundin <CONF>

    So your doc says keyTab=/keytabs/zkhost1.keytab

    when you need keyTab="/keytabs/zkhost1.keytab"

    (I filed a bug with Zookeeper to improve the error message someday, as well. I wouldn't have been confused for hours if it had said something like "unable to parse, expected quote" –  ZOOKEEPER-2365 - JAAS configuration section error is confusing Open )

    1. Hi Dan - thanks for pointing that out. Seems we had it right in one part of the page, but not in another, so that's fixed. I also added a sentence to point out that double quotes are required for the keyTab path.

    2. Thanks Dan. Just realized another similar error reporting goof up in ZK.

      If we don't put a trailing semi colon after the curly braces, the same issue happens. Although I am not 100% sure, I think this is an issue (as is the double quotes thing) on certain operating systems. For instance, I don't remember any such issues on Fedora with the same jaas config, but struggled to have it work on RedHat.

      Cassandra Targett, can you please add a semi-colon after the Server section of the ZK's jaas file?

  7. I'd like to add couple of examples for using SolrJ for/with delegation tokens:

    1. Get a delegation token:
        private String getDelegationToken(final String renewer, final String user, HttpSolrClient solrClient) throws Exception {

          DelegationTokenRequest.Get get = new DelegationTokenRequest.Get(renewer) {

            @Override

            public SolrParams getParams() {

              ModifiableSolrParams params = new ModifiableSolrParams(super.getParams());

              params.set("user", user);

              return params;

            }

          };

          DelegationTokenResponse.Get getResponse = get.process(solrClient);

          return getResponse.getDelegationToken();

        }

       

    2. Create a HttpSolrClient that uses delegation tokens:
      HttpSolrClient client = new HttpSolrClient.Builder("http://localhost:8983/solr").withDelegationToken(token).build();
    3. Create a CloudSolrClient that uses delegation tokens:
      CloudSolrClient client = new CloudSolrClient.Builder()
                      .withZkHost("http://localhost:2181")
                      .withLBHttpSolrClientBuilder(new LBHttpSolrClient.Builder()
                          .withResponseParser(client.getParser())
                          .withHttpSolrClientBuilder(
                              new HttpSolrClient.Builder()
                                  .withKerberosDelegationToken(token)
                          ))
                              .build();
    1. Thanks Ishan! I added this in the page.