Apache Solr Documentation

6.3 Ref Guide (PDF Download)
Solr Tutorial
Solr Community Wiki

Older Versions of this Guide (PDF)

6.4 Draft Ref Guide Topics

Meta-Documentation

This Unreleased Guide Will Cover Apache Solr 6.4

Skip to end of metadata
Go to start of metadata

With SolrCloud your configuration files are kept in ZooKeeper. These files are uploaded in either of the following cases:

  • When you start a SolrCloud example using the bin/solr script.
  • When you create a collection using the bin/solr script.
  • Explicitly upload a configuration set to ZooKeeper.

Startup Bootstrap

When you try SolrCloud for the first time using the bin/solr -e cloud, the related configset gets uploaded to ZooKeeper automatically and is linked with the newly created collection.

The below command would start SolrCloud with the default collection name (gettingstarted) and default configset (data_driven_schema_configs) uploaded and linked to it.

You can also explicitly upload a configuration directory when creating a collection using the bin/solr script with the -d option, such as:

The create command will upload a copy of the data_driven_schema_configs configuration directory to ZooKeeper under /configs/mycollection. Refer to the Solr Control Script Reference page for more details about the create command for creating collections.

Once a configuration directory has been uploaded to ZooKeeper, you can update them using the Solr Control Script

It's a good idea to keep these files under version control.

Uploading Configuration Files using bin/solr or SolrJ

In production situations, Config Sets can also be uploaded to ZooKeeper independent of collection creation using either Solr's Solr Control Script or the CloudSolrClient.uploadConfig java method.

The below command can be used to upload a new configset using the bin/solr script.

It is strongly recommended that the configurations be kept in a version control system, Git, SVN or similar.

Managing Your SolrCloud Configuration Files

To update or change your SolrCloud configuration files:

  1. Download the latest configuration files from ZooKeeper, using the source control checkout process.
  2. Make your changes.
  3. Commit your changed file to source control.
  4. Push the changes back to ZooKeeper.
  5. Reload the collection so that the changes will be in effect.

Preparing ZooKeeper before first cluster start

If you will share the same ZooKeeper instance with other applications you should use a chroot in ZooKeeper. Please see Taking Solr to Production#ZooKeeperchroot for instructions.

There are certain configuration files containing cluster wide configuration. Since some of these are crucial for the cluster to function properly, you may need to upload such files to ZooKeeper before starting your Solr cluster for the first time. Examples of such configuration files (not exhaustive) are solr.xml, security.json and clusterprops.json.

If you for example would like to keep your solr.xml in ZooKeeper to avoid having to copy it to every node's solr_home directory, you can push it to ZooKeeper with the bin/solr utility (Unix example):


  • No labels

21 Comments

  1. Possible typo, first table, first row, rightmost column

     

    I think -bootstrap_confdir=<directory> is maybe supposed to be -Dbootstrap_confdir=<directory>

  2. zkcli.sh -cmd upconfig, I wonder whether `-solrhome` is needed, the command line help of zkcli.sh upconfig does not contain `-solrhome` option.

    If need solr home,  I have two cores and two replica, I would set it to 'example/cloud/node1/solr' or 'example/cloud/node1/solr/xx_shard1_replica1' ?

    Thanks

    1. I have never used the -solrhome option with upconfig.  My SolrCloud version is only 4.2.1, and I don't think that option even existed then.  As far as I know, only -confdir and -confname are required options beyond -cmd and -zkhost, which are required for all commands.

  3. I use solrcloud 5.1.0, I add some fields, use zkcli.sh upconfig the configs files, and restart solr.

    But the when I insert data with added fields, commit & optimized, the solr query result doesn't contain the new added fields yet.

    Is there any wrong with my steps? Or do I need some other action to make my query results contain the added fields? Thanks very much.

    1. What exactly did you add to your schema?  Were the new fields set to stored="true"?  Are you sure that the documents with the new fields matched the query you submitted?  Any document indexed before you updated the schema would not have those fields.

      1. I just want to add some fields, and I set stored="true".

        I tried insert new docs contains fields I added, but restart solrcloud and commit & optimized . New docs appear, but added fields not appear.

  4. I referenced security.json in the new paragraph "Preparing ZooKeeper before first cluster start". Is that ok or should we wait until Authentication and Authorization Pluginspage is published?

    1. I see that Security page just got published, so then it should be ok (smile)

  5. Do we need something here about how to create a chroot in zookeeper for users who want one?  I personally think we should be recommending a chroot for all installs, and letting users know it is optional.

    1. Agree that we should recommend chrooot as default. We could mention it and link to Taking Solr to Production#ZooKeeperchroot ? See also my comment on that page, can't we use "makepath" to create the chroot - do we need "bootstrap"?

  6. Pretty much new to ZooKeeper with SOLR. But here goes . . . 

    I have a ZooKeeper cluster running.  When I start Solr Cloud as follows

    ./solr -e cloud -z xx.xx.xx.xx:2181,xx.xx.xx.xy:2181,xx.xx.xx.xz:2181

    Then after I execute

    $ bin/solr create -c mycollection -d data_driven_schema_configs

    I get the following

    ERROR: Error loading config name for collection gettingstarted

    And in solr.log

    INFO  - 2015-10-19 16:43:38.587; [   ] org.apache.solr.common.cloud.ZkStateReader$2; Updated cluster state version to 8
    
    INFO  - 2015-10-19 16:43:41.543; [c:gettingstarted s:shard2 r:core_node1 x:gettingstarted_shard2_replica2] org.apache.solr.cloud.ZkController; Could not find collection configName - pausing for 3 seconds and trying again - try: 2
    
    INFO  - 2015-10-19 16:43:41.547; [c:gettingstarted s:shard1 r:core_node2 x:gettingstarted_shard1_replica2] org.apache.solr.cloud.ZkController; Could not find collection configName - pausing for 3 seconds and trying again - try: 2
    
    INFO  - 2015-10-19 16:43:42.125; [   ] org.apache.solr.servlet.HttpSolrCall; [admin] webapp=null path=/admin/info/system params={wt=json} status=0 QTime=26
    
    INFO  - 2015-10-19 16:43:42.197; [   ] org.apache.solr.handler.admin.CollectionsHandler; Invoked Collection Action :clusterstatus with params action=CLUSTERSTATUS&wt=json
    
    INFO  - 2015-10-19 16:43:42.204; [   ] org.apache.solr.common.cloud.ZkStateReader; Load collection config from:/collections/gettingstarted
    
    ERROR - 2015-10-19 16:43:42.208; [   ] org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: Error loading config name for collection gettingstarted
    
            at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:162)
    
            at org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(ClusterStatus.java:136)
    
            at org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$19.call(CollectionsHandler.java:614)
    
            at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:166)
    
            at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
    
            at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:675)
    
            at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443)
    
            at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
    
            at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
    
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
    
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
    
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
    
           

     

    I don't see how "gettingstarted" has anything to do with "mycollection"


    The following works however

    ./solr -e cloud -noprompt
    ./solr create -c mycollection -d data_driven_schema_configs
    
    Connecting to ZooKeeper at localhost:9983 ...
    
    Uploading /solr3/solr-5.3.1/server/solr/configsets/data_driven_schema_configs/conf for config mycollection to ZooKeeper at localhost:9983
    
    Creating new collection 'mycollection' using command:
    
    http://localhost:7574/solr/admin/collections?action=CREATE&name=mycollection&numShards=1&replicationFactor=1&maxShardsPerNode=1&collection.configName=mycollection
    
    {
    
      "responseHeader":{
    
        "status":0,
    
        "QTime":2253},
    
      "success":{"":{
    
          "responseHeader":{
    
            "status":0,
    
            "QTime":2056},
    
          "core":"mycollection_shard1_replica1"}}}

     

    I have verified that my Zoo Keeper cluster SEEMS to be working fine.

    1. When you start the cloud example (-e cloud), the first thing it does (once the cloud is running) is create a collection named "gettingstarted".  I'm guessing something went wrong with that process.

      We may have a bug in the start scripts when you combine "-e cloud" with the -z option to use your own Zookeeper ensemble.

      1. Shawn I am pretty sure that's it exactly. To get around this error I had to do upload a dummy config to ZooKeeper and link the 'gettingstarted' collection to it before start

         

        ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir ./server/solr/configsets/data_driven_schema_configs/conf -confname myconf -z xx.xx.xx.xx:2181
        ./server/scripts/cloud-scripts/zkcli.sh -cmd linkconfig -collection gettingstarted -confname myconf -z xx.xx.xx.xx:2181
  7. I read an aritcle said we can put some library files on zk for Solrcloud to access. If true, I'd suggest put some example here.

    1. Is this (already in the reference guide) what you are talking about?

      https://cwiki.apache.org/confluence/display/solr/Blob+Store+API

  8. To update or change your SolrCloud configuration files:

    1. Download the latest configuration files from ZooKeeper, using the source control checkout process.
    2. Make your changes.
    3. Commit your changed file to source control.
    4. Push the changes back to ZooKeeper.
    5. Reload the collection so that the changes will be in effect.

    I just want to know how to "Download the latest configuration files from ZooKeeper". All examples on internet is about pushing the changes back to ZooKeeper. Is there any example? 

    1. Hi, here's a very short example of downloading and uploading specific configuration files from Zookeeper:

      1. Thanks so much, it does solve my problem.

      2. You can also use the bin/solr script, with the zk command, as described in the section: Solr Control Script Reference#ZooKeeperOperations. This page needs to be updated for that new option.