Apache Solr Documentation

6.5 Ref Guide (PDF Download)
Solr Tutorial
Solr Community Wiki

Older Versions of this Guide (PDF)

Ref Guide Topics

Meta-Documentation

*** As of June 2017, the latest Solr Ref Guide is located at https://lucene.apache.org/solr/guide ***

Please note comments on these pages have now been disabled for all users.

Skip to end of metadata
Go to start of metadata

In addition to the logging options described below, there is a way to configure which request parameters (such as parameters sent as part of queries) are logged with an additional request parameter called logParamsList. See the section on Common Query Parameters for more information.

Temporary Logging Settings

You can control the amount of logging output in Solr by using the Admin Web interface. Select the LOGGING link. Note that this page only lets you change settings in the running system and is not saved for the next run. (For more information about the Admin Web interface, see Using the Solr Administration User Interface.)

 
The Logging screen.

This part of the Admin Web interface allows you to set the logging level for many different log categories. Fortunately, any categories that are unset will have the logging level of its parent. This makes it possible to change many categories at once by adjusting the logging level of their parent.

When you select Level, you see the following menu:

 
The Log Level Menu.

Directories are shown with their current logging levels. The Log Level Menu floats over these. To set a log level for a particular directory, select it and click the appropriate log level button.

Log levels settings are as follows:

Level

Result

FINEST

Reports everything.

FINE

Reports everything but the least important messages.

CONFIG

Reports configuration errors.

INFO

Reports everything but normal status.

WARN

Reports all warnings.

SEVERE

Reports only the most severe warnings.

OFF

Turns off logging.

UNSET

Removes the previous log setting.

Multiple settings at one time are allowed.

Log level API

There is also a way of sending REST commands to the logging endpoint to do the same. Example:

Choosing Log Level at Startup

You can temporarily choose a different logging level as you start Solr. There are two ways:

The first way is to set the SOLR_LOG_LEVEL environment variable before you start Solr, or place the same variable in solr.in.sh / solr.in.cmd. The variable must contain an uppercase string with a supported log level (see above).

The second way is to start Solr with the -v or -q options, see Solr Control Script Reference for details. Examples:

Permanent Logging Settings

Solr uses Log4J version 1.2 for logging and is configured using server/resources/log4j.properties. Take a moment to inspect the contents of the log4j.properties file so that you are familiar with its structure. By default, Solr log messages will be written to SOLR_LOGS_DIR/solr.log.

When you're ready to deploy Solr in production, set the variable SOLR_LOGS_DIR to the location where you want Solr to write log files, such as /var/solr/logs. You may also want to tweak log4j.properties. Note that if you installed Solr as a service using the instructions provided at Taking Solr to Production, then see /var/solr/log4j.properties instead of the default server/resources version.

When starting Solr in the foreground (-f option), all logs will be sent to the console, in addition to solr.log. When starting Solr in the background, it will write all stdout and stderr output to a log file in solr-<port>-console.log, and automatically disable the CONSOLE logger configured in log4j.properties, having the same effect as if you removed the CONSOLE appender from the rootLogger manually.

Also, in  log4j.properties the default log rotation size threshold of 4MB is most likely too small for production servers and should be increased to a larger value (such as 100MB or more).

Java Garabage Collection logs are rotated by the JVM when size hits 20M, for a max of 9 generations. Old GC logs are moved to SOLR_LOGS_DIR/archived. These settings can only be changed by editing the start scripts.

On every startup of Solr, the start script will clean up old logs and rotate the main solr.log file. If you changed the log4j.appender.file.MaxBackupIndex setting in log4j.properties, you also need to change the corresponding setting -rotate_solr_logs 9 in the start script.

You can disable the automatic log rotation at startup by changing the setting SOLR_LOG_PRESTART_ROTATION found in  bin/solr.in.sh or bin/solr.in.cmd to false. 

Logging Slow Queries

For high-volume search applications, logging every query can generate a large amount of logs and, depending on the volume, potentially impact performance. If you mine these logs for additional insights into your application, then logging every query request may be useful. On the other hand, if you're only concerned about warnings and error messages related to requests, then you can set the log verbosity to WARN. However, this poses a potential problem in that you won't know if any queries are slow, as slow queries are still logged at the INFO level. Solr provides a way to set your log verbosity threshold to WARN and be able to set a latency threshold above which a request is considered "slow" and log that request at the WARN level to help you identify slow queries in your application. To enable this behavior, configure the <slowQueryThresholdMillis> element in the query section of solrconfig.xml:

Any queries that take longer than the specified threshold will be logged as "slow" queries at the WARN level.

  • No labels

14 Comments

  1. It strikes me as a little odd that this page links to the old wiki.  IMHO that info should be incorporated here, or possibly on a new cwiki page.  If nobody else is clamoring to fix this, I can work on it over the next few days.  At the very least, I'll get an issue in JIRA later today so we don't lose track of it.  I may need some guidance, especially if consensus is that we create a new page on this wiki.

  2. Proposed addition with SOLR-6919. Actual log name subject to change.

    Logging Queries Prior To Execution

    By default, all queries are logged after they complete, with additional information such as result count and query time. However, you may find yourself in a situation where certain queries do not complete, and thus never get logged. For increased visibility into these queries you have two options:

    1) Consult the Jetty access logs and compare the results with your Solr logs. The queries in the access log not present in the other logs are the ones that do not complete.

    2) Enable pre-query logging in Solr. This will cause queries to be logged before they are executed, but without the useful metrics, like query time, mentioned earlier. You can enable this by setting:

  3. In multi-shard setup it is useful to be able to change log level in runtime without going to each and every shard's admin page.

    For example, we set the logging to WARN level during massive posting sessions and back to INFO, when serving user queries.

    In solr 4.10.2 these one-liners do the trick:

     

    curl -s http://localhost:8983/solr/admin/info/logging --data-binary "set=root:WARN&wt=json"

    curl -s http://localhost:8983/solr/admin/info/logging --data-binary "set=root:INFO&wt=json"

    1. Hey buddy thanks a lot for that command. Can you help me out with getting the query logs from solr with similar kind of  "curl" command, that would really be helpful for me.

       

      Thanks in advance

      1. By looking at the admin UI logging tab in my browser with firebug open, I was able to see the requests that the admin UI makes to retrieve logging data.  This is what I found:

        http://server:port/solr/admin/info/logging?wt=json&since=1432560329768

        The "since" value is a timestamp in the internal java representation, which is similar to the UNIX epoch, expressed in milliseconds instead of seconds.

        1. Oh, you were after the query logs.  Those are at INFO, which normally doesn't show up in the admin UI.  I don't know if there's an option you can give the logging handler to see INFO messages.

    2. I added this as an example in the page above. Is the logging API documented somewhere else in this guide?

  4. I want to change the name of log file for solr from solr.log to solr.log.TimeStamp (here timestamp is the time Solr was started). Can someone please let me know how to do that? Thanks in advance

    1. You'd probably need to ask on the log4j mailing list to make sure, but I suspect that this isn't going to be something log4j can do.  You can very likely add something to the solr start script that sets a system property with the start timestamp, which you can then use in log4j.properties to describe the filename.

      1. Thanks Shawn..that worked

  5. The table says "WARNING", but the correct setting in the log4j.properties file is "WARN".

  6. Could you please help me with the exact commands to set logging level to "ERROR " for both solr and zookeeper ?  Need to set parameters along with below commands ( Which M already using )
    solr -c -m 8g -p 8983 -z

    Thanks

     

    1. Please ask questions like this on the Solr User mailing list and not in the documentation comments, which is for reporting documentation problems.