You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 42 Next »

This is a Work In Progress. Please feel free to edit if you have permissions or ping the dev list if you have suggestions.

Starting with Solr 9, we have the capability of building with both Ant and Gradle. How long we'll continue to support Ant is under discussion; Gradle will be preferred going forward. This page is intended to help people use Gradle to build Solr, with special attention to tips for people already familiar with the Ant build process.

I want to especially thank Dawid Weiss for all his work on this, without his help and guidance (ok, his willingness to do most of the heavy lifting), we wouldn't be able to even consider transitioning to Gradle yet or even in the foreseeable future.


Gdub

There's a helpful project at: https://github.com/dougborg/gdub that lets you run targets from submodules rather than having to do everything top level with gradlew.
Where you might normally do ./gradlew solr:core:test from the root directory, gdub lets you do cd solr/core; gw test. This is not necessary, just a convenience.

Setup

If you get through these steps, you're ready to start using Gradle regularly:

  1. Install Gradle, see: https://docs.gradle.org/current/userguide/installation.html
  2. Get a current version of Lucene/Solr with the usual "git clone https://github.com/apache/lucene-solr.git dest"
  3. cd dest
  4. ./gradlew help
  5. ./gradlew helpAnt - this will give you the Gradle commands to use to do operations you already know how to do in Ant.
  6. At this point, Gradle is installed and running and you're able to start using it regularly.


Gradle on trunk/9.0 only

Gradle is only supported as of Solr 9.0+ (current trunk, unreleased as of July 2020). However, since that's where almost all development is done, we should start using Gradle there.

Currently, you can use either Ant or Gradle to build trunk. Gradle already has features that Ant does not. Ant will likely fall further behind Gradle as time progresses.

Useful commands:

Here are some commands that will get you started (all prefixed by ./gradlew or just gw if you've installed gdub).

  • help - of course.
  • helpAnt - lists Gradle commands that are the equivalent of what you're used to in Ant.
  • assemble -  will create a runnable Solr instance in ./solr/packaging/build/solr-#.#.#-SNAPSHOT. However, it will delete that directory next time it's run.
  • dev - will do what assemble does, but will write the output code to ./solr/packaging/build/dev and will not delete the directory first. Use this if you're developing code, instantiating Solr and recompiling and need to preserve, say, your collections.
  • tasks - lists all the tasks you can execute.
  • check - will run all of the checks (ant precommit+) and run all of the tests. Please get in the habit of running this! More below for why in "Differences from Ant".
    • Adding -x test will run all the validation checks but not run tests (this is the rough equivalent of ant precommit)

Differences from Ant:

Some operations that are simply different from Ant. Here are some hints to help with these:

  • Gradle runs fastest when it uses a daemon. The first time you use gradle, it'll take some time (usually 10-20 seconds) to start the daemon and build caches. You'll see a message about "starting daemon". Subsequent runs should mostly skip this step
    • You may want to change org.gradle.daemon.idletimeout=900000 in your gradle.properties file (written the first time you run gradle) to a longer interval (this is in ms) to avoid restarting the daemon as frequently.
  • The gradle build will fail on un-suppressed warnings. Trunk compiles cleanly, all warnings are suppressed or have been fixed (mostly suppressed). The first option if your new code generates more warnings is to change the code so it doesn't. Failing that add a SuppressWarnings annotation, but please try to fix the code first.
    • If you're working on code and can remove some of these suppressions, please do. It was too big a task to try to fix all the warnings by fixing the code all at once. We should improve from here.
  • The gradle validation checks fail on log messages that follow wasteful patterns. ./gradlew helpValidateLogCalls describes how to avoid these errors. This is part of ./gradlew check.
  • Test classes are not compiled by the assemble and classes tasks and are not part of the packaging. The testClasses task will compile test classes.
  • It's not necessary to run the idea task before opening the project, just "open or import" and choose the root directory of your source tree.

Running from inside IntelliJ

IntelliJ integration is useful. There's nothing special about our support for Idea, it's just the one that's gotten the most exercise. In particular the gradle window lets you easily execute tasks. In the case of failures because of the -Werror flag and just click to the source. The classes and testClasses tasks are particularly useful here as they just compile the Java code without the extra work the assemble task does.

Running tests in IntelliJ seems to take longer, we're investigating why.

Please do not let IntelliJ "fix" it's inspections by adding the IntelliJ-specific suppression. That said, it's useful to look at the inspections, they may be highlighted. Some of them are not useful, any tips on how to turn off specific ones appreciated. For instance, I personally like if (something == false) because I think if (!something) is easy to misread, but IntelliJ prefers the latter. There's no intention here to make all our code conform to IntelliJs inspections. That said, looking at them can be instructive.

Other IDEs:

Please feel free to add any tips about Eclipse, Netbeans etc. As mentioned above, IntelliJ was just what has been used most.

Troubleshooting:

We've run into some rough edges that are outlined here:

  • When switching back and forth between Gradle on trunk and 8x where you have to use ant, there are some Gradle-specific files that pollute your 8x tree. Delete them freely.
    • But note that when you go back  to using Gradle it'll write a new "gradle.properties" file at the root of your source tree that you may have to re-edit if you've changed it.
  • I've occasionally seen gradle stop with a message about thrashing memory when running the check task. There are two short-term solutions here: 
    • ./gradlew --stop will halt the daemon, releasing memory
    • Increase the memory allocated to Gradle by editing the settings in your gradle.properties file.
    • On the TODO list is to figure out why 3G (the current default which I sometimes bump even higher) and address whatever is causing that.





  • No labels