Page History

Note: this page is

This page is a quick HOWTO explaining how to easily contribute patches to Nutch. It assumes you are using some form of Unix.

Getting the source code

First of all, you need the Nutch source code.

Create a directory in which you want to store the Nutch source code on your local drive, clone the Nutch git repository and cd into the Nutch project folder:

> cd somewhereOnYourDisk
> git mkdir nutchclone https://github.com/apache/nutch.git
> cd nutch

then get the source code on your local drive using SVN.

...

Alternatively, you can use Apache's gitbox mirror. Please see UsingGit for further details.

Working time

Now it is time to work.

...

• All public classes and methods should have informative javadoc.
• Unit tests are encouraged (http://www.junit.org).

Building a patch

First of all, please perform some minimal non-regression tests by:

• rebuilding the whole Nutch code
• executing the whole unit tests.

Building Nutch

> cd somewhereOnYourDisk/nutch
> ant

After a while, if you see

BUILD SUCCESSFUL

all is ok, but if you see

BUILD FAILED

please, read carefully the errors messages and check your code.

Unit Tests

> cd somewhereOnYourDisk/nutch
> ant test

After a while, if you see

BUILD SUCCESSFUL

all is ok, but if you see

BUILD FAILED

please, read carefully the errors messages and check your code. Detailed error logs are found in `build/test` (core classes) or `build/my-plugin/test` for plugins (here the plugin "my-plugin").

It is possible to run individual unit tests (useful during development), see :

• run unit tests from a single core test class (use class name without package path):

  No Format
  ant test-core -Dtestcase=TestMimeUtil

  
  

• run unit tests for a specific plugin:

  No Format
  ant test-plugin -Dplugin=protocol-okhttp

  
  

• or exclude test files by patterns:

  No Format
  ant -Dtest.exclude='TestCrawlDb*.java **/TestNutchServer*' test

  
  

See also bin/nutch junit and WritingPluginExample-1.2#Unit_testing.

...

If you are perfectionist you can also perform some functional tests by running Nutch. Please refer to the NutchTutorial

Opening a Pull-Request on Github

See the README on https://github.com/apache/nutch and the pull-request template how to open a pull-request on Github.

Creating a patch

Although a pull-request on Github is the prefered way of contribution, we still accept patches (not all contributors are on Github). In order to create a patch, just type from the root of the Nutch directory :

svn diff > myBeautifulPatch.patch
vi myBeautifulPatch.patch

or

git diff --no-prefix > myBeautifulPatch.patch
vi myBeautifulPatch.patch

...

This will report all modifications done on Nutch sources on your local disk and save them into the myBeautifulPath.patch file. Then edit the patch file in order to check that it includes ONLY the modifications you want to add to the Nutch SVN git repository.

Remember to generate a patch against a live branch, i.e. trunk master for Nutch 1.x and 2.x for Nutch 2.x. The other branches are snapshots of past releases and the code might have evolved since then.

...

1. try to reproduce the problem. If you're not able to reproduce a problem, it's impossible to test whether a patch really resolves it.
2. try to get a clear understanding of the problem
3. have a look at the patch file: Ideally, you'll get and understanding about the solution proposed by the patch (no problem if not)
4. apply the patch, see below #Applying_patches
5. build Nutch and test whether the problem disappeared
6. does the patch break other things (add regressions)?

...

1. report your finding in Jira or on nutch-dev. It's always better to have one review more, than to introduce a regression because of insufficient testing.

Applying patches

A properly generated patch can be automatically applied to the source tree. The patch utility is one tool to apply patches. Change into the Nutch root folder and run:

> patch -p0 <path_to.patch

or

git apply path_to.patch

Do not ignore the output of patch, it may indicate errors. Applying a patch may (partially) fail, if the source code has changed meanwhile. A good starting point to learn more about patches it the Wikipedia article Patch.