In order to setup your development environment, you would need a Linux system with administrative privileges and Internet connection such as Ubuntu or CentOS. You would also need sufficient disk space for checking out and building the code, installing various database/other software that you may need for your testing.



Please see the docs for the latest release of Sqoop 1 http://sqoop.apache.org/docs/. Some of the information below might be outdated

Getting ready to build

Once you have your Linux system ready with sufficient disk space and Internet connection, go ahead and install the following software:

Building the Sources

To get the source code, checkout the subversion "trunk" using the following command:

$ git clone https://git-wip-us.apache.org/repos/asf/sqoop.git

Once you have the code, you can build it by the following command:

$ cd sqoop
$ ant jar-all

You can use the clean target to delete previously built files from the workspace and run jar-all again to do a fresh build.

To see a list of all available targets that are available in the build type the following command:

$ ant -p

If you prefer working in Eclipse, you can generate the necessary project definitions as follows:

ant eclipse

Once these definitions are generated, you can import them in Eclipse as an existing project.

Running Tests

Running unit tests

Sqoop source code contains many unit tests that exercise its functionality. These tests can be run simply by using the following command:

ant test

Create third-party lib directory

Create a directory somewhere convenient on your development system. This directory will hold all the JDBC drivers that the tests will use. Once created, create (or edit) the build.properties file in Sqoop workspace root directory and set the the full path of this directory as the value of the property sqoop.thirdparty.lib.dir. For example:

sqoop.thirdparty.lib.dir=/opt/ws/3rd-party-lib

Setting up and running third-party tests

Third-party tests are end-to-end integration tests that exercise the basic Sqoop functionality against third-party databases. You should run these tests in order to rule out regression when testing any changes to the core system. Before you run these tests, you must setup the following databases:

Setting up MySQL
Setting up PostgreSQL
Setting up Oracle
Running third-party tests

Once you have installed and configured all the above databases - MySQL, PostgreSQL and Oracle, you are now ready to run the third-party tests. To run them issue the following command:

$ ant test -Dthirdparty=true

Setting up and running manual tests

Certain third-party tests are categorized as Manual tests since these were introduced at a later stage and adding them to the third-party suite of tests would have resulted in ever test environment requiring new database installation.

Setting up SQL Server
Setting up DB2 Server

The default username, password and database can be also overridden using following properties:

Running manual tests

Once you have installed and configured all the above databases - SQL Server and DB2, you are now ready to run the manual tests. To run them, issue the following command:

$ ant test -Dmanual=true

Building documentation

To build Sqoop documentation, run the following command from the workspace root directory:

$ ant docs

This will generate the documentation in the directory build/docs directory. To see the documentation, open the file build/docs/index.html in a web browser, where you will find the links to user and developer guides. All the man pages that are generated by this are available directly under build/docs directory with the extension <name>.1.gz. You can look at these man pages without installing them by the following comamnd:

$  man -l sqoop.1.gz

Building tar-ball

To build the tar-ball for distribution, use the following command:

$ ant tar

This will produce a tar-ball distribution file with a name sqoop-<version>.tar.gz under the build directory.