In addition to this guide, also see the latest Flink documentation on the website.

Requirements for developing and building Flink

• Unix-like environment (We use Linux, Mac OS X, and Cygwin)
• git
• Maven (at least version 3.1.1)
• Java 8
• Python

Clone the repository

Apache Flink’s source code is stored in a git repository which is mirrored to GitHub. The common way to exchange code on GitHub is to fork the repository into your personal GitHub account. For that, you need to have a GitHub account or create one for free. Forking a repository means that GitHub creates a copy of the forked repository for you. This is done by clicking on the Fork button on the upper right of the repository website. Once you have a fork of Flink’s repository in your personal account, you can clone that repository to your local machine.

git clone https://github.com/<your-user-name>/flink.git

The code is downloaded into a directory called flink.

Azure Pipelines for CI

See the Azure Pipelines Wiki page for details on setting up Azure for CI.

Proxy Settings

If you are behind a firewall you may need to provide Proxy settings to Maven and your IDE.

For example, the WikipediaEditsSourceTest communicates over IRC and need a SOCKS proxy server to pass.

Setup an IDE and import the source code of Java/Scala

The Flink committers use IntelliJ IDEA (recommended) and Eclipse IDE to develop the Flink Java/Scala code base.

Minimal requirements for an IDE are:

• Support for Java and Scala (also mixed projects)
• Support for Maven with Java and Scala

IntelliJ IDEA

The IntelliJ IDE supports Maven out of the box and offers a plugin for Scala development.

• IntelliJ download: https://www.jetbrains.com/idea/
• IntelliJ Scala Plugin: http://plugins.jetbrains.com/plugin/?id=1347

Check out our Setting up IntelliJ guide for details.

Eclipse Scala IDE

For Eclipse users, we recommend using Scala IDE 3.0.3, based on Eclipse Kepler. While this is a slightly older version, we found it to be the version that works most robustly for a complex project like Flink.

Further details and a guide to newer Scala IDE versions can be found in the How to setup Eclipse docs.

Note: Before following this setup, make sure to run the build from the command line once (mvn clean install -DskipTests; see below).

1. Download the Scala IDE (preferred) or install the plugin to Eclipse Kepler. See How to setup Eclipse for download links and instructions.
2. Add the “macroparadise” compiler plugin to the Scala compiler. Open “Window” -> “Preferences” -> “Scala” -> “Compiler” -> “Advanced” and put into the “Xplugin” field the path to the macroparadise jar file (typically “/home/-your-user-/.m2/repository/org/scalamacros/paradise_2.10.4/2.0.1/paradise_2.10.4-2.0.1.jar”). Note: If you do not have the jar file, you probably did not run the command line build.
3. Import the Flink Maven projects (“File” -> “Import” -> “Maven” -> “Existing Maven Projects”)
4. During the import, Eclipse will ask to automatically install additional Maven build helper plugins.
5. Close the “flink-java8” project. Since Eclipse Kepler does not support Java 8, you cannot develop this project.

Import the source code

Apache Flink uses Apache Maven as build tool. Most IDEs are capable of importing Maven projects.

Setup an IDE and import the source code of Python

The Flink committers use PyCharm(recommended) to develop Flink Python code base.

PyCharm

The PyCharm is a friendly IDE to supports python development.

• PyCharm download: https://www.jetbrains.com/pycharm/

Check out our Setting up PyCharm guide for details.

Import the source code

The Python code of Apache Flink can be opened by PyCharm directly. The open path should be pointed to the flink-python module.

Build the code

To build Flink from source code, open a terminal, navigate to the root directory of the Flink source code, and call:

mvn clean package

This will build Flink and run all tests(without python test case). Flink is now installed in build-target.

To build Flink without executing the tests you can call:

mvn -DskipTests clean package


Run a single/set of tests with maven

If you want to run a single/set of tests you can do this via maven:

mvn verify -Dtest=TestToRun

If you build multiple modules as part of the test run you might want to add -DfailIfNoTests=false:

mvn verify -Dtest=TestToRun -pl flink-runtime -am -DfailIfNoTests=false

Note that Flink uses the surefire plugin to run unit and integration tests. That is why you can also specify integration tests via -Dtest=SomeITCase.

Run Python Test

After building the Flink source code, you can run Python test in the flink-python module:

sh dev/lint-python.sh