Sling Jenkins Setup

History and current status

We maintain a (large) number of jobs on the ASF Jenkins instance. Historically, we had a few large reactor builds but those proved to be of limited usefulness:

• the builds were quite slow to complete - 50 minutes on average
• if a module in the reactor failed, the whole build failed

In practice, this meant that the Sling CI jobs were red almost all of the time.

To solve these issues we have decided to create individual jobs for each module. Given that Sling has a large number of modules, it is unrealistic to manage them all manually. As such, we have started using the Jenkins Job DSL Plugin . This allows us to programatically create jobs based on a groovy script.

Views

We currently have the following views defined:

• Sling - holds all Sling jobs
• Sling-Dashboard - holds all Sling jobs that need attention ( failed, unstable )

Managing jobs

Since all the job management is now done via a Groovy script manual job creation is discouraged. Altering an existing job is also discouraged since the changes will be undone when the jobs are next regenerated.

Jobs are managed by using a 'seed' job, which processes the DSL script. The job is currently located at https://builds.apache.org/view/S-Z/view/Sling/job/sling-seed-build . Typically there is no need to touch this job.

The script controlling the job management is located at https://svn.apache.org/repos/asf/sling/trunk/tooling/jenkins/create_jobs.groovy . To add a new job, add a new entry in the 'modules' list, for instance

diff --git a/tooling/jenkins/create_jobs.groovy b/tooling/jenkins/create_jobs.groovy
index 871010c..692e246 100644
--- a/tooling/jenkins/create_jobs.groovy
+++ b/tooling/jenkins/create_jobs.groovy
@@ -210,6 +210,9 @@ def modules = [
         location: 'bundles/extensions/repoinit/parser'
     ],
     [
+        location: 'bundles/extensions/foo'
+    ],
+    [
         location: 'bundles/jcr/api'
     ],
     [

Commit the change to SVN and on the next invocation of the seed job your desired job will be created.

Managing inter-job dependencies

The downstream property can be defined for a job to manually define dependencies. Typically it's not required as the Maven Jenkins plug-in automatically infers the dependencies from the pom.xml files. However, this is currently not supported for the provisioning model, as the Maven plugin is unaware of those dependencies. In that scenario, dependencies need to be declared manually.

Also note that if a job has manual dependencies and is built for multiple Java versions, only the first job will have declared dependencies. The reason is that only the first job will deploy artifacts, and creating dependencies for the second job will lead to useless job triggers.

Pax-Exam tests with SNAPSHOT dependencies

Pax-Exam tests using SNAPSHOT versions are problematic since they have no way of knowing the general Maven context and they don't obey the custom Maven repositories. As such, we need to manually define the Apache SNAPSHOT repository for those tests. One issue were this was done is SLING-6079 - Getting issue details... STATUS .

If you're using the org.apache.sling.testing.paxexam module, this is already incorporated as of version 0.0.3-SNAPSHOT.

Performing more complex changes

If you wish to perform more complex changes, affecting the logic of the script, it's recommended to test these changes on a separate Jenkins instance. This way you minimise the chances that something goes wrong on the ASF Jenkins instance and you get very quick feedback. As of 2016-10-01, a run on sling-seed-build on builds.apache.org takes between 10 and 90 minutes. The fastest way to setup Jenkins locally ( until we fix SLING-6062 - Getting issue details... STATUS ) is to use a local docker instance.

docker run -p 8080:8080 -p 50000:50000 -v `pwd`/data:/var/jenkins_home jenkins

After configuring Jenkins for the first time, perform the following steps:

1. Install the following plug-ins:
   1. Job DSL plug-in
   2. Javadoc plug-in
   3. Maven plug-in
   4. Test stability plug-in
2. Configure the following Jenkins tools:
   1. JDK 1.7 (latest)
   2. JDK 1.8 (latest)
   3. Maven 3.3.9
3. Create a Freestyle job with a Job DSL build step. You can either have it pull from https://svn.apache.org/repos/asf/sling/trunk/tooling/jenkins, and run the create_jobs.groovy script, or use an inline script for faster prototyping

If you wish to test the actual job execution, label the master with the "Ubuntu" label. Otherwise, only the seed job will run.

Limitations

We rely on the Jenkins Maven Plugin to automatically detect relationship between jobs, e.g. from org.apache.sling.distribution.core to org.apache.sling.distribution.api . However, this does not take into account the provisioning model. To work around this, we currently set up some very broad rules, such as 'all entries under bundles/ or installer/ trigger a launchpad build'. Also, we manually set some dependencies between some of the launchpad testing jobs. We might revisit this to see if we can do something more fine-grained.

References:

1. Jenkins Job DSL Plugin - https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
2. SLING-6061 - Getting issue details... STATUS
3. dev@sling.apache.org - sling ci builds per module : http://markmail.org/message/l6wc4bu5pzw5td3d
4. dev@sling.apache.org - CI alternatives for Sling:  http://markmail.org/message/mdn4anwe6kxqxa2z
5. builds@apache.org - Splitting a large build into _many_ smaller builds : http://mail-archives.apache.org/mod_mbox/www-builds/201609.mbox/%3C1472811065.24075.8.camel%40apache.org%3E