Page tree
Skip to end of metadata
Go to start of metadata

Highlights

The location ./test-infra/jenkins contains the defined Jenkins jobs are defined in  These jobs are written using the Job DSL using Apache Groovy. Job definitions should be a simple as possible and ideally identify a single Gradle target to execute.

Testing Changes

The following are some Testing Changes tips that could help you:

  • Use Local Dockerized Jenkins to test changes locally without breaking the project’s job definitions. 
  • Test a PR that includes Jenkins changes, but it is potentially destructive because the job definitions are a shared resource.
  • Use the “Run Seed Job” trigger phrase to update the job definitions on a PR. This will cause the job definitions to be read and parsed, and they will be active until the next scheduled execution on beam_SeedJob or beam_SeedJob_Standalone.

Triggering jobs

Beam committers can trigger a job with the Jenkins UI. Non-committers can trigger a job if there is a trigger phrase.

Job suffixes

Each post-commit and pre-commit job file defines several jobs with different suffixes.

  • For pre-commits, there are _Commit, _Phrase, and _Cron suffixes.
  • The _Commit job happens with every push to a pull request.
  • The _Phrase happens when the trigger phrase is entered as a comment in the pre-commit.
  • The _Cron pre-commit happens post-commit on the master branch as a signal of whether the pre-commit would pass without any changes.

Job labels

Consider the following tips regarding Jenkins-jobs-related tags:

  • Most Beam Jenkins jobs specify the label beam, which uses beam executors 1-15.
  • Performance test jobs specify the label beam-perf, which uses beam executors 16.
  • Website publishing job specifies the label git-websites, which allows publishing generated documentation to the asf-site branch.

How to access a test history

To access a test history:

  1. Look at this URL:
    https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/lastCompletedBuild/testReport/junit/org.apache.beam.runners.fnexecution.data/GrpcDataServiceTest/testMessageReceivedBySingleClientWhenThereAreMultipleClients/history/
  2. Go to failed job -> test result -> navigate to failed test -> history


Replace relevant fields in URL. There might be issues getting there for succeeded tests. 

How to delete jobs that are no longer needed

To delete jobs that are no longer needed in Jenkins: 

  1. Remove the `.groovy` file associated with the job. For more information, see 

    Removes Python 2 and Python 3.5 Postcommit jobs. #12891.

  2. Delete the job history from Jenkins.
    1. Open https://ci-beam.apache.org/ and log in with your Apache Credentials.
    2. Select the job you want to remove.
    3. Click Delete project in the left side panel.


Jenkins Infrastructure Setup

16 GCE instances support Jenkins' continuous integrating jobs. The instance group 'apache-beam-jenkins-jnlp-group' manages the compute instances in the Google Cloud project 'apache-beam-testing'. Each instance has 16 CPUs and 104GB of memory.

Instance Name (Jenkins agent names are same)Jenkins Label
apache-beam-jenkins-1beam
apache-beam-jenkins-2beam
apache-beam-jenkins-3beam
apache-beam-jenkins-4beam
apache-beam-jenkins-5beam
apache-beam-jenkins-6beam
apache-beam-jenkins-7beam
apache-beam-jenkins-8beam
apache-beam-jenkins-9beam

apache-beam-jenkins-10

beam
apache-beam-jenkins-11beam
apache-beam-jenkins-12beam
apache-beam-jenkins-13beam
apache-beam-jenkins-14beam
apache-beam-jenkins-15beam
apache-beam-jenkins-16beam-perf

Current installations

  • apache-maven:3.5.4
  • Docker:18.09.4
  • Go:1.12
  • kubectl
  • open-jdk-11
  • open-jdk-1.8
  • Pip:8.1.1
  • Python:2.7.12
  • Python:3.5.2
  • Python:3.6.8
  • Python:3.7.3
  • Python:3.8.3
  • Virtualenv:16.4.3

How to install and upgrade software on Jenkins workers

All software updates must be performed and verified in a temporary experimental compute instance. To install or upgrade tools for Jenkins instances:

  1. Create a compute instance with the most recently created boot image. 

    gcloud compute instances create [INSTANCE_NAMES ...] --project=apache-beam-testing --zone us-central1-b --image-family=jenkins-slave-boot-image --machine-type=n1-highmem-16

    You can create the instance through VM Instances in Cloud Console or by running the gcloud command.

  2. Connect using ssh to the instance via Cloud Console or gcloud command.

  3. Install or upgrade the required tools:

    gcloud compute --project "apache-beam-testing" ssh --zone "us-central1-b" "apache-beam-jenkins-1"
  4. Verify the tool by running corresponding Beam tests.

  5. Verify backwards compatibility by running Beam PostCommits against all SDKs.

    • Clone Beam repository from Git:

      git clone https://github.com/apache/beam.git && cd beam
    • Run your tests by Gradle wrapper. For example: 

      ./gradlew :PythonPostCommit --contine --<args>
    • Remove beam repository after verification.

  6. Create a new image from the experiment instances and named it as jenkins-slave-boot-image-YYYYMMDD. Put the image in the family of 'jenkins-slave-boot-image'.

  7. For each instance:

    1. Stop the VM, then click edit.

    2. Delete the existing boot disk by clicking the 'X'.

    3. Create a new boot disk by using the newly generated image.

    4. Name disk as 'apache-beam-jenkins-[1,16]-YYYYMMDD'.

    5. Make sure the boot disk 'Model' is 'Boot, read/write'.

  8. Go to the Pantheon Disks and remove all deprecated Jenkins slave disks.

  9. Ask a PMC member to reboot the Jenkins executors. (For PMC, you'll need to ssh to the VMs and run the command listed in the Beam SVN repo, https://svn.apache.org/repos/private/pmc/beam/accounts/JenkinsWorkersSecrets.txt.)




  • No labels