Introduction

.asf.yaml is a branch-specific YAML configuration file that a project may create (using a text editor of your choice) and put in the root of a git repository to control various features such as

• notification schemes
• website staging
• github settings
• pelican builds

It operates on a per-branch basis, meaning you can have different settings for different branches, and only those with an active .asf.yaml file will kick off a feature. Metadata settings (repo settings, features, labels,) are not branch-dependent, and should exist in the main (default) branch.

Important notes before you start using .asf.yaml

• .asf.yaml only works with git repositories. There is no equivalent at the moment for Subversion repositories.
• Do not use the document separator -- in your .asf.yaml file, it will cause parsing to fail.
• The configuration file is specific to the branch in which it resides, (with the exception of repository metadata (features, labels, settings) and only code blocks with a whoami matching the branch name will run.
• The configuration file holds a great deal of power, as it controls a host of automated systems.
• When using a feature in .asf.yaml, make sure that you have discussed what you propose with the entire project first, and have understood what the configuration changes will do to your (and the team's) workflow and project resources.
• You can add configuration blocks to an .asf.yaml file in any order; they do not depend on each other or flow from one to the next.

Implemented Features

Notification settings for repositories

Projects can set their notification targets for commits and github issues/PRs via .asf.yaml:

notifications:
  commits:      commits@foo.apache.org
  issues:       issues@foo.apache.org
  pullrequests: dev@foo.apache.org
  jira_options: link label worklog

Setting up notification schemes via .asf.yaml can only happen in the master/trunk or default branch of a repository, and each configuration change will cause your project's private@ list to receive a notification of the change, for review purposes.

Taking a look at your old (pre-.asf.yaml) configuration

If you wish to take a look at the default (old style) configuration for a repository, please visit https://gitbox.apache.org/schemes.cgi?$repository-name-here , for instance https://gitbox.apache.org/schemes.cgi?lucene-solr

Splitting email notifications based on context

PRs and issues can be divided into sub categories, if a project wishes to split up the open/close emails and the comments/code review parts.
For instance, if a project wants new PRs to send an email to dev@foo, but want any comments on that PR to go to issues@foo, they can employ the following configuration:

notifications:
  commits:              commits@foo.apache.org
  # Send all issue emails (new, closed, comments) to issues@
  issues:               issues@foo.apache.org
  # Send new/closed PR notifications to dev@
  pullrequests_status:  dev@foo.apache.org
  # Send individual PR comments/reviews to issues@
  pullrequests_comment: issues@foo.apache.org

Likewise, issues can be split into issues_status and issues_comment for sending issue emails to the appropriate targets.

The hierarchy for determining the right mail target for an action is:

1. If a specific status or comment target is specified, use that
2. otherwise, if a global issue/pullrequest target exists, use that
3. otherwise, fall back to the originally configured targets when the repository was set up
4. finally, fall back to dev@project for issues/PRs and commits@ for commits

Jira notification options

You can use the file to enable Jira notifications that will fire when an issue or a pull request has a ticket in its title, such as "[TICKET-1234] Improve foo bar".
Options can be a set of one or more the following:

• comment: Add the PR/issue event as a comment in the referenced Jira ticket.
• worklog: Add the event as a worklog entry instead of a comment in the referenced Jira ticket.
• label: Add a 'pull-request-available' label to referenced tickets.
• link: Add a link to the GitHub PR/issue when it's opened to the referenced Jira ticket.

Concatenate the options you want to use as a string list, like this:

notifications:
  ...
  jira_options: link label comment

Web Site Deployment Service for Git Repositories

The staging and publish features of the .asf.yaml file in a git repository manages web site deployment.

NOTE : Web site staging and publishing features are applied using the repository in which you have specified staging and publishing . Thus, only specify it within the repository that contains your web site material, or you could end up just seeing a list of source code files from your source repository.

Primer

A basic staging and publishing profile to get you started could be:

# Staging and publishing profile for yourproject-website.git:
staging:
  profile: ~
  whoami:  asf-staging


publish:
  whoami:  asf-site

This configuration enables a staging (live preview) web site at yourproject.staged.apache.org using the asf-staging branch of your repository, as well as deploying the asf-site branch of the repository to your main web site at yourproject.apache.org. Details below:

Staging a web site preview domain

To enable staging (live preview) of your project's web site,  add a staging entry to the site repository's .asf.yaml file.
As an example, take the imaginary yourproject-website.git with an .asf.yaml file containing the following entry:

staging:
  profile: beta

This would stage the current branch at https://yourproject-beta.staged.apache.org/ . You can add multiple staging profiles and thus multiple branches staged for preview. This can be helpful when doing A/B evaluations of website contents and features.

You can also omit the profile value, and stage directly at https://yourproject.staged.apache.org/ . ( "~" (tilde) means "no value" in YAML):

staging:
  profile: ~

Preventing branch-override on cloning a branch

Set a protection on multi-tenancy by specifying a whoami setting. If the setting's value does not match the current branch, no checkout/update will happen. You can have this in the .asf.yaml file on the asf-staging branch:

staging:
  profile: ~
  whoami:  asf-staging

When you clone that branch to a new branch like asf-staging-copy, the staging web site server will notice that the value of whoami does not match asf-staging-copy, and will ignore that branch until the whoami is updated to match it.

Publishing a branch to your project web site

Note: if you have previously used gitwcsub for web site publishing, your first publish action using .asf.yaml will cause any existing gitwcsub or svnwcsub subscription to stop working. This ensures that there are no race conditions or "repository fights" going on when you publish.

To publish a branch to your project web site sub-domain (yourproject.apache.org), set up a configuration block called publish in your .asf.yaml file. Enable branch-protection through the whoami parameter, like so:

publish:
  whoami: asf-site

If, for whatever reason, a project wishes to revert back to gitwcsub for publishing, remove the publish feature in your .asf.yaml file.

Specifying a non-apache.org hostname

By default, web sites will be published at $project.apache.org where $project is the sub-domain name of your project, as determined by the repository name.
Some projects have special domains, like openoffice.org, and may publish to these by specifying a hostname attribute in the publish configuration block:

publish:
  whoami:   asf-site
  hostname: www.openoffice.org

NOTE: You cannot specify your $project.apache.org hostname with this setting. It has to be inferred to prevent abuse. Also, please do not abuse this feature, thanks!

Specifying a sub-directory to publish to

To publish to a sub-directory of the web site URL, specify a subdir value. Such checkouts can be useful for sub-projects.
For instance, if httpd wished to check out a repository into httpd.apache.org/subproject, they could use the following configuration:

publish:
  whoami:    asf-site
  subdir:    subproject

CMS/pelican sub-directories for static output

The staging and deployment servers support both the content/ sub-dir and the pelican build output/ sub-dir as the root directory for the web site. Thus, the website root can be any of:

• The root of the git branch
• The content/ directory at the root of the branch
• The output/ directory at the root of the branch

GitHub settings

Repository metadata

Projects can update their GitHub metadata (repository description, homepage and labels) via .asf.yaml like this:

github:
  description: "JSONP module for Apache Foobar"
  homepage: https://foobar.apache.org/
  labels:
    - json
    - jsonp
    - foobar
    - apache

NOTE : Metadata changes will only apply if  you specify them in the .asf.yaml file in the master (or otherwise default) branch of a repository

Repository features

Projects can enable/disable GitHub repository features to provide better support.

github:
  features:
    # Enable wiki for documentation
    wiki: true
    # Enable issue management
    issues: true
    # Enable projects for project management boards
    projects: true

Merge buttons

Projects can enable/disable the "merge PR" button in the GitHub UI and configure which actions to allow by adding the following configuration (or derivatives thereof):

github:
  enabled_merge_buttons:
    # enable squash button:
    squash:  true
    # enable merge button:
    merge:   true
    # disable rebase button:
    rebase:  false

GitHub Pages

Projects can enable/update GitHub Pages settings, using GitHub for website publishing, by specifying which branch (and optional path) to publish:

github:
  ghp_branch:  master
  ghp_path:    /docs

The ghp_branch setting can ONLY be either master or gh-pages.

The ghp_path setting is ONLY valid if ghp_branch is set to master, and MUST be either null (~) or /docs.

Branch Protection

Projects can enable branch protection in their repos, including most of the sub-level protection features, such as 'require status checks to pass before merging' , 'approval by at least $n people' , 'require pull request reviews' and more.

github:
  protected_branches:
    main:
      required_status_checks:
        # strict means "Require branches to be up to date before merging".
        strict: true
        # contexts are the names of checks that must pass
        contexts:
          - gh-infra/jenkins
          - another/build-that-must-pass
 
      required_pull_request_reviews:
        dismiss_stale_reviews: true
        require_code_owner_reviews: true
        required_approving_review_count: 3
 
      required_signatures: true
    branch_b:
      required_signatures: true
  

NB (1): Enabling any of the above checks overrides what may have been set previously, so you'll need to add all the existing checks to your .asf.yaml to reproduce those previously set manually by Infra.

NB (2): If you need to remove a required check in order to push a change to .asf.yaml, you will need to create an Infra Jira ticket with a request to have the check manually removed.

If all you want to do is prevent force pushes for example, just enable protected branches to a branch name without any further configuration :-

github:
  protected_branches:
    master

More info on this can be found on the PR1678 that implemented this cool feature.

Jenkins PR Whitelisting

For projects using Jenkins for CI testing, PRs are generally only built when a committer submits one. Projects MAY choose to designate a GitHub person as a 'safe/reliable' person using the jenkins/github_whitelist feature:

jenkins:
  github_whitelist:
    - janedoe
    - githubmonkey
    - papasmurf1234

The GitHub IDs listed here would have access to start builds based on PRs in addition to the committers on the project.

Static web site content generation

Jekyll CMS

Projects can build their websites automatically using Jekyll. This solution allows the use of custom plugins. Content generated this way can be staged or pushed directly to production when it is used in conjunction with the staging or publish configuration options.

Optionally, a named output directory may be specified as outputdir. If a value is not specified for this property, it defaults to 'output'.

To set up an automatic build (as we did in the old Apache CMS system), add a jekyll section to .asf.yaml

jekyll:
  whoami: jekyll-source-branch
  target: asf-staging-jekyll       # output branches need to be asf-site OR asf-staging*
#  outputdir: outputdir            # MAY be needed, but generally can be left out

Pelican CMS

Projects can automatically build web sites using the Pelican CMS System and even have the result either staged or pushed directly to production (with the addition of a staging or publish configuration, as seen above).

To set up an automatic build (much the way the old Apache CMS system works), add a pelican section to .asf.yaml:

pelican:
  whoami: master
  target: asf-site

The above configuration generates the site using pelican and pushes only the created output to the asf-site branch. An example web site repository that uses the pelican auto-build feature is: https://github.com/apache/infrastructure-website.

Our Pelican builds support GFM (GitHub-Flavored Markdown), meaning you can edit web sites using the GitHub UI and instantly get a preview of your page before pushing it to the build/publish process.

GFM is enabled by default, but will default to standard markdown if you have PLUGINS defined in your pelicanconf.py file. To explicitly enable GFM along with other manually defined plugins, you may specify pelican-gfm as a plugin, and it will be woven into the build.

Furthermore, you can build off one branch and publish to another using the target parameter, as seen above. If you leave this parameter out, the build process pushes the generated site to the same branch it built from (in the output/ base directory).

Pelican auto-builds support using different themes via the theme argument to specify the directory that contains your theme. This is equivalent to the -t switch in pelican:

Building and publishing at the same time

You can build and publish your website at the same time by employing both the pelican and publish configurations in your .asf.yaml file:

pelican:
  whoami: master
  target: asf-site

publish:
  whoami: asf-site

The configuration snippet above would, when present in both master and asf-site branches, build the web site from the master branch, then push the result to the asf-site branch and publish that branch as your project web site.

Likewise, you can employ auto-build-and-stage:

pelican:
  whoami: master
  target: asf-site

staging:
  whoami: asf-site
  profile: ~

This would build your site from the master branch, push the result to the asf-site branch and then stage that result on your staging domain.

Upcoming features

These features have not been implemented in production yet, but are documented here for future use.

none in the pipelines right now...