ASF Buildbot Configuration update workflow

• Gavin McDonald are the svn bits still accurate for buildbot 3.2 ? - dfoulks

Introduction

This page describes the workflow that Buildbot goes through whenever anyone makes configuration changes.

The simplistic version (50k')

1. Commit to svn (at svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/[master.cfg | projects/*.conf])
2. cron job on aegis runs every 5 minutes and check to see if any changes to svn . (i.e. 'svn st -u | grep \M')
3. if true for step 2; then do an svn up to pull in those changes onto the buildmaster.
4. Become the buildmaster user: sudo -Hsu buildmaster
5. use the built in buildbot checkconfig tool to check the updated config for syntax and coding errors.
6. if step 5 passes ok then send a SIGHUP via the buildbot reconfig command to reload the configuration files.
7. if step 5 fails then we don't perform a reconfig but rather fail and send an email to the infrastructure-cvs mailing list.

And thats it. Simple. Some notes about that you should be aware of:-

The Good:

• Self service, no need to wait for infra to apply a configuration change to your Buildbot config.
• After 5 minutes if all is well your config is up and running.
• Bad configs (syntax or otherwise) are not applied to a running config and so Buildbot keeps running.

The Bad:

• Bad configs, although not applied, are still pulled in; and so effectively block ALL other good changes from being applied!
• The item above is so bad it's worth reading it again !!!

Possible changes to the workflow:

The above bad scenario bites us again and again so Infra is looking to change the workflow so that bad configs do not affect
other projects.

Ideas include:

1. A staging area for the buildbot checkconfig|reconfig to do its thing
2. in conjunction with 1. utilise the svn pre-commit hook and only actually commit on a passing config, reject otherwise.
3. similar to 2. but we put the configuration in git.

(todo: add more stuff)...