Overview

We've taken Continuum, ActiveMQ, and a whole lot of long nights and days and created a system that can push continuum builds onto a jms queue (activemq) where are then picked up by GBuild Agents (slimmed and extended version of continuum) running in the gbuild network. The results are then pushed onto a jms topic where and aggregted by another agent in the gbuild network. Putting more work on the queue is as easy as dropping a properties file in a directory.

A really cool aspect of this is that we can throw more machines at it in the middle of a run. It's just jms queue/consumer underneath so new consumers can be added at anytime! The whole thing is very extendible and not specific to tck testing, so we could work in performance testing and whip up results agents to analyze the performance data. Sky is the limit!

We tried to reuse Continuum as as much of this code as possible is destined to them. The idea is that we only should need to maintain a few extensions for the things that are specific to us or they don't want.

Basic Parts

Executing Agent (runs builds)
Producing Agent (creates builds)
Results Agent (aggregates build results)

Each agent consumes or produces on a queue or topic (or both). You can have as many of each as you need or what. They can run in the same process or separate processes. All messages are Maps that are serialized to ObjectMessages onto either the BUILD.TASKS queue or the BUILD.RESULTS topic.

Excecuting Agent

Runs continuums various continuum componentry and "update-from-scm" and "execute-builder" actions. When this is done, the code has been updated and built. Has extendible pre and post checking of incoming builds and outgoing build results. Specifically there is a BuildAgentExtension (when spelling is fixed) that has a pre(Map incoming) method and a post(Map incoming, Map outgoing) method.

Implemented Extensions so far:

  • FileIncludeExtension: Checks the build request for a configurable header format. Any headers matching that format are assumed to point to files that should be read in and included as text in the response. The file must be one under the project's work directory. - HeaderIncludeExtension: Any headers that match the configured pattern are copied as-is from the build request to the response message.

Producing Agent

Scans a directory for properties files that define builds to add to the build queue. This part could use some work. The implementation that uses properties files is fine for shoving lots of work onto a queue in a really easy way, but this is where we also need some more integration with continuum as a frontend. An implementation of A Producing Agent could be an alternate implementation of the Continuum build queue that just turns the build tasks into items that go onto the jms build queue. Once you do something like that, you need to consider how things work in Continuum in regards to it's single threaded nature that seems to be assumed in places of the code.

Results Agent

Very bare bones agent that consumes from teh BUILD.RESULTS topic and invokes BuildResultsExtensions (when spelling is fixed). Performs no other work aside from invoking the extensions. We can really go nuts with extensions here. No limit to the ones we can creat to log, track, compile, report, analyze or otherwise process the results data in various ways.

The only BuildResultsExtension so far is one that scans for entries in the results map matching a regex ("^include.*" is the standard). It takes the value of that entry and writes it into a file. The name of the file can be created using entries in the map as well. So for example:
{project.id}/{build.name}-{date}.properties

Some things we could add or do

  • System status topic and agent
  • A page to display system status (the status of machines online)
  • Notification topic and agent that delegates to the notification manager
  • Something to browse the queue and displays the contents
  • A results extension to record how long each build took (data already available)
  • Something to use the average time for a build definition and the work in the queue and estimates the total hours of work to be done the divides it by the number of machines online and estimates a completion time.
  • Make client connection a component so it can easily be shared between agents
  • Security: maybe ssh-keys for all agents. use SignedObject and SealedObject for message integrity and host verification.
  • Sort properties files when creating build definitions.
  • Have DirectoryMonitor file updated events add definitions to the queue. Allows for builds to be in tasks dir and us to just exec 'touch foo.properties' to queue up the builds inside.
  • Add gbuild agent version to message headers
  • No labels