An integral part of the continuous integration process is a step in which the source code and resources of a build are validated via automated testing. This validation is typically executed using tools which can run unit, integration, and/or functional tests. One of the challenges of integrating automated testing into a continuous integration build, for a Flex/AIR application, is the need to execute the Flash Player or AIR environment to produce test results. Both the Flash Player and AIR environment currently require windowing environments to execute Flex/AIR applications correctly; this includes applications which are test runners provided by most of the major unit testing frameworks for Flex/AIR/ActionScript. FlexUnit 4 provides a mechanism, via Ant, by which unit and integration test executions can be automated to work with various continuous integration solutions.

It is the goal of the page to be a living document with users contributing their solutions for attempting to get FlexUnit running under various continuous integration solutions.

Platforms

Below are some suggestions on integrating FlexUnit to execute headlessly in your build process based on platform. Please note, that if attempting to execute a continuous integration build, headlessly, on MacOSX, it may be possible to reproduce the scenario described under the Linux section; at this time however, this scenario has not been validated.

Windows

• Windows has the ability to execute a build "headlessly" in a CI server, such as Hudson, if the solution is integrated as a Windows service executing with as the LocalSystem user. The LocalSystem user appears to have the ability to launch processes requiring windowing such as the Flash Player and AIR. Please note, that if you choose to execute your CI server as LocalSystem, there are security concerns with this implementation however (http://www.google.com/search?q=security+problem+localsystem see Google).

Linux

• Most installations of CI solutions are made on Linux machines not running a windowing environment, such as X-Windows. Because of the requirement to execute the Flash Player and AIR in a windowed environment, it is not possible to execute a CI build on these types of installations unless the dependencies for a windowing environment have been installed. In addition to this depenendency, an intermediate process is required to redirect windowing output to an emulated display. The most common implementation of this solution is the use of X-Windows as a windowing environment and Xvnc, or xvfb, to do the redirection.
• If you're using Hudson as a your CI solution, the Xvnc plugin is recommended to act as an implementation of the description above. An implementation of vncserver must be installed along with X-Windows.
• If you're using a CI solution that does not have a similar plugin (as described above) and you can utilize Ant from your build process, the [Ant Task] may be a solution to consider. The Ant task has a port of Hudson's Xvnc plugin within it, relying on the same dependencies as the plugin and operating in a similar fashion. The difference, however, is that the Xvnc support baked into the FlexUnit Ant task will only wrap the execution of the FlexUnit test runner, not other processes from your build script.
• If you're using Maven and FlexMojos, then you should use the built in support for FlexUnit via the unit-testing-support dependency. Although FlexUnit does not handle support for the mechanism by which tests are executed in the custom FlexMojos test runner, its use is advised over integrating the FlexUnit Ant task via the ant-run-plugin to better adhere to the "Maven way". FlexMojos utilizing xvfb for its windowing redirection, which has different dependencies from the Xvnc support found in the FlexUnit Ant task and the Hudson Xvnc plugin. For more information on running FlexUnit in FlexMojos, see their wiki article.
• When running Xvnc around a build, if you notice that you receive timeout errors from the Flash Player during test runs, check out the ~/.vnc/xstartup file for the user running your CI server. By default, at the end of this file a call is made to associate a window manager with any VNC server session. By default, the window manager used is twm. Unfortunately, twm often has a default configuration requiring a user interaction in the form of a click to create a screen in the running VNC session. Since the CI process is headless, twm will cause your test run to fail since it's waiting for a mouse click.
  
  To verify this may be your problem, you can try a couple things:
  • Attach to the VNC session created by your build process with a VNC client; when you see a call made to open the Flash Player in the build output, move the mouse around on the VNC client screen. If you see a 3x3 grid appear which when clicked causes the Flash Player to appear, then you're using most likely using twm.
  • Install whereis and run whereis twm, if you haven't edited the xstartup script and its found, then Xvnc is trying to use twm.
  • You have three options if you discover twm is interfering with your test runs.
  • You can remove the following line from your ~/.vnc/xstartup script resulting in no window management when you connect with a VNC client to watch your build execute (basically no chrome):

    twm &

  • Disable the click restriction by configuring twm to place the window automatically. The easiest way to do so is to add this line to your ~/.twmrc file:

    RandomPlacement

    If that file doesn't exist already, you can use the system-wide twm configuration as a template. You should find it at /etc/X11/twm/system.twmrc or /usr/share/X11/twm/system.twmrc or similar. Copy that file to ~/.twmrc and add RandomPlacement. Alternatively, you can completely ignore the system's default twm configuration by starting with an empty ~/.twmrc file.
  • You could use a different window manager without the click restriction. metacity typically comes pre-installed and is pretty lightweight. For a list of window managers that work with XWindows, check out http://xwinman.org/. Assuming metacity is located on the path of the user running your CI server you could replace the above line with:

    metacity &

  • It's my experience you'll run into this problem on CentOS since twm is installed by default. You may not however run into this issue on Ubuntu, since twm is not installed by default hence making it seem like no window manager is being used by VNC. I hope someone can benefit from this work as it's eaten up tons of hours for me.
    
    
• If you're unable to launch the stand-alone Flash Player debugger in a VNC session, try to validate the Flash Player dependencies when working with Linux (http://kb2.adobe.com/cps/153/tn_15380.html). Additionally, if running a 64-bit OS, validate any 64-bit dependencies for the player (http://kb2.adobe.com/cps/000/6b3af6c9.html#main_installLinux).
• If you're still having socket timeout problems, make sure you are using an updated version of Flash player (I had to use version 11). This may also cause a segmentation fault if you're using xvfb-run
• If you're unable to you any of the above mechanisms with your CI solution, you may want to consider calling out to Xvnc or xvfb directly in your build script. If you are considering xvfb, you may also want to look into xvfb-run, which simplifies the process of using xvfb considerly.

Examples

To see samples of build scripts written using Ant and Maven, please check out the CI Sample Project Information.