A brief introduction

Since its birth in 1999, OSGi has steadily been gaining popularity as the component model of choice for Java. Originally designed as a framework for home gateways and other embedded systems, it has since moved on to desktop and enterprise systems and embraced by major software vendors.

OSGi allows you to build your systems out of well defined, reusable components, which brings both exciting new benefits as well as a set of challenges. One of these is the actual management of components, deployed on different target systems.

Traditional, monolithic systems usually have fairly straightforward and linear release schedules. It is quite common for most target systems to have the same version and configuration of such a system.

Component based systems can easily be customized on a per target case, meaning there might be many different configurations out there. When components are developed over time, they might be released individually and a lot more often than these traditional systems.

Keeping track of what is installed where becomes quite a challenge if the number of components and targets grows, and this is where Apache ACE comes in.

Apache ACE is a software distribution framework. It is written as a set of OSGi components and consists of three major subsystems:

1. dependency management, which handles the complexity of managing the dependencies between component, aggregating them into groups and licenses and associating those to targets
2. deployment management, which ensures that the right components get installed onto the right targets in a robust and scalable way
3. feedback management, which collects life cycle feedback on the target and aggregates that on a central server

A typical topology consists of:

• a server, holding the components and their metadata
• multiple target systems, which all connect to the server
• a client, on which the user interacts with the server

Currently, there are two different versions of Apache ACE you can use:

1. A file based version that allows you to basically use ACE similar to the Apache Felix File Installer, only centrally on the server for any number of targets.
2. The full version with a web based UI, which allows you to centrally manage components, organize them and distribute them to targets.

In both cases, the Apache ACE software, which consists of a set of OSGi bundles, gets deployed on a server. A target can be any OSGi framework (Apache Felix, Equinox or Knopflerfish) with the Apache ACE management agent installed. This agent will connect to the server, identify itself and poll for updates.

Getting started with the file based server

Introduction

This guide shows the basic functionality of ACE: deploying bundles to a remote OSGi framework running a management agent.

Using ACE to create a remote file installer

Apache Felix FileInstall can be used to automatically 'watch' directories for new bundles and configuration files, which will be processed.

To get roughly the same functionality, but remote, we can use the simplest server target available.

1. Build Apache ACE from source
2. Start the server from the core/deploy/target/dev-server-filebased directory by launching the run.sh or run.bat file
3. Start a gateway from the core/deploy/target/dev-gateway directory by launching the run.sh or run.bat file. The target will show some exceptions relating to auditlog information; that's correct, this server does not support auditlog information.
4. Create a version directory (anywhere), should comply with: deploymentpackage version e.g. 1.0.0
5. Copy your bundles-to-be-deployed into the 1.0.0 version directory. Suggestion: use core/lib/servicebased.host-1.0.0.jar. Once deployed a UI will popup.
6. Next, find the store directory in the server folder
7. Create a <gatewayId> directory in the store-dir: in this case configuredGatewayID
8. Copy the entire version directory, 1.0.0 with the containing jar into configuredGatewayID
9. Optionally check the available version with your browser, see http://cwiki.apache.org/confluence/display/ACE/Communication+Gateway+and+Server
10. That's it! Notice the gateway has started your bundle.

To update a target, add directories with higher version numbers. In these version directories, drop any bundles you want to have provisioned. That's it! Remember that any version you create will be installed at most once: if you create a version directory, and then start adding files to it, you run the risk that your empty folder will get installed, and all additional bundles will be removed from the target. It's easiest to create the version folder somewhere else on your system, and move it to the target's directory when you're done.

Slightly advanced

• To use a 'real' remote server, you can edit the target's configuration. Edit conf/org.apache.ace.discovery.property.cfg to contain your server IP (or name). Remember that, when using multiple targets, each target should have its own name. This can be found in conf/org.apache.ace.identification.property.cfg.

How does this work?

Simply speaking, the target will poll for updated versions, and the server will create a deployment package based on the contents of the file system.

The (configurable) scheduler will, by default, check every two seconds for an update. It does so by connecting to the server it finds in conf/org.apache.ace.discovery.property.cfg, on the endpoint /deployment. It first checks whether there are new versions by looking up /deployment/<gatewayID> (you can try this all in a browser, by the way). If there is a newer version than the one already installed, it will request the newest version, stating its own version.

The server receives the requests for either a list of versions, or a deployment package for some version, and passes it on to the resident deployment provider (in this case, a FileBasedProvider). This will check the file system, and inform the StreamGenerator of the deployment package to be generated. The StreamGenerator will generate a fixpackage (see OSGi compendium, 114.3.3). This will then be returned to the target.

The target will receive the stream, and pass it on to its DeploymentAdmin, which will take care of installing the package.

Getting started with the web UI based server

Introduction

This guide shows the core functionality of ACE, organizing bundles and other artifacts into logical groups and associating those with specific targets, all done using a convenient web interface.

Using ACE and its Web UI

Note that the Web UI is still very much a work in progress. No interaction design whatsoever has been applied and the actual interface is very much in flux. It might be these instructions are outdated because of that.

The following steps can be followed to get started:

1. Checkout and build ACE.
2. Start the server with integrated Web UI. Go to core/deploy/target/dev-server-webui. Start it using the run.sh or run.bat file.
3. Launch a browser, point it to http://localhost:8080/webui/
4. Click on 'Retrieve' to get the initial version of the repository.
5. Copy some sample bundles into the 'store' folder of the server.
6. Click on the '+' in the Bundles column and select one or more bundles to import from the OBR (you have to select them one at a time for now) and hit 'Save'.
7. Create a group by hitting the '+' in the groups column.
8. Create a license by hitting the '+' in the licenses column.
9. Drag a bundle on a group to create an association.
10. Drag a group to a license to create an association.
11. Start a target. Go to core/deploy/target/dev-gateway. Start it using the run.sh or run.bat file.
12. The target should show up in the Web UI. To associate it to a license, drag that license onto the target.
13. Click on 'Store' to actually store changes on the server, which should trigger the actual deployment of the bundle to the target.

How does it work?

...

Overview of ACE components

Introduction

Apache ACE consists of a number of components, which can be deployed more-or-less independently.

Overview

ACE consists of several components, distributed over a number of nodes in the network. The image above shows the most elaborate situation, consisting of five nodes, each running their part of some functionality of ACE.

The system has the following nodes,

• Client A client system has functionality for altering the configuration of the deployment process.
• Server The server contains all necessary metadata to allow provisioning, and acts as a hub for logging information.
• Relay A relay server keeps a copy of the metadata of the server, and can provide a target with all functionality a 'regular' server can. Note that this allows for offline provisioning, and note that there can be a chain of relay servers.
• Target The target is the system to which software is to be provisioned.
• OBR The OBR contains the actual software to be provisioned.

The nodes together have the following components,

• Deployment See below.
• Logging See below.
• Identification Each target needs a unique name; the Identification components provide this functionality.
• Discovery Targets and relays need to find the next server 'upstream' from the current node.
• Configuration ACE uses its own configuration mechanism, which works by reading configuration files, and putting that information into Config Admin.
• Scheduling Some components needs to have scheduled actions to allow logging and provisioning; the Scheduling component helps with that.

Given this information, it might also be useful to compare this with the Software Architecture.

Alternative topologies

TODO create more professional looking images

The three topologies below are just examples; you can take (almost) any component, and drop it somewhere in your network, all components are intended to be distributed.

Single server

In the case of a single server, the OBR and the required server components are deployed on one system, and all target nodes connect to that one. This is the scenario that Getting Started with the File Based Server uses.

Offline

This scenario can be used in the case of a service engineer who takes his laptop to a machine to be serviced. Assuming that these devices do not have internet connectivity (for cost- or security reasons), the engineer loads up his laptop with the most recent situation from the server (this will happen automatically), and later connect it to the machine. The machine will get its new software, and upload its logs to the service laptop; the logs will later be synced back to the server.

Firewalled

In this scenario, the administration is done on the 'big bad internet', but the server on the internet only contains metadata. The actual code is located in the OBR's behind the firewall; here, we also see the distribution possiblities.

Provisioning

Software provisioning is the main task of ACE. The provisioning components are responsible for determining the set of bundles to be deployed to a given target, creating a deployment package, and feeding the deployment package to the Deployment Admin.

Components

TODO show the components of the provisioning mechanism, and where they operate. Remember to point out that the gateway contains separate bundles, but there is also an integrated management agent.

Further reading

TODO create a child page that describes the provisioning part in more detail

Logging and reporting

ACE contains a logging and reporting facility. This mechanism allows a target to send back logging information to the server, and allows the storing of arbitrary key-value pairs.

The logging mechanism is extensible with extra logging information. The basic implementation will log framework events (e.g., starting and stopping of bundles), and deployment events.

Components

• o.a.a.server.log The server's log bundle handles the access to the access logs on the server, and provides a servlet that is used for synchronizing logs between target (or relay) and server.
• o.a.a.server.log.store Stores the log on the file; this functionality is separated to allow some other storage mechanism than the file system.

• o.a.a.server.log.task This task is provided to the o.a.a.scheduler so the relay server can synchronize its logs with the 'master' server.

• o.a.a.log Contains general interfaces for the gateway's logs.
• o.a.a.log.listener Listens for framework events, and logs those to the gateway log.
• o.a.a.gateway.log Provides a Log service to which events can be logged.
• o.a.a.gateway.log.store Stores the logs on the gateway until they can be synchronized.

• o.a.a.scheduler The scheduler is used to, configurably, run a task with a given interval. This is used by both the gateway and the relay to synchronize the log.

Further reading

TODO create a child page that shows what the logging is about, and how to add your own logging to the mechanism.

Configuring the management agent

The management agent we use is highly configurable and consists of a couple of components that can be combined in different ways. This document describes those options in detail.

The bundles that are needed for the management agent are:

• org.apache.ace.identification.ifconfig implements and publishes the Identification service, which is used to identify the gateway. This bundle uses ifconfig to get to the MAC address of the network card.
• org.apache.ace.discovery.property implements the Discovery service, which uses a configured property to "discover" the URL of the provisioning server.
• org.apache.ace.deployment contains the actual deployment engine to install and update deployments.
• org.apache.ace.deployment.task contains the schedulable task that invokes the deployment engine.
• org.apache.ace.scheduler a simple scheduler that schedules tasks.
• org.apache.ace.configurator a bundle that picks up configurations from a "conf" folder and puts them in Configuration Admin. It uses plain property files that are named after the symbolic names of the bundles they configure (with .cfg extension). Factories are represented as directories (with .cfg files inside).
• org.apache.ace.gateway.log log synchronization task, which synchronizes local logs with the provisioning server.
• org.apache.ace.gateway.log.store local log store, stores all logs locally.
• org.apache.ace.log log contains some logging interfaces.
• org.apache.ace.log.listener listener for log events that need to be recorded to the audit log.
• org.osgi.compendium OSGi compendium bundle, contains interfaces.
• org.osgi.mobile OSGi mobile compendium bundle, contains interfaces.
• javax.servlet servlet API, required by the OSGi compendium.
• org.apache.felix.dependencymanager dependency manager implementation bundle.
• org.apache.felix.configadmin configuration admin service implementation, needed to configure certain aspects of the management agent.
• org.apache.felix.eventadmin event admin implementation, needed to listen to certain vital events during deployment.
• org.apache.felix.deploymentadmin deployment admin implementation from Apache Felix.
• log_all log service, any implementation will do here, this one is from Knopflerfish.
• consolelogger dumps the log to the console, using any log service implementation.

So, what can you actually configure?

• The management agent itself can be configured and uses Configuration Admin. The configurator bundle is merely a way to get data in Configuration Admin, but could be replaced if you, for example, want to hardcode this information in a different way.
• The scheduler actually makes sure the tasks (which are services) are invoked regularly. You can replace it with your own scheduler that uses a completely different algorithm to invoke these tasks.
• The identification service determines the way the management agent identifies itself to the server.
• The discovery service determines the way the management agent actually "discovers" the server.