Overview

Currently, there is not a good way of surfacing Geronimo's server information so that an administrator can monitor the server's status. The architecture of using MBeans is established, but not fully exploited. This enhancement will take advantage of what Geronimo currently offers and extend it so that a server can tap into a cluster of servers and extract information from specific Geronimo servers or even aggregates of Geronimo servers.

The goal is to have at least one machine be able to reach out to all Geronimo servers in order to fetch data or even alter their state. This will be especially useful in the case of someone having to monitor a large number of Geronimo servers.

How it works (proposed)

As part of the design, there are five major components.

Management Node(s): An instance of Geronimo that has sole purpose of managing and viewing information from Server Nodes, must include admin console.
Management Plugin: Will reside on all Management Nodes and will handle the connection to the Remote Control Plugin to pull relevant information and control the machine.
Connection Information DB: Resides on either a Management Node or a different database server, has Server Name, IP:Port information and JMX Authentication information for all Server Nodes. Management Nodes use this information to connect to the Server Nodes
Server Node(s): An instance of Geronimo that runs as usual, can be Little G with our MBean and anywhere up from there.
Remote Control Plugin: Will be present on all Server Nodes, handle statistics collection, control the thread which takes snapshots of the server's current state.

The proposed solution would incorporate a scalable design, using the same methods to keep track of statistics and read them locally that it would be pulling them remotely from N machines. This is accomplished by creating a centralized MBean (Remote Control Plugin) that will work to be a tie-in point for all the statistics, taking a 'snapshot' of the current state of things every X minutes and recording in a database or XML file for a predetermined amount of time before there is rollover. With this MBean serving as a relay point for this data, we can simply connect to it via JMX through the Management Plugin, or a different remote machine (i.e. a proposed 'management node') to pull the information, as well as gain control over functions such as simple start/stop/restart, or in the future could be expanded to deploying/uninstalling/redeploying across an entire cluster.

Due to the use of JMX, there will not be an active connection from the management node to the server nodes at all times, and will instead be an on-demand connection that will be used only when control or information is requested, resulting in little network overhead.

Management Plugin Architecture

The management plugin resides on the management node, and serves as the user interface for requesting, processing, and then displaying statistical information from the server node(s), as well as providing an interface for a significant degree of control over them, including

• Server group management
  • Deploy/undeploy across multiple, set attributes across multiple, start/stop/restart multiple
• Historical statistics view for 1-n servers
• Ability to custom-define statistics collection per-machine, per-group, across entire cluster
• Manipulate snapshot interval, retention period
• Enable/disable archiving, define archiving time/retention

Currently Implemented

• Simple Tomcat statistics graph generation for localhost only
  • Bytes/sec in
  • Bytes/sec out
  • Requests/sec
  • Errors/sec
  • Bytes/Request
  • Processing Time/Request
• 1 hour time frame, 1 minute interval snapshots 

Current Dependencies

• Geronimo 2.x Tomcat minimal
• Admin Console Plugin
• Pluto Plugin
• Dojo Plugin

Remote Control Plugin Architecture

The RCP will reside on the Server Node and will be in charge of preparing information for the Management Node. Since there are many MBeans that the admin could potentially wish to track, the layers of abstraction as shown in the diagram below is a must.
Each "Controller" will be on entity that will be in charge of dealing with a certain number of related MBeans. With this model, as more MBeans are added (e.g. more components are installed) the developer can write another "Controller" and have all other pieces of RCP untouched. The bottle neck of this process is still the "MasterController." This is the point of entry for the Management Plugin. With the proper information, the "MasterController" will dynamically invoke the correct "Controller." The "Controller" will then deal with the information of the MBean in the way that the Management Plugin requested.

Taking snapshots

As of now, the RCP will spawn a thread, if it does not already exist, to capture the server's essential information. The thread will store the snapshot information on the server itself in the form of an XML. In order to preserve the history and not have a ridiculously large XML file, there will be a monthly archive of the server's information stored locally. The information stored in these XML files are what is passed on to the Management Plugin for further processing.

Current Goals

1. Extract stats using the JSR-77 model
2. Provide support for Jetty stats
3. Provide interface to add/remote/update servers controlled and monitored
4. Tie start/stop of the snapshot thread with the start/stop of the Geronimo Kernel. Currently, the Management Node has to manually start the snapshot process (not practical).
5. Provide interface to control the stopping/starting of components within server.
6. Upgrade portlet to be compatible with the new Pluggable Administration Console.
7. Design a good way to enumerate all existing MBeans available on the Server Nodes for the Management Plugin.

Current Concerns

1. Security of the JMX connection information residing on the Connection Information DB

Screenshots