Page History

{scrollbar}

ExcerptThis farming system uses plugins directly . Therefore, it and is thus decoupled from the deployment system and allows running the plugin packaging (or "deployment") and farm administration on . It enables the plugin based farming administration for different servers or machines. The current implementation uses multicast for node discovery. Farming information is stored in a database using jps. The data model allows a single administration server to manage:

• multiple farms (currently sometimes , also termed clusters)
• multiple lists of plugins per farm ( in an mXn M X N relationship)
• multiple plugins per plugin list

When a node starts up, it broadcasts a service advertisement including an optional node name (defaulting to host:jndi-port) and a farm name. When the administration server detects this as new, it instructs the new node to install all the plugin lists associated with that farm. If the plugins are already installed, this is a no-opthe system provides no further operation; otherwise the plugins are downloaded and installed from the plugin repository specified in the plugin list.

Administrative Farming administrative operations include:

• Add a plugin list to a farm
• Add a plugin to a plugin list
• Add a plugin to a plugin list and the list to a farm
• Remove a plugin list from a farm
• Remove a plugin from a plugin list

...

• a plugin from a plugin list

...

These operations are available through the GShell cluster/deploy command .

Missing features

...

and are discussed in details in the #Add and remove plugins and plugin lists section.

Architectural Considerations

Two Plugin based farming requires two separate sets of functionality are required for plugin based farms:

• A plugin farm controller that tracks the farms node, plugin lists, and plugins and issues instructions to the nodes.

...

• A geronimo plugin repository accessible to all farm nodes. This can be a geronimo server running the geronimo-as-maven servlet, the local file system maven repository, a file based repository served by httpd, or a maven repository manager such as nexus. If a geronimo server is used, it can be the same server as the plugin farm controller or a separate server.

Possible plugin-oriented production workflow using maven

This scenario assumes use of maven as a build tool and separate development, quality assurance (QA), and production groups. It needs two maven repositories, for development and production artifacts. These can be simple file/httpd based repositories or something like nexus. Code transfer between groups is in the form of geronimo plugins packaged by the geronimo car-maven-plugin.

1. Development group deploys plugins into the development repository.
2. QA group tests these candidate plugins and upon acceptance deploys them to the production repository.
3. Production group distributes plugins to appropriate farms.

The farming information is stored in the PluginFarmDatatabase database on the farm controller.

Supporting plugins and assemblies

• oorg.aapache.ggeronimo.configs/plugin-farm-member//car (plugins/clustering/plugin-farm-member) plugin turns a server into a plugin farm node. You can configure the multicast address and port, cluster name, and node name in <WASCE_HOME>/var/config/config-substitutions.properties. It is probably most convenient to not configure node name and rely on the default host:jndi-port.
• oorg.aapache.ggeronimo.assemblies/geronimo-plugin-farm-node//car (plugins/clustering/geronimo-plugin-farm-node) extends the framework assembly with the plugin-farm-member plugin. This is all you need to start up a farm node.
• oorg.aapache.ggeronimo.configs/plugin-farm//car (plugins/clustering/plugin-farm) is the administration plugin for plugin-based farms. Currently it 's is only accessible through the gshell Gshell cluster/deploy command.
• oorg.aapache.ggeronimo.configs/plugin-farm-datasource//car (plugins/clustering/plugin-farm-datasource) is a derby datasource used by the plugin farm. It is intended to be replaced in production with a remote datasource. As always, the replacement plugin should use an <artifact-alias> element to redirect dependencies on plugin-farm-datasource to itself.

In my sandbox until we find a better location or solution, as examples

• o.a.g.configs/controller-as-repo//car (sandbox/djencks/assemblies/controller-as-repo) is a non-classloader plugin that sets the default plugin repository to be the admin server. This currently only works if both the admin server and all the farm nodes are running on the same machine – obviously only a toy example scenario.
• o.a.g.assemblies/geronimo-tomcat-farm-controller//tar.gz|zip (sandbox/assemblies/geronimo-tomcat-farm-controller) is a server assembly that extends the tomcat javaee5 server with the plugin farm controller

GShell cluster/deploy command syntax

The GShell cluster/deploy command takes one argument, the action, which must be add or remove, and options for the farm, plugin list, and plugin.

-f (--farm) farm name
-l (--pluginlist) plugin list name
-a (--pluginartifact) plugin artifact name

In addition there are some inherited options for connecting to the server:

-s (--hostname, --server) server name or IP address
-p (--port) server port
-u (--user) user name
-w (--password) password
--secure (use secure communication to server)

Example walkthrough

Make sure that the version of the plugins comply with your server.

Setup steps

The following steps demonstrates the plugin based farming This example demonstrates the plugin farm on a single server, using nodes that all share the same geronimo installation, and using the . The admin server acts as the farm controller, application deployment server, and plugin repository. We'll install one of the sample applications on the admin server and deploy it to the farm. I've only tested this with all components built locally on one machine, but some such as the geronimo-plugin-farm-node should be available from the apache snapshot repository.

Farm node setup

1. Obtain the geronimo-plugin-farm-node assembly, such as by building plugins/clustering.
2. Unpack it and copy the var directory to multiple servers, say server1 through server5. On a unix like system this can be done with a script like

   Code Block
   #!/bin/bash tar xzf geronimo-plugin-farm-node-2.2-SNAPSHOT-bin.tar.gz cd geronimo-plugin-farm-node-2.2-SNAPSHOT for s in {1..5} do sn="server${s}" mkdir ${sn} cp -r var ${sn}/ #uncomment to actually start the servers in the background # ./bin/gsh geronimo/start-server -G server.name=${sn} -G config.substitution.PortOffset=${s}0 >${sn}.log & done cd ..

   Here's a fancier script that can both start and stop servers but still illustrates my bash illiteracy:

   Code Block

   #!/bin/bash BASE=geronimo-plugin-farm-node-2.2-SNAPSHOT if [ "$1" = "stop" ] ; then ( cd $BASE for server in server*; do pid=$(ps ax | grep "gshell-bootstrap.jar geronimo/start-server -s $server" | grep -v 'grep' | cut -c 1-6) if [ $pid ] ; then echo "Killing gshell for $server at pid $pid" kill -9 $pid fi pid=$(ps ax | grep "javaagent.*$server" | grep -v 'grep' | cut -c 1-6) if [ $pid ] ; then echo "Killing $server at pid $pid" kill -9 $pid fi done ) fi rm -rf $BASE tar xzf $BASE-bin.tar.gz ( cd $BASE for s in {1..5} do sn="server${s}" mkdir ${sn} cp -r var ${sn}/ if [ "$1" = "start" ] ; then ./bin/gsh geronimo/start-server -s ${sn} -G config.substitution.PortOffset=${s}0 >${sn}.log & fi done )

3. Start the servers with command lines like this:

   Code Block
   ./bin/gsh geronimo/start-server -G server.name=server1 -G config.substitution.PortOffset=10

   substituting appropriately for server1 and 10

Farm controller setup

1. Check out bits of my sandbox

   Code Block
   svn co https://svn.apache.org/repos/asf/geronimo/sandbox/djencks/assemblies

2. Build the plugin and assembly

   Code Block
   cd assemblies/controller-as-repo mvn clean install cd ../geronimo-tomcat-farm-controller mvn clean install cd target tar xzf geronimo-tomcat-farm-controller-2.2-SNAPSHOT-bin.tar.gz ./geronimo-tomcat-farm-controller-2.2-SNAPSHOT/bin/gsh geronimo/start-server

Deploying a sample to the farm

...

svn co https://svn.apache.org/repos/asf/geronimo/samples/trunk
cd trunk
mvn clean install

...

The setup steps for plugin based farming includes:

1. #Set up farm controller
2. #Set up farm node
3. #Deploy a sample plugin to the farm
4. #Add and remove plugins and plugin lists

Set up farm controller

1. Start the controller server and install the following plugins in the server repository. For example, for a version 2.2 server, the plugins are stored in the http://geronimo/apache/org/plugins/geronimo-2.2^{Image Added} repository:

• org.apache.geronimo.configs/plugin-farm//car
• org.apache.geronimo.configs/plugin-farm-datasource//car
• org.apache.geronimo.configs/plugin-farm-member//car
  2. Open the config-substitutions.properties file and set the following attributes:

  Panel
  borderStyle solid title Excerpt from config-substitutions.properties
  ClusterName=farm name NodeName=name ServerHostname=Node_IP DefaultPluginRepository=share install plugin repository

  where
• ClusterName: defines the name of the farm that the server works in.
• NodeName: is optional and defaults to host:jndi-port. For example, if the host IP for this server is 9.186.10.157 and the jndi-port is 1099, the default Nodename=9.186.10.157:1099.
• ServerHostname:sets the node IP of this server.
• DefaultPluginRepository: defines the plugin repository accessible to all farm nodes. For example, you can use the maven repository as the share install plugin repository by defining DefaultPluginRepository=http\://9.186.10.157\:8080/plugin/maven-repo.
  3. Restart the server.

Set up farm node

Use the following steps to set up every node server in the farm:
1. Start the node server and install the org.apache.geronimo.configs/plugin-farm-member//car plugin in the server repository. For example, for a version 2.2 server, the plugin is stored in the http://geronimo/apache/org/plugins/geronimo-2.2^{Image Added} repository.
2. Open the config-substitutions.properties file and set the following attributes:

3. Add the share install plugin repository to the plugin-repositories.properties file under <WASCE_HOME>/var/config/ for each farm node, including access username and password. For example:

where

• the password manger in this example is in plain text. However, it is recommended to encrypt your password with the deploy encrypt command.

4. Restart the server.

Deploy a sample plugin to the farm

Install a plugin to the farm controller with command. For example, use the Gshell command as follows to install a HelloWorld.car plugin:

deploy/install-plugin D:/HelloWorld.car -s 9.186.10.157

• -s: is a command option that indicates the hostname of the server. For this example, the hostname is 9.186.10.157.

Add and remove plugins and plugin lists

Use the cluster/deploy Gshell command to verify the farm that you set up. For example, add and remove plugins and plugin lists with the following commands:
Use this command to add a plugin list p1 to farm cluster1:

cluster/deploy add -c cluster1 -l p1 -s 9.186.10.157 

Use this command to add the org.apache.geronimo.samples/HelloWorld/1.0/car plugin to plugin list p1:

cluster/deploy add -l p1 -a org.apache.geronimo.samples/HelloWorld/1.0/car -s 9.186.10.157

Use this command to add the org.apache.geronimo.samples/HelloWorld/1.0/car plugin to plugin list p1 and add p1 to farm cluster1:

cluster/deploy add -c cluster1 -l p1

...

 -a org.apache.geronimo.samples/

...

HelloWorld/

...

1.

...

0/car 

...

-s 9.186.10.157

Use this command to remove a plugin list p1 from farm cluster1:

In the server logs for the admin server and the farm nodes you should see that the application is installed along with all dependencies. You should be able to see the bank sample on the nodes at for example, http://localhost:8120/bank/^{Image Removed}

Removing plugins and plugin lists

In the gsh console above try

cluster/deploy remove -lc pluginList1cluster1 -a l p1

Use this command to remove the org.apache.geronimo.samples/

...

HelloWorld/

...

1.0/car plugin from plugin list p1:

2-SNAPSHOT/car  
# add back the plugin...
cluster/deploy addremove -f cluster1 -l pluginList1p1 -a org.apache.geronimo.samples/bank-tomcatHelloWorld/21.2-SNAPSHOT0/car 
# remove it again
cluster/deploy remove -f cluster1 -l pluginList1   

Configuration of plugin repositories.

The example above is set up, for convenience and portability, to use the farm controller server as the plugin repository, and to use localhost as the server name. This means no custom configuration is needed to run everything on one machine but is obviously incomplete for any serious use.

The accessible plugin repositories are configured in each node in the var/config/plugin-repositories.properties file. This needs to include the appropriate plugin repository with any needed credentials. One way to do this is to write a non-classloader plugin to install a new var/config/plugin-repositories.properties to replace the default. You'd assemble a plugin node server including this additional plugin and distribute that to the nodes.

In addition, the controller also needs to have this plugin repository configured as the default in var/config/config-substitutions.properties in this entry:

DefaultPluginRepository=http\://localhost\:8080/plugin/maven-repo

Future work

-s 9.186.10.157

Access the Plugin portlet form the Administration console to verify the installation result for all farm members.

Future work

The following features could be added in the future:

• Provide Administration console support for the administrative operations.
• Enhance enhancing the monitoring console to find nodes using multicast discovery.
• Provide better support for setting up and starting multiple servers on one geronimo installation.
• Enhance enhancing the car-maven-plugin to assemble servers with multiple instance support in the assembly.