Airavata Deployment Guide

How to start the Airavata System Components

In Airavata 0.10 distribution, you will find three main distributions as

apache-airavata-server - airavata server distribution
apache-airavata-xbaya - airavata xbaya distribution
apache-airavata-client - necessary dependencies if someone needs to use airavata-client apis to create his own application

We define <AIRAVATA_SERVER_HOME> as the location where you extract apache airavata server binary and <AIRAVATA_XBAYA_HOME> as the location where you extract airavata xbaya distribution. To start Airavata server users have to use the scripts provided in <AIRAVATA_SERVER_HOME>/bin. If you want to start XBaya, you have to use scripts provided in <AIRAVATA_XBAYA_HOME>/bin. If you are running all the components with default setup, you have to start the components in the following order:

<AIRAVATA_SERVER_HOME>/bin/airavata-server.sh
- This starts the WS-Messenger component and the GFac service in the Apache Axis2 Container.
- This points to the default derby database server, which runs on port "1527"
- If you are starting with a different registry URL, you have to make the corresponding changes in the <AIRAVATA_HOME>/bin/airavata-server.properties file.
<AIRAVATA_XBAYA_HOME>/bin/xbaya-gui.sh
- Now since the GFac Service is running with a registry, XBaya can be started to run your applications.
- Note that if you want XBaya to run with normal Web services and not with the GFac Service, you do not have to start the Airvata Service. In this case, you can simply use Xbaya alone. However, if you require provenance, you have to start the registry.

How to start the Airavata System Components with MySQL

By default, Airavata is running using Apache Derby as the backend database. You can use MySQL database instead of default derby database if you wish.

Create a new user called $user with password $password in running MySQL instance
Create a mysql database in your server with required name for user airavata (Assume name is:persistent_data)
Copy mysql driver jar (mysql-connector-java-5.1.6.jar) to <AIRAVATA_SERVER_HOME>/lib
Edit jdbc connection URLs at airavata-server.properties file that resides in <AIRAVATA_SERVER_HOME>/bin. You can achieve this by just uncommenting out parameters under "#configuration for mysql" section and commented-out the default derby configurations.
registry.jdbc.url=jdbc:mysql://localhost:3306/persistent_data
registry.jdbc.driver=com.mysql.jdbc.Driver
Edit jdbc connection URLs for "basicAccessAuthenticator" in authenticators.xml file that resides in <AIRAVATA_SERVER_HOME>/bin. Change "userName" and "password" parameters to appropriate database configuration values. A template MySQL configurations for "basicAccessAuthenticator" can be found in authenticators.xml under <!- For MySQL -> comment.

<authenticator name="basicAccessAuthenticator" class="org.apache.airavata.services.registry.rest.security.basic.BasicAccessAuthenticator"
   enabled="true" priority="8" userstore="org.apache.airavata.security.userstore.JDBCUserStore">
   <specificConfigurations >
   <!-For MySQL->
       <database>
           <jdbcUrl>jdbc:mysql://localhost:3306/persistent_data</jdbcUrl>
           <userName>$user</userName>
           <password>$password</password>
           <databaseDriver>com.mysql.jdbc.Driver</databaseDriver>
           <userTableName>Users</userTableName>
           <passwordHashMethod>SHA</passwordHashMethod>
           <userNameColumnName>user_name</userNameColumnName>
           <passwordColumnName>password</passwordColumnName>
       </database>
   </specificConfigurations>
</authenticator>

6. Go to <AIRAVATA_SERVER_HOME>/bin and start the script airavata-server.sh. Now Airavata server will start with MySQL database (in this case persistent_data database).

7. When starting XBaya, it should also point to the same database that we specified when starting Airavata server.

How to Deploy Airavata on Tomcat

In the airavata 0.6 distribution, you will notice apache-airavata-server-0.6-war.zip file. This is the war file that you need to use if you want to deploy airavata in tomcat.

Extract apache-airavata-server-0.6-war.zip/ apache-airavata-server-0.6-war.tar.gz.
Inside apache-airavata-server-0.6-war folder, there will be two war files called airavata-server.war and airavata-registry.war
Copy these two files in to <TOMCAT_HOME>/webapps folder and start tomcat server
Now the airavata server will run on top of apache tomcat.
By default airavata will run using derby database. If you want to change this to MySQL, you will have to update airavata-server.properties and authenticators.xml which resides in following locations
1. <TOMCAT_HOME>/webapps/airavata-server/WEB-INF/classes/airavata-server.properties
2. <TOMCAT_HOME>/webapps/airavata-registry/WEB-INF/classes/airavata-server.properties
3. <TOMCAT_HOME>/webapps/airavata-server/WEB-INF/classes/authenticators.xml
4. <TOMCAT_HOME>/webapps/airavata-registry/WEB-INF/classes/authenticators.xml
5. Also you will need to copy mysql driver jar to <TOMCAT_HOME>/webapps/airavata-server/WEB-INF/lib/
You also need to increase default memory level in Tomcat. To do that set following environment variable (In Unix),
export CATALINA_OPTS='-Xms2048m -Xmx2048m -XX:MaxPermSize=1024m'

To deploy a Message Box on Tomcat.

Copy AIRAVATA_HOME/repository/services/messagebox-0.8-SNAPSHOT.jar to webapps/axis2/WEB-INF/services directory.
Update airavata-server.properties in <TOMCAT_HOME>/webapps/airavata-server/WEB-INF/classes/ and <TOMCAT_HOME>/webapps/airavata-registry/WEB-INF/classes/
1. You will only need to comment out the default derby configuration and uncomment-out the mysql configuration
2. Make sure the database you specified already created
Download and copy a latest database library (Apache Derby or MySql) to webapps/axis2/WEB-INF/lib directory
1. For MySql, we have tested with mysql-connector-java-5.1.25-bin.jar
Copy directory AIRAVATA_HOME/bin/database_scripts to webapps/axis2/WEB-INF/classes directory.
Copy all the jars from AIRAVATA_HOME/lib director to webapps/axis2/WEB-INF/lib directory. (You have to copy all the jars if you deploy all the services of airavata in to one tomcat otherwise you only need to copy service specific dependent jars. We hope to document that soon) Note: If you need to deploy MessageBroker in the same tomcat you can follow the same steps 1 and 2.

To deploy a GFac Axis2 Service on Tomcat.

Copy AIRAVATA_HOME/standalone-server/repository/services/gfac-axis2-interface-.jar to webapps/axis2/WEB-INF/services directory.
Create a configuration file in webapps/axis2/WEB-INF/lib/repository.properties. A sample configuration can be looked up in AIRAVATA_HOME/standalone-server/conf/repository.properties. Based on your setup configure the GSI security credentials and certificate paths in the properties file.

Note: keep the same properties file in TOMCAT_HOME/bin directory too.. This issue will be fixed in later release.

To deploy Interpretor Service on Tomcat

Copy AIRAVATA_HOME/standalone-server/repository/services/xbaya-gui-.jar to webapps/axis2/WEB-INF/services directory.
Create configuration file in webapps/axis2/WEB-INF/lib/xbaya.properties and fill up the following property names values based on your gsi security credentials. myproxy.user myproxy.password (This step will be required from 0.3 release if you are using previous release users do not have to create xbaya.properties file)

How do I use only WS-Messenger component

Currently first release of Airavata doesn't have a separate pack for WS-Messenger. But users can check out the source of Airavata and build only WS-Messenger and build will create a pack which contains only WS-Messenger components and it can be used as the WS-Messenger binary distribution. In Airavata code go to AIRAVATA_CODE/modules/ws-messenger directory and run mvn clean install, after successful build you will find the pack(Zip file) at AIRAVATA_CODE/moduels/ws-messenger/distribution/target/.

How to Deploy WS-Messenger with Mysql

Currently WS-Messenger supports Derby and Mysql databases. Users have to change AIRAVATA_SERVER_HOME/bin/airavata-server.properties to change the message Box and message Broker databases.

First create a mysql database in your server with required name (Assume name database name is:wsmg and username/password is : airavata/airavata).
Create a tables in the above database using the following scripts.
- AIRAVATA_SERVER_HOME/bin/database_scripts/mgsBox-mysql.sql
- AIRAVATA_SERVER_HOME/bin/database_scripts/msgBroker-mysql.sql
Change msgBox.properties and msgBroker.properties files with database driver name and database urls as below.
msgBox.jdbc.driver=com.mysql.jdbc.Driver
msgBox.jdbc.url=jdbc:mysql://localhost:3306/wsmg?user=airavata&password=airavata
broker.jdbc.driver=com.mysql.jdbc.Driver
broker.jdbc.url=jdbc:mysql://localhost:3306/wsmg?user=airavata&password=airavata
Change the driver class name based on your driver and copy your driver to AIRAVATA_SERVER_HOME/lib directory and start the server. You are ready to go with Mysql database.

How can I run WS-Messenger without using any database.

Yes, WS-Messenger supports in-memory messaging. If you want to change message broker or message box to be in memory you simply have to change the appropriate property in msgBroker.properties and msgBox.properties as below.

For Message Box - AIRAVATA_HOME/standalone-server/conf/msgBox.properties - msgBox.usedatabase=true

For Message Broker - AIRAVATA_HOME/standalone-server/conf/msgBroker.properties - broker.storage.type=memory

How to use XBaya Arguments

XBaya can be started using AIRAVATA_XBAYA_HOME/bin/xbaya-gui.sh in the binary distribution. During XBaya startup you can give number of arguments for quick detail you can run xbaya-gui.sh with -help.

-help

-config - Users can put their configuration in to a properties file and give that file path as the argument value. In the properties file users can use following property names. gpel-engine-url,gfac-url,msg-box-url,msg-broker-url,dsc-url.

-title - Title that you want to show in the XBaya GUI, if you set title as "My Workflow" You will see a title at the Top of the window as "Workflow - My Workflow".

-enableProvenanceSmartRun - This parameter can be used to enable smart rerun feature, which will keep track of inputs and outputs, if user provide the same inputs to the same service which was ran previously, Xbaya will not invoke the service again, it will simply use the output of the previous invocation. [true/false]

-enableProvenance - This is a parameter to let XBaya to store inputs and outputs in to Registry. [true/false]

How XBaya is going to pick Available GFac URLs

When you start the Registry at very first and then GFac service, GFac service is registering its URL with Registry, GFac does that frequently, so if you have n number of GFac nodes running everyone is registering its URL. Since Every GFac node is registering in a given time interval, if one node goes down that URL in Registry become invalid.. When XBaya runs, it picks the set of URLs which are valid in that point.. and show them in the GFac URL list in XBaya Menu.

How to enable Workflow data provenance.

When XBaya Starts users can give few startup arguments, if you provide "-enableProvenance true" then XBaya will record all the input/output values in to Registry. And users can view the input/output data from JCR Registry Browser.

How to view my data like workflows, and documents like, Host Description,Service Description and Deployment Descrition.

XBaya has a feature of showing your content in your Registry. It will show your available, Host Descriptions, Service Descriptions, Deployment Descriptions, saved workflows etc. So you have a better understanding on what you have in your Repository.

Airavata deployment properties in airavata-server.properties

###########################################################################
# Airavata Embedded-Tomcat Server Configuration
###########################################################################
# By default, tomcat runs on port 80. If the port is changed, it will be
# have to be specified with port properties. This will be picked up by
# the registry service to register the service end-points.

port=8080

# Axis2 server automatically picks up IP address from axis configuration,
# but some DHCP enables machines do not report correct ip addresses,
# in which case, the IP address can be manually specified.

#ip=192.2.33.12

#This property will enable https and stops http, during the url registration, https urls will be stored and
# http urls will not get registered, because currently airavata supports only one url for each service
#enable.https=false
#system properties used by services to register service URLs
system.gateway=default
system.user=admin
system.password=admin
airavata.server.url=http://localhost:8080/airavata/services/registry

###########################################################################
# ----------------------------------Airavata Registry Configuration ------------------------------------------
###########################################################################
#for derby [AiravataJPARegistry]
registry.jdbc.driver=org.apache.derby.jdbc.ClientDriver
registry.jdbc.url=jdbc:derby://localhost:1527/persistent_data;create=true;user=airavata;password=airavata
# MySql database configuration
#registry.jdbc.driver=com.mysql.jdbc.Driver
#registry.jdbc.url=jdbc:mysql://localhost:3306/persistent_data
registry.jdbc.user=airavata
registry.jdbc.password=airavata
start.derby.server.mode=true
validationQuery=SELECT 1 from CONFIGURATION
jpa.connection.properties=MaxActive=10,MaxIdle=5,MinIdle=2,MaxWait=60000,testWhileIdle=true,testOnBorrow=true
# Properties to setup registry service
default.registry.user=admin
default.registry.password=admin
default.registry.password.hash.method=SHA
default.registry.gateway=default
#for rest [RegistryClient]
#registry.jdbc.url=http://localhost:9080/airavata-services

#user defined registry accessor classes
#class.provenance.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry
#class.configuration.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry
#class.descriptor.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry
#class.project.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry
#class.user.workflow.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry
#class.published.workflow.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry

############################################################ ###############
# Airavata Workflow Interpreter Configurations
###########################################################################

runInThread=true
provenance=true
provenanceWriterThreadPoolSize=20
gfac.embedded=true

#
# Security Configuration used by Airavata Generic Factory Service
# to interact with Computational Resources.
#

###########################################################################
# Airavata GFac MyProxy GSI credentials to access Grid Resources.
###########################################################################

myproxy.server=myproxy.teragrid.org
myproxy.user=ogce
myproxy.pass=
myproxy.life=3600
# XSEDE Trusted certificates can be downloaded from https://software.xsede.org/security/xsede-certs.tar.gz
trusted.cert.location=/Users/lahirugunathilake/Downloads/certificates
# SSH PKI key pair or ssh password can be used SSH based authentication is used.
# if user specify both password authentication gets the higher preference

###########################################################################

################# ---------- For ssh key pair authentication ------------------- ################

###########################################################################
#public.ssh.key=/path to public key for ssh
#ssh.username=username for ssh connection
#private.ssh.key=/path to private key file for ssh
#ssh.keypass=passphrase for the private key

#######################################################################################
################# ---------- For ssh username password authentication ------------------- ################

#######################################################################################

#If your deployment is using ssh authentication with a remote resource for file transfering or job submission

#following parameters can be used, if you need to do third party file transfer between ssh resources, credential

#configuration has to be done in gfac-config.xml

#ssh.username=username for ssh connection
#ssh.password=Password for ssh connection

###########################################################################
# ------------------------Airavata Message Broker Basic Configurations -------------#####

# ------------------------This module is currently not in use ------------------------#####
###########################################################################

# Default derby database configuration
broker.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver
broker.jdbc.url=jdbc:derby:wsmg;create=true;user=airavata;password=airavata

# Note: This database will only be used if 'broker.storage.type=persistent'
# is set in advanced properties below.
# MySql database configuration
#broker.jdbc.driver=com.mysql.jdbc.Driver
#broker.jdbc.url=jdbc:mysql://localhost:3306/wsmg?user=airavata&password=airavata

###########################################################################
# Airavata Message Box Basic Configurations,
# the Advanced section provides detailed custom configuration properties.

#This module is currently not in use
###########################################################################

# Default derby database
msgBox.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver
msgBox.jdbc.url=jdbc:derby:wsmg;create=true;user=airavata;password=airavata

# MySql database configuration
#msgBox.jdbc.driver=com.mysql.jdbc.Driver
#msgBox.jdbc.url=jdbc:mysql://localhost:3306/wsmg?user=airavata&password=airavata

###########################################################################
# ------------------------ Advanced Message Broker Configurations --------------------------------------

----------------------------This module is currently not in use ---------------------------------------------
###########################################################################

#socket time out in milliseconds for sending messages. (defaults is 20000)
broker.socket.timeout=60000

broker.storage.type=persistent
#broker.storage.type=memory

#specifies wether delivery component should be started or not.
broker.start.delivery.thread=true

#fixed thread pool based delivery
#broker.delivery.method=pcrew

#dynamic thread pool based delivery
#broker.delivery.method=parallel

#single thread delivery
broker.delivery.method=serial

#number of message delivery failures before a url become black listed (default is 2)
#broker.msg.delivery.retries=2

#time period (in seconds) a url will be kept blacklisted (default is 5 seconds)
#consumer.expiration.time.gap=5

#maximum number of messages to be send to a one consumer/url at time.
#applicable if 'broker.delivery.method' is 'pcrew' . (default is 10)

#sending.batch.size=10

#size of the thread pool. only applicable if 'broker.delivery.method' is 'pcrew'. (default is 4)
#sending.thread.pool.size=4

#
# Advanced Message Box Configurations
#
msgBox.usedatabase=true
messagePreservationDays=2
messagePreservationHours=0
messagePreservationMinutes=0
messagePerservationIntervalDays=0
messagePerservationIntervalHours=1
messagePerservationIntervalMinutes=0

###########################################################################

###---------------------------REGISTRY API IMPLEMENTATION---------------------------###

###########################################################################

class.registry.accessor=org.apache.airavata.persistance.registry.jpa.impl.AiravataJPARegistry
#class.registry.accessor=org.apache.airavata.rest.client.RegistryClient

###########################################################################

###---------------------------Computational Middleware Configurations---------------------------###

###########################################################################

#enable.application.job.status.history=true
#http://localhost:8080/axis2/services/RegistryService?wsdl
registry.service.wsdl=http://localhost:${port}/${server.context-root}/services/RegistryService?wsdl

# If false, disables two phase commit when submitting jobs
TwoPhase=true

###########################################################################
###---------------------------Monitoring module Configurations---------------------------###

###########################################################################

#Can add multiple monitoring implementation with comma separated full qualified class names

monitors=org.apache.airavata.job.monitor.impl.pull.qstat.QstatMonitor,org.apache.airavata.job.monitor.impl.LocalJobMonitor
#,org.apache.airavata.job.monitor.impl.push.amqp.AMQPMonitor
#This is the amqp related configuration and this lists down the Rabbitmq host, this is an xsede specific configuration
amqp.hosts=info1.dyn.teragrid.org,info2.dyn.teragrid.org
proxy.file.path=/Users/lahirugunathilake/Downloads/x509up_u503876
connection.name=xsede_private

###########################################################################
###---------------------------Orchestrator module Configurations---------------------------###

###########################################################################
job.submitter=org.apache.airavata.orchestrator.core.impl.EmbeddedGFACJobSubmitter
job.validator=org.apache.airavata.orchestrator.core.validator.impl.SimpleAppDataValidator
submitter.interval=10000
threadpool.size=10
start.submitter=true
embedded.mode=true
enable.validation=false
orchestrator=org.apache.airavata.orchestrator.server.OrchestratorServer

###########################################################################

###---------------------------API Server module Configurations---------------------------###

###########################################################################
apiserver=org.apache.airavata.api.server.AiravataAPIServer

###########################################################################

###---------------------------Airavata Server Configurations---------------------------###

###########################################################################
servers=apiserver,orchestrator
#shutdown.trategy=NONE
shutdown.trategy=SELF_TERMINATE

Space shortcuts

Page tree

How to start the Airavata System Components

How to start the Airavata System Components with MySQL

How to Deploy Airavata on Tomcat

How do I use only WS-Messenger component

How to Deploy WS-Messenger with Mysql

How can I run WS-Messenger without using any database.

How to use XBaya Arguments

How XBaya is going to pick Available GFac URLs

How to enable Workflow data provenance.

How to view my data like workflows, and documents like, Host Description,Service Description and Deployment Descrition.

Airavata deployment properties in airavata-server.properties