Tutorial 05 - PHP Reference Gateway for Airavata - Gateway Admin Guide

Introduction to PHP Reference Gateway

Important

Setting up Gateway Preferences, registering and modifying Compute Resources, Application Modules, Application Interfaces, and Application Deployments are tasks for gateway administrators. To follow below steps your user account requires administrator privileges. After you create the user account send a request to Airavata Mailing List  to obtain admin access. Test-drive gateway link: https://testdrive.airavata.org

Tutorial I - Experiment Statistics

View Experiment Summary

1. Gateway admin can search for any experiment in the gateway by using the experiment ID. Here gateway displays all the information related to experiment creation as well as execution on the resource.

Experiment Statistics

1. Gateway admin can view all the experiments in the gateway for a specific date time range. Experiments within the date time range is grouped by the experiment status.
2. Admin can pick and choose experiments based on the status and investigate. Admin can pick a specific experiment and view details.
3. Here the admin has the option of canceling an ongoing experiment OR launching an experiment which is in CREATED state.
4. In order to test a failed experiment admin can clone and launch the particular experiment as one of his own. This is useful when required to test prior to responding to a user query.

Tutorial II - Manage User Roles and Their Users

Create and Manage User Roles

1. In order to create the user roles, navigate to Admin Dashboard → Roles.
2. Gateway Admin can create user roles and assign existing users to the roles. 
   NOTE: Currently there are pre defined roles which will be used to provide different levels of access. Creating a role in PGA is only the name of one of those pre defined roles. Admin will NOT be able to create a total new role; or provide new set of access to users.
3.  Pre-defined user roles; 
   1. Admin - Gateway admin is the only user with this role. Has complete access admin dashboard features; create, modify and delete compute resources, user roles, Applications ,etc...
   2. Admin-read-only - This is more use to the users who are part of the gateway admin group. This allows view-only access to all admin dashboard features.
   3. Gateway-user - All the active end-users are in this category. 
   4. Pending-user - This is the role assigned to users when they create the account and prior to give access to create and launch experiments.

Manage Users within a User Role

1. Gateway admin can assign and remove users from user roles.
2. When the role changes users will be notified via email with message 'Your privileges have changed!'
3. For changes to take place users always has to log out and log in again.
   
   

Tutorial III - Register and Manage Compute Resources

Register Compute Resource

1. To register a new compute resource navigate to ‘Admin Dashboard → Compute Resource → Register’.
   Mandatory fields are tagged with ‘*’
2. In the page; enter
   1. Hostname (Required to communicate with the resource)
   2. Host Aliases
   3. IP addresses
   4. Resource Description
   Save above information. Upon saving another field; Maximum Memory Per Node ( In MB ) will appear. Enter the memory per node and 'Save Changes'.
3. Upon saving 5 more tabs will appear;
   1. Queues: Enter job queue information in the resource
   2. File System: Enter file path locations of the compute resource
   3. Job Submission Interfaces: Protocol information required for job submissions.
   4. Data Movement Interfaces: Protocol information required for input and output data stagings and transfers.
   5. Reporting: This is to report exact gateway user information. When end user information is requested by the resource or managing body then this feature should be enabled.
      

Queue Tab

1. In Queue tab; click ‘Add Queue’ and enter;

1. Name (This is a unique and mandatory value. Once entered cannot be modified)

2. Description

3. Max Run Time

4. Max Nodes

5. Max Processors

6. Max Jobs

7. Max Memory (in MB)

2. After entering information save. ‘Reset’ is to clear any unsaved data entered.

3. After the queue is saved it will get listed under ‘Existing Queues’ and can modify by clicking the icon 'pen' or can delete by clicking icon 'trash can'.

4. When adding queues can add multiple queues by clicking ‘Add Queue’ Button.

  File System Tab

1. In this tab admin user has to enter information about important or used file locations in the resource.

2. Enter resources file path locations to folders;

1. HOME - HOME folder path of the compute resource

2. LOCAL -

3. LOCALTMP

4. SCRATCH

5. ARCHIVE

Job Submission Interface

1. This is the 3rd tab in Register Compute Resource process.
2. Click ‘Add a New Job Submission Interface’ and then select a job submission protocol from the available protocols;
   1. LOCAL
   2. SSH
   3. GLOBUS
   4. UNCORE
   5. CLOUD
3. If you selected a protocol which is yet to be available, you will see the message; “<Protocol Name> protocol not being set up right now.Please choose another option”
4. Once a protocol is selected enter below parameter for the protocol;
   1. Resource Manager - Auto-populated
   2. Resource Manager Type - Select from existing list; FORK, PBS, SLURM,LSF and UGE
   3. Push Monitoring Endpoint - User has to enter the path
   4. Job Manager Bin Path - User has to enter the path
   5. Job Manager Commands - User can enter commands for;
      1. SUBMISSION
      2. JOB MONITORING
      3. DELETION
      4. CHECK JOB
      5. SHOW QUEUE
      6. SHOW RESERVATION
      7. SHOW START
      IMPORTANT: These commands will be used in the resource to submit, monitor, delete, etc..submitted jobs.
5. Once all data is entered click ‘Add Job Submission Protocol’ to save.
6. Once added the particular protocol will not exist to add again.
7. Per resource user can enter multiple protocols and when more than one protocol exists user has to enter priority for them. Click ‘Update Priority’, it will display added protocols and the user can enter priority. These fields are numeric fields.

Data Movement Interface

1. In the tab click ‘Add a New Data Movement Interface’ and select a protocol from the available list.

2. Mandatory fields are be marked with a ‘*’

3. Enter all parameters for the selected data movement interface and click ‘Add Data Movement Protocol’ button to save the parameters.

Reporting

1. Click 'Enable Gateway Usage Reporting'.
2. Add required fields; mandatory ones are marked with '*'.

Browse Compute Resource

1. Navigate to the browse page through; Main Menu → Compute Resource → Browse.
2. Name and the ID of all existing compute resources will be displayed. The user can also search for the resource by giving the name in the search field.
3. Through this page user can Edit, View or Delete a compute resources. Icons for each action are;
   1. Edit → Click compute resource 'Name'
   2. View → Click  the icon ‘View’ 
   3. Delete → Click icon 'Trash can'
   4. Enabled → Check the box to enable the resource for gateways.
      
4. NOTE: Above actions are only possible if the user has the correct user role. These actions are controlled using user roles.
   
   

Tutorial IV - Register and Manage Storage Resources

Register Storage Resource

1. To register a new storage resource navigate to ‘Admin Dashboard → Storage Resource → Register’.
   Mandatory fields are tagged with ‘*’
2. In the page; enter
   1. Hostname (Required to communicate with the resource)
   2. Resource Description
3. Upon saving Data Movement Interfaces tab will appear.

Data Movement Interface

1. In the tab click ‘Add a New Data Movement Interface’ and select a protocol from the available list.

2. Mandatory fields are be marked with a ‘*’

3. Enter all parameters for the selected data movement interface and click ‘Add Data Movement Protocol’ button to save the parameters.

Browse Storage Resource

1. Navigate to the browse page through; Main Menu → Storage Resource → Browse.
2. Name and the ID of all existing storage resources will be displayed. The user can also search for the resource by giving the name in the search field.
3. Through this page user can Edit, View or Delete a compute resources. Icons for each action are;
   1. Edit → Click compute resource 'Name'
   2. View → Click  the icon ‘View’ 
   3. Delete → Click icon 'Trash can'
4. NOTE: Above actions are only possible if the user has the correct user role. These actions are controlled using user roles.

Tutorial V - Create & Search Application Module, Application Interface, and Application Deployment

Create Application Module

1. Navigate to Admin Dashboard --> App Catalog → Module and click ‘Create a New Application Module’ button.

2. Enter all required information. Mandatory fields are marked with ‘*’

Search Application Module

1. Navigate to Admin Dashboard --> App Catalog → Module. All available application modules will be listed in the page.
2. User can search for a specific module by entering the module name or part of the module name in the text box above the list OR select from the already available list.
3. Her user has the option of;
   1. Editing - Click on the 'Pen' icon in order to edit application module information
   2. Deleting - Click on the 'trash bin' icon in order to delete and user will be displayed with a confirmation message
   3. Viewing - Click on the 'Expandable' icon at the far righthand corner. Will expand and display available information.
      
   available application modules.
4. In an application module the unique key is the auto generated Application ID at the time of saving the application module. In PGA Application Modules name can be repeated if the gateway desires.

Create Application Interface

1. Navigate to Admin Dashboard --> App Catalog --> Interface and click on 'Create a new Application Interface' button.
2. Here user can enter;
   1. Application Interface Name (Mandatory)
   2. Application Description
   3. Application Module (User can select multiple modules per interface)
   4. App Application Input and Output (Gateway admin to enter application input and output related parameters. An application interface can have one or many inputs and outputs and also in the form of String, Integer, UR, STDERR. etc...)
3. Click 'Create'

Search Application Interface

1. Navigate to Admin Dashboard --> App Catalog --> Interface and click on 'Create a new Application Interface' button.
2. User can search for a specific interface by entering the interface name or part of the interface name in the text box above the list OR select from the already available list.
3. User has the option of;
   1. Editing - Click on the 'Pen' icon in order to edit application interface information
   2. Deleting - Click on the 'trash bin' icon in order to delete and user will be displayed with a confirmation message. User can delete the whole interface or selected output or input form the interface.
   3. Viewing - Click on the 'Expandable' icon at the far righthand corner. Will expand and display available information.
   available application interfaces.
4. In application interface the unique key is the auto generated Application Interface ID at the time of saving the application interface. In PG Application Interface name can be repeated if the gateway desires.
   
   

Create Application Deployment

1. Navigate to App Catalog --> Deployment and click on 'Create a new Application Deployment' button.
2. Here user can select;
   1. Application Module (Mandatory)
   2. Application Compute  Host (Mandatory)
   3. Application Parallelism Path (Mandatory)
3. User can enter;
   
   1. Application Executable Path (Mandatory)
   2. Add Module Load Command
   3. Library Append Path, Prepend Path
   4. Environment
   5. Pre and Post job commands
4. Click 'Create'

Search Application Deployment

1. Navigate to App Catalog --> Deployment and click on 'Create a new Application Deployment' button.
2. User can search for a specific deployment by entering the deployment name or part of the deployment name in the text box above the list OR select from the already available list.

User has the option of;

1. Editing - Click on the 'Pen' icon in order to edit application deployment information
2. Deleting - Click on the 'trash bin' icon in order to delete and user will be displayed with a confirmation message. User can delete the whole deployment or selected output or input form the deployment.
3. Viewing - Click on the 'Expandable' icon at the far righthand corner. Will expand and display available information.

available application deployments.

Tutorial VI - Setting Gateway Preferences

1. Navigate through right hand corner 'Admin Dashboard menu item and from there select 'Gateway Preferences' from the left top hand corner options.
2. Clicking on 'Gateway Preferences' will open Gateway Settings and there the gateway admin can enter gateway preferences for each resource.
3. To add a compute resource first select a resource available for the given list of values.
4. Ones selected a resource parameters required will be listed to enter values;
   1. Login username for the resource
   2. Preferred job submission protocol
   3. Preferred data movement protocol
   4. ...
   5. ...
   6. Scratch location
   7. Project number
5. Above entered values will be picked up at the time of executing experiment jobs at the remote compute resource.

Tutorial VII - Credential Store

1. This is to generate SSH token + key for SSH communication with compute resources and storage resources.

Tutorial VIII - Notices