Final Report

• Title

  Apache Custos Baremetal Deployment

• Background

  Although Apache Custos is currently deployed on a Kubernetes cluster — its size and complexity may not be ideal for a relatively small application like Custos. This project aims to develop an alternative deployment architecture for Custos that aims to simplify its deployment and backup process.

• Design/description of work

  I started off by shadowing Isuru’s dev work for merging all Custos Core microservices into one module and Custos Integration services into another module.

  https://github.com/apache/airavata-custos/pull/292

• The next step was to design the new deployment architecture. Apache Airavata uses ansible extensively for a variety of tasks. We decided on using ansible scripts to set up and deploy Custos. We decided on 3 servers —
• • Custos (the spring boot application runs here, MySQL DB)
    I wrote the following roles for Custos environment:
    env_setup: to install software dependencies and SSL certs on the server
    nginx: configure and start Nginx reverse proxying on the server
    database: set up and start MySQL DB used by Custos
    zookeeper: set up and start zookeeper
    kafka: set up and start zookeeper
    custos: set up and start Custos core and integration services.
  • Keycloak (PostgreSQL DB)
    Along with the env_setup and nginx roles that are common across the 3 servers, I created the Keycloak role that includes the PostgreSQL configuration along with the Keycloak application configuration. It also starts the PostgreSQL and Keycloak servers.
  • Vault (Hashicorp Consul DB)
    Just like Custos and Keycloak servers — I used the env_setup and nginx roles for the Vault server too. Additionally, I created the consul role and the vault role to configure and start hashicorp consul and hashicorp vault.

https://github.com/apache/airavata-custos/pull/290

• In the next phase of development, I wrote migration scripts for the 3 DBs. The migrate_db role is run on each of the old servers and the new servers are run with the migrate_db tag.

https://github.com/apache/airavata-custos/pull/304

• This DB migration was made optional by setting up the migrate_db tag. Here’s the latest PR:

https://github.com/apache/airavata-custos/pull/305

• Here’s the readme for the project:
  https://github.com/apache/airavata-custos/blob/baremetal/ansible/README.md

• Future work

The DB migration process for Vault needs additional work to load the data into the consul server accurately. Additionally, I am working on an academic paper comparing Apache Custos’ baremetal deployment architecture with that of the existing Kubernetes deployment cluster.

I loved the GSoC experience with the Apache Airavata group and I will cherish the bonds that I have formed with my mentors and other contributors.