...
Engage various stake holders to discuss the use-case, assess feasibility, and understand the process and associated timelines (~1 week)
Internalize different configuration options and learning how to construct different Gobblin flows. Write one job config for each of the tasks outlined above (~1 week)
Review the work, set-up a dark launch to test the system. Iteratively continue on this process until everything works as expected (~1 week)
Work towards productionizing the flows (~ 1 week)
Setup monitoring for validation purposes (~1 week)
Optimistically, a new application will have to spend 4-5 weeks before their needs are met - not including the time spent in follow-ups, prioritization asks etc. If the nature of the request changes in the interim, the process might take significantly longer.
...
Gobblin as a Service (GAAS) overall Architecture
The figure above provides a high level architectural view of Gobblin-as-a-Service.
...
Service Cluster (Frontend / Backend)
GaaS will itself run in a cluster mode for high availability. This cluster will consist of identical nodes backed by a VIP for load balancing. Each node in the cluster will host a frontend server and a backend server.
...
Service Core Architecture (modules running on each machine)
As described in the previous section, each node / machine in the cluster will run identical subcomponents.
...