Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


  1. State Machine

    State machine tells what state should the job or instance go to on a certain action. // TODO - Add details of all actions and state transitions.
  2. Notification Services

    For launching an instance scheduler needs to check if all the prerequisites for launching a new instance have been met. For example:
    1.  Is it time to launch a new instance, according to the schedule of the job?
    2. Is data available in the input locations?
    For the first version, scheduler , will have only time based criteria to schedule the jobs. This is supported through through AlarmService.
  3. State Store

    This component is responsible to maintain the latest state of all the jobs and their instances. For the purpose of unit testing we use an in-memory database but for production deployments we will support persistent relational databases like mysql etc. For the purpose of state management, two tables - scheduler_jobs and scheduler_job_instances are maintained.   //TODO list the schema of both the tables and the schema.
  4. Core Scheduler Engine 

    This is the core of the scheduler engine. It orchestrates the other components and services. It listens to the notification services and launches new instances on the basis of that. It listens to various commands like submit/pause/delete etc., finds transitions using the State Machine, subscribes for various notification services and persists state in the state store.



There are two ways to interact with the scheduler.


    There is a REST API to interact with the scheduler. //TODO list various REST API and their responses.
  2. CLI

    Various commands to interact with the Scheduler will be exposed on the CLI as well. CLI will interact with the Scheduler through REST API. //TODO list all CLI commands



Scheduler Job Schema


Scheduler Job Instance Schema