Another example is the Task Scheduler. Getting commands from users using the REST API, it allocates multiple Evaluators and submits Tasks.
It is a basic Task Scheduler example using Reef-webserver. The application receives the Task (shell command) list from user and execute the Tasks in a FIFO order.
REST API
Users can send the HTTP request to the server via URL :
http://{address}:{port}/reef-example-scheduler/v1
And the possible requests are as follows:
/list
: lists all the Tasks’ statuses./clear
: clears all the Tasks waiting in the queue and returns the number of Tasks that have been removed./submit?cmd=COMMAND
: submits a Task to execute COMMAND and returns the Task id./status?id=ID
: returns the status of the Task with the id, “ID”./cancel?id=ID
: cancels the Task with the id, “ID”./max-eval?num={num}
: sets the maximum number of Evaluators.
The result of each task is written in the log files - both in the driver’s and the evaluators’.
Reusing the Evaluators
You can find the method retainEvaluator()
in SchedulerDriver
:
/** * Retain the complete evaluators submitting another task * until there is no need to reuse them. */ private synchronized void retainEvaluator(final ActiveContext context) { if (scheduler.hasPendingTasks()) { scheduler.submitTask(context); } else if (nActiveEval > 1) { nActiveEval--; context.close(); } else { state = State.READY; waitForCommands(context); } }
EventHandler
for the CompletedTask
event is invoked. An instance of CompletedTask
is then passed using the parameter to get the ActiveContext
object from the CompletedTask
. We can reuse this Evaluator by submitting another Task to it if there is a Task to launch.Using the -retain false argument disables this functionality and allocates a new Evaluator for every Task.