Page tree
Skip to end of metadata
Go to start of metadata

Problem Statement

Currently, when a developer wants to change the configuration of a cluster, say, create a region (or destroy an index, or update an async event queue) and have the change persisted in the cluster configuration for incoming servers, there is no public API for them to do so. They will have to replicate the effort of the equivalent gfsh command to achieve the same effect. It would be nice if we can expose what these commands do to a public API.

Product Goals 

The developer should be able to save their configuration to the Cluster Management Service without having to restart the servers.

Obtain the cluster management service from a cache when calling from a client or a server.

Pass config object to the cluster management service.

CRUD operations support for config objects.

User Goals

Create a more modular product to allow for easy extension and integration. The beneficiaries of this work are the developers working on different parts of the code such as Spring Data for Apache, Queries for Lucene index, storage for the JDBC connector, other Geode developers and anyone who wants to change the configuration of the cluster (create/destroy regions, indices or gateway receivers/senders etc), have these changes replicated on all the applicable servers and persisted in the configuration persistence service for new joining servers.

What We Have Now:

Our admin rest API "sort of" already serves this purpose, but it has these shortcomings:

  1. It's not a public API
  2. The API is restricted to the operations implemented as Gfsh commands, as the argument to the API is a gfsh command string.
  3. Each command does similar things yet not consistent with each other.

Below is a diagram of the current state of things:

commands

From the current state of commands, It's not easy to extract a common interface for all the commands. And developers do not want to use a gfsh command strings as a "makeshift" API to call into the command. We are in need of a unified interface and a unified workflow for all the commands as well.

Proposal

A new Cluster Management Service (CMS) which has two responsibilities:

  • Update runtime configuration of servers (if any running)
  • Persist configuration (if enabled)

highlevel

CMS API is exposed as a new endpoint as part of "Admin REST APIs", accepting configuration objects (JSON) that need to be applied to the cluster. CMS adheres to the standard REST semantics, so users can use POST, PUT, DELETE & GET to create, update, delete or read respectively. API returns a JSON body that contains a message describing the result along with returning standard HTTP status codes.

Create API

APIStatus CodeResponse Body

 

 

 

 

 

 

 

 

Endpoint: http://locator:8080/configure?ifNotExists=true

Method: POST

Headers:

security-username: user1

security-password: password1

Body:

Request Body
{
  "regionConfig": {
      "name"  : "Foo",
      "refId" : "REPLICATE"
  }
}
201
Success Response
{
    "message": "Region created on server1, server2, server3"
}
400
Success Response
{
    "message": "Region type is a required parameter"
}
400
Success Response
{
    "message": "Region with name '/Foo' already exist"
}
401
Success Response
{
    "message": "missing authorization parameters"
}
403
Success Response
{
    "message": "user1 not authorized for DATA:MANAGE"
}

Update/Delete/Get API

APIStatus CodeResponse Body



Endpoint: http://locator:8080/configure?ifExists=true

Method: DELETE

Headers:

security-username: user1

security-password: password1

Body:

Request Body
{
  "regionConfig": {
      "name"  : "Foo"
  }
}

 

 

 

 

 

 

PS: Only DELETE is illustrated.

For PUT/GET respective

request/responses are assumed.

 

200

Success Response
{
    "message": "Region '/Foo' destroyed on Server1, Server2, Server3"
}
401
Success Response
{
    "message": "missing authorization parameters"
}
403
Success Response
{
    "message": "user1 not authorized for DATA:MANAGE"
}
404
Success Response
{
    "message": "Region with name '/Foo' does not exist"
}

 

Let's look at some code to see how users can use this service. The below example shows how to create a region using CMS.

Curl (any standard REST client)

Curl
curl http://locator.host:8080/configure?ifNotExists=true -XPOST -d '
{
  "regionConfig": {
      "name"  : "Foo",
      "refId" : "REPLICATE"
  }
}'

On Client 

 

Client
public class MyApp {
  public static void main(String[] args) {
    //1. Get the service from Cache
    ClientCache cache = new ClientCacheFactory().addPoolLocator("127.0.0.1", 10334).create();
    ClusterManagementService cms = cache.getClusterManagementService();
    
    //2. Create the config object, these are just JAXB generated POJOs
    RegionConfig regionConfig = new RegionConfig(); //These are JAXB generated configuration objects
    regionConfig.setName("Foo");
    regionConfig.setrefId("REPLICATE");
    
    //3. Invoke create, update, delete or get dependening on what you want to do.
    ConfigResult result = cms.create(regionConfig, null, true); //create(config, memberOrGroup, ifNotExists) returns a ConfigResult or throws an exception   
  }
}

onClient-Sequence

On Server

Here's how one can use CMS on a server.

Server
public class MyFunction implements Function<String> {
  @Override
  public void execute(FunctionContext context) {
    //1. Get the service from cache
 	Cache cache = context.getCache();
    ClusterConfigurationService cms = Cache.getClusterManagementService();
    
    //2. Create the config object, these are just JAXB generated POJOs
    RegionConfig regionConfig = new RegionConfig(); //These are JAXB generated configuration objects
    regionConfig.setName("Foo");
    regionConfig.setrefId("REPLICATE");
    
    //3. Invoke create, update, delete or get dependening on what you want to do.
    ConfigResult result = cms.create(regionConfig, null, true); //create(config, memberOrGroup, ifNotExists) returns a ConfigResult or throws an exception
  }
}


onServer-Sequence

Behind the scenes

Following the effort here, Configuration Persistence Service, we already have a set of configuration objects derived from the cache XML schema. This would serve a common object that developers would use to configure the config instance first and then ask the cluster management service to persist it, either on the cache(create the real thing on an existing cache) or on the configuration persistence service (persist the configuration itself). 

ConfigElement

 

On the locator side, the configuration service framework will just handle the workflow. It's up to each individual ClusterConfigElement to implement how it needs to be persisted and applied. 

Pros and Cons:

Pros:

  1. a common interface to call either on the locator/server/client side.
  2. a common workflow to enforce behavior consistency.
  3. Modularized implementation. The configuration object needs to implement the additional interfaces in order to be used in this API. This allows us to add functionality gradually and per function groups.

Cons:

  1. Existing gfsh commands need to be refactored to use this API as well, otherwise we would have duplicate implementations, or have different behaviors between using this API and using gfsh commands.
  2. When refactoring gfsh commands, some commands' behavior will change if they want to strictly follow this workflow, unless we add additional APIs for specific configuration objects.

Migration Strategy:

Our current commands uses numerous command options to configure the behavior of the commands. We will have to follow these steps to refactor the commands.

  1. combine all the command options into one configuration objects inside the command itself.
  2. have the command execution call the public API if the command conforms to the new workflow. In this step, the config objects needs to implement the ClusterConfigElement
  3. If the command can't use the common workflow, make a special method in the api for that specific configuration object (we need to evaluate carefully. we don't want to make too many exceptions to the common workflow)

The above work can be divided into functional groups and have the different group share the workload.

Once all the commands are converted using the ClusterManagementService API, All the command classes are just a facade of collecting the options values, build the config object and call into the API. At this point, the command objects can only exist on the gfsh client.

The end architecture would look like this:

migration

Project Milestones

  1. API is clearly defined
  2. All commands are converted using this API
  3. Command classes only exist on Gfsh client. The GfshHttpInvoker uses the rest API to call this ClusterConfigurationService with the configuration objects directly.

 

  • No labels

1 Comment

  1. The REST API design does not follow the best practice for designing a REST API. Rather relying on the payload, we should probably do something like:

    VERBPATHAction
    GET/geode/v1/regionsReturns all the regions created
    POST/geode/v1/regions/customercreates a region "customer" based on the payload
    GET/geode/v1/regions/customergets the current configuration of the region named customer
    DELETE/geode/v1/regions/customerdestroys the customer region