To be Reviewed By: 1 July 2021

Authors: Mario Ivanac

Status: Draft | Discussion | Active | Dropped | Superseded

Superseded by: N/A

Related: N/A

Problem

In our current deployment of geode in kubernetes, we have also deployment of clients in that same kubernetes cluster.

Since we have multi-site configuration, we have defined "hostname-for-clients" (remote IP) for remote sites and remote clients to contact this site.

Now, the problem is for those clients deployed in internal network (in the same kubernetes cluster as geode), when they want to connect to geode:

  • client sends request (LOCATOR_LIST_REQUEST),
  • and in response (LOCATOR_LIST_RESPONSE) according to current logic in geode, it gets "hostname-for-clients" external address, as a address to connect to.

So in this case, for internal clients, communication will go from internal network to external, and then from external to internal. This increases latency, and we should communicate over the shortest path.

Goals

For internal clients to get internal host address of locators to connect to.

Anti-Goals

-

Solution

Proposal of solution is to update clients pool interface.

Idea is that client will send in LocatorListRequest which will contain new indicator RequestLocatorInternalAddress, and locator will in case this indicator is set to true,

reply with locators internal host addresses.

Link on PR for this solution: https://github.com/apache/geode/pull/5948

Only flaw of this solution, is that if external clients request internal locator address.

  • We can solve this with enforcement to use mTLS.
  • or we can include new configuration parameter which will contain info about local subnet range. And when we receive request for internal address, we will check source IP address against new parameter, to verify this is internal client.


Changes and Additions to Public Interfaces

Update of clients Pool interface, and message LocatorListRequest with new parameter requestInternalAddress.

Performance Impact

Performance should be better, for this specific use case.

Backwards Compatibility and Upgrade Path

Changes will be made in backward compatible way.

Prior Art

What would be the alternatives to the proposed solution? What would happen if we don’t solve the problem? Why should this proposal be preferred?

FAQ

Answers to questions you’ve commonly been asked after requesting comments for this proposal.

Errata

What are minor adjustments that had to be made to the proposal since it was approved?

  • No labels

5 Comments

  1. Anthony Baker

    What changes to the public API are you proposing?  Can you provide some example code that shows how this might look for an application?

  2. Mario Ivanac

    Example of impacted code is in linked PR above. Impacts are adding new function setRequestLocatorInternalAddressEnabled() on PoolFactory and ClientCacheFactory.

  3. Dan Smith

    For servers, you can actually do this already with server groups I think.

    You can actually have each member listen on more than one server port, potentially binding on different interfaces. In the java API this would look something like below, although you can also do this with cache.xml

    CacheServer externalServer = cache.addCacheServer()
externalServer.setHostnameForClients("some_external_name")
externalServer.setBindAddress("some_interface")
externalServer.setGroups("external")
externalServer.start()

CacheServer internalServer = cache.addCacheServer()
internalServer.setHostnameForClients("some_internal_name")
internalServer.setBindAddress("other_interface")
internalServer.setGroups("internal")
internalServer.start()


    On the client, you just set the group you want to target

    PoolFactory.setServerGroup("external")


    I'm not sure this is doable in the quite the same way for locators, but they should be less performance critical as far as what address they are using.

  4. Dan Smith

    Another alternative might be to solve this with DNS. Your hostname-for-clients could be a DNS name that resolves to different addresses depending on where the client is and what DNS server it is talking to. If you can do a DNS based solution I think that would be ever better.

  5. Ernest Burghardt

    Hello Mario Ivanac

    I wanted to check if you are still interested in pursuing this RFC...

    If you are the proposed solution still needs to address:

    • Benefits (with some sample performance data)
    • Security concerns/issues with this
    • Case where an external client tries to fetch the internal info (outside of k8s env - what happens in this case)
    • How other client connection could be impacted or can use similar in future (Java, C++, JMX, Gfsh, pulse connections)
    • Requirement for Backward compatibility, rolling upgrade tests

    Thanks,

    EB