To be Reviewed By: 3/20/2020

Authors: Dan Smith, Ernie Burghardt

Status: Draft | Discussion | Active | Dropped | Superseded

Superseded by: N/A

Related: GEODE-7852 [DROPPED] Client side configuration for a SNI proxy

Problem

Users would like to be able to put Geode servers behind a proxy. This would allow users to avoid having to allocate public IP addresses for all of the servers, as well as being able to control and monitor access to the servers.

Currently, this is not very well supported. Geode clients discover servers using locators. It is possible to set the hostname-for-clients on all servers to point to the proxy server, but this is not ideal. Geode clients rely on being able to connect to specific  servers for features like PR single hop and client server queues. This means that the clients need a protocol that will allow them to tell the proxy which server they connect to.

Anti-Goals

  • It is an anti-goal to deal with configuring the proxy itself.
  • It is an anti-goal to support all different kinds of proxies. In the future we may consider supporting other types of proxies, but this proposal is to support one specific type - a SNI proxy.
  • We're not recommending any specific SNI proxy
  • This proposal does not include supporting a proxy for WAN or P2P connectivity. In the future we may add support for that as well.

Solution

We will add a way for the user to set a SNI proxy for the client to connect to the servers through. When this option is set, all connections from the client to the locator or the servers will use this SNI proxy. When configuring a Pool, the names of the locators or the names of the servers provided in the pool configuration will be passed as the Server Name Indicator field when the client connects to the proxy. The proxy must be able to resolve these names and connect to the correct locator or server.

This setting ideally should be at a pool level, because different pools may need to use different proxies.

To make it as easy as possible for users to extend our proxy implementation logic or implement their own, we will introduce a more general pool setting; a custom SocketFactory.  This will allow the users to override the creation of client-server sockets. We will provide an implementation of this SocketFactory that will connect their client to a SNI proxy.

The way to configure the SNI proxy will therefore look something like this:

poolFactory.setSocketFactory(ProxySocketFactories.sni("proxyHostname", 443));


In order for this proxy to work, the SNI hostname field in the TLS handshake must be set to the name of the locator or server we are trying to connect to. With a slightly larger SocketFactory interface, we could potentially only send this SNI hostname when an SNI proxy is configured. However, we propose that the Geode client will always send the SNI hostname field as part of the handshake. We feel adding the SNI hostname will only be helpful to users, who may decide to use to to present different SSL certificates for different hostnames, or other use cases for this hostname.


Below are the details on the new methods and interfaces added to the API to support this

Modified classes:

PoolFactory {  

/**
* Set the socket factory used by this pool to create connections to both locators (if
* configured using {@link #addLocator(String, int)}) and servers.
*
* see {@link SocketFactory}
* See {@link ProxySocketFactories}
*
* @param socketFactory The {@link SocketFactory} to use
* @return a reference to <code> this </code>
* @since Geode 1.13
*/
PoolFactory setSocketFactory(SocketFactory socketFactory);
}
 
ClientCacheFactory {
  /**
   * (see description of PoolFactory above)
   */
  setPoolSocketFactory(SocketFactory socketFactory)
}


New classes

package org.apache.geode.cache.client;

/**
* A socket factory used to create sockets from a client to locators or servers.
*
*
* Sockets returned by this factory will have the rest of the configuration options
* specified on the {@link Pool} and on the {@link ClientCache} applied to them. In particular,
* sockets returned by this factory will be wrapped with SSLSockets if ssl is enabled
* for this client cache based on {@link ConfigurationProperties#SSL_ENABLED_COMPONENTS}.
* Sockets return by this factory should not be SSLSockets. For modifying SSL settings,
* see {@link SSLParameterExtension}
*
* Sockets returned by this factory should be in an unconnected state, similar to
* {@link Socket#Socket()}
*
* This factory can be used for configuring a proxy, or overriding various socket settings.
*
* @see PoolFactory#setSocketFactory(SocketFactory)
*/
public interface SocketFactory {
/**
* Create an unconnected tcp socket for establishing a client.
*
* @return an unconnected socket
*/
 Socket createSocket() throws IOException;
}
package org.apache.geode.cache.client.proxy;

public class SniSocketFactory implements SocketFactory {
  public SniSocketFactory(String hostname, int port) {
  ...
}
 
public class ProxySocketFactories {
   public static SocketFactory sni(String hostname, int port)  
}


XML Support


We will also add support for the new SocketFactory option in XML. The SocketFactory can be added using the standard mechanism for declaring classes in Geode's XML - implementing declarable and passing in a provided set of properties

<pool>  
  <socket-factory>
    <class-name>com.company.app.CustomSocketFactory</class-name>
     <parameter name="proxyHost">
       <string>proxy.company.com</string>
     </parameter>  
  </socket-factory>
</pool>


Performance Impact

Connecting through a proxy may impact the performance of client/server messaging, but it is up to the user to decide if they want to use this feature or not. SNI proxies do require the use of TLS, which also adds overhead.

Backwards Compatibility and Upgrade Path

This is a client side setting so there should be no backwards compatibility or upgrade concerns with the setting. We are sending a new piece of information as part of the TLS handshake, but this is a TLS extension and it will be ignored by servers and locators.

One concern with this SocketFactory approach is that is including the use of blocking, Java 1.0 sockets in the API. If in the future we try to upgrade the internals of the client to use SocketChannel or netty or rsocket, we will have a difficult time continuing to support this SocketFactory API and may break users' custom SocketFactory implementations.

Prior Art

Alternative ways to introduce a proxy would be:

  • Set the hostname-for-clients to point to the proxy. However this does not let the client connect to specific servers, because from the client's point of view there is only one server (the proxy). This makes client single hop no longer functional. It also makes it hard for the client to maintain redundancy of server to client queues, for subscriptions.
  • Use a SOCKS5 proxy instead.


Geode also added support to set the SNI hostname in the client hello as part of GEODE-7414. With those changes a user can provide a SSLParameterExtension callback that can modify any of the SSLParameters including the SNI hostname. If a proxy of type SNI is set and the SSLParameterExtension is also set, the SSLParameterExtension will run after geode has set the SNI hostname, and can potential modify it.

FAQ

Is this platform specific?

  • No this feature is intended to be platform agnostic.

Does this require a firewall?

  • This is an implementation detail that will be left to the users discretion.


Errata

What are minor adjustments that had to be made to the proposal since it was approved?


References


  • No labels

24 Comments

  1. >This factory can be used for configuring a proxy, or overriding various socket settings.
       * For modifying SSL settings, see {@link SSLParameterExtension}


    Can you elaborate on this statement? If the SocketFactory is supposed to return Sockets that can be wrapped in SSLSocket then the Socket implementation returned can't be an SSLSocket can it? 

    I would also elaborate on the API documentation that this Socket should not be in a connected state when returned.


    1. I see the "(unconnected)" mentioned in the interface. I the some of that detail in the pool documentation should be duplicated or moved to the factory documentation. I also wouldn't reduce the importance of the word "unconnected" in parentheses. It is very important that the returned Socket is not connected.

    2. I tweaked it - moved information to the SocketFactory interface, emphasized unconnected Sockets, and cautioned about SSLSockets. See what you think.

  2. I'm a little confused as to how the 

    public class Proxies {
       public static SocketFactory sni(String hostname, int port) 
    }

    aids in making this extendible? Is there an expectation that any future implementation (say SOCKS) will be another factory method? Maybe a completely different implementation, as long as it satisfies the criteria of returning a `SocketFactory`?

    1. Proxies is just a little syntatic sugar, eg:

      poolFactory.setSocketFactory(Proxies.sni(host, 443));

      instead of

      poolFactory.setSocketFactory(new SniProxySocketFactory(host, 443));


      So you can still create your own custom SocketFactory, or if we put more of them into geode we can add another method to Proxies.

      I'm not too set on having the Proxies class. But I think it's helpful, especially for the javadocs being able to call out "see Proxies" rather than "see SniProxySocketFactory, Socks5ProxySocketFactory, etc..."

      1. I think a specific SocketFactory, like SniProxySocketFactory or SocksProxySocketFactory, isn't necessarily bad. As long as it is of type `SocketFactory`.

  3. Imo, the following API is misleading.

    poolFactory.setSocketFactory(Proxies.sni("proxyHostname", 443));

    When I read this, I expect to be providing a `SocketFactory`, which is not what I read when I read `Proxies`. Maybe `Proxies` can be renamed to `ProxySocketFactory`? That would read easier for me.


    1. Hmm, so maybe this?

      poolFactory.setSocketFactory(ProxySocketFactories.sni(host, 443))
      1. I like that.. I does convey intent better than the "syntactial sugar" of `Proxies` (smile)

        1. OK. I renamed it to ProxySocketFactories for now. If folks would rather I just delete it, I'm fine with that too.

      2. I like this better than the original Proxies class. I could still do without it too. Call me indifferent on having it at all.

  4. I am a little concerned that we are basing a PUBLIC API on `SocketFactory`, which is still an experimental driver.

    Even looking at the SocketFactory, I struggle to understand why a SocketFactory would be stateful in its implementation. Why it would not take a data object and return a `Socket` alludes me. (but this is completely off topic).

    I cannot, in good conscience, support an API that is based on something that is still experimental. 

    1. Are you talking about org.apache.geode.experimental.driver.SocketFactory? That is a completely different class. This is a newly proposed SocketFactory interface - org.apache.geode.cache.client.SocketFactory.

      1. Ah.. ok... I just searched for `SocketFactory` and found the experimental driver. So you mean the `PoolFactory` will take a `org.apache.geode.cache.client.SocketFactory`? If so, ignore this. Nothing to see here.

  5. At some point we'll need to update the cache XSD's pool-type to support this.

    1. But do we really? Of the use cases for proxy how many do you think will also still be using XML to configure. What if this is an API only feature until there is an ask for it?

    2. I thought we are "deprecating" or at least not advocating to use the cache.xml format anymore... OR maybe it is just me.

  6. Cache.xml is in the docs and isn't deprecated

    1. Ok. I went ahead and added an Xml configuration section. We'll add support for users to pass in a SocketFactory through xml.

      1. My concern is that we don't recommend using cache.xml configuration for the server-side, as it is being super-ceded by ClusterConfiguration. If that statement still holds true, we are now advocating to have cache.xml configuration for the client. Which I believe is a little contradictory to the server configuration.

        I'm with Jacob Barretton this one, why not wait until we have demand for this to be in cache.xml and then address it.

    1. I agree... +1... just go... but I don't want the cache.xml debate to hold it up...