Status

Current state: In-progress

Discussion thread: 

JIRA: KNOX-928 - Getting issue details... STATUS

Motivation

The Default Topology feature as it stands now seems to be broken, there were multiple attempts made to fix it but the recent refactoring seems to have broken it. This feature is useful and some advantages. Also it could be sometimes confusing to use it and difficult to troubleshoot and maintain (since it uses the same port as all other topologies use). 

This KIP proposes a similar functionality but proposes that they use a different port. The requests to this topology do not have to include the {gateway} and {topology} elements. So essentially it will be Default Topology feature but now we can have support for multiple topologies (listening on different ports). Off-course the standard port configured through gateway config will work just the way it always has, so all the existing features will not be affected.

Say, we have three topologies alongside our default 'sandbox' topology, 'eerie', 'ontario' and 'huron'.  These three topologies will listen on 3 separate ports, say, 5443, 6443 and 7443. Default gateway port will not be  affected by these ports and these topologoes can be accessed from the default port (i.e. 8443).

In the above example one should be able to make a successful request to

https://{gateway-host}:5443/webhdfs

https://{gateway-host}:6443/webhdfs

https://{gateway-host}:7443/webhdfs

https://{gateway-host}:8443/gateway/eerie/webhdfs 

https://{gateway-host}:8443/gateway/ontario/webhdfs

https://{gateway-host}:8443/gateway/huron/webhdfs

This would provide interesting possibilities for existing hadoop cli's.

Architecture

The following is the startup sequence diagram for this feature. Changes are highlighted in blue.

Configuration

Configuration for this feature will be in gateway-site.xml config file.

Following are the settings relevant to this feature

• The property gateway.port.mapping.enabled is used to turn feature ON/OFF.
• The property gateway.port.mapping.{topologyName} and it value {port} maps {topologyName} to {port}.

Example:

    <!-- Optional, true by default-->
    <property>
        <name>gateway.port.mapping.enabled</name>
        <value>true</value>
        <description>Enable/Disable gateway topology port mapping feature.</description>
    </property>

    <!-- Multi Port Gateway -->
    <property>
        <name>gateway.port.mapping.eerie</name>
        <value>5443</value>
        <description>The HTTP port for the Gateway.</description>
    </property>
    
    <property>
        <name>gateway.port.mapping.huron</name>
        <value>6443</value>
        <description>The HTTP port for the Gateway.</description>
    </property>
    
    <property>
        <name>gateway.port.mapping.ontario</name>
        <value>7443</value>
        <description>The HTTP port for the Gateway.</description>
    </property>

Improvements

1. Need to address the inability for the hadoop java client to deal with spnego challenges from the DN redirect.