Status

Current stateIn-progress

Discussion thread

JIRA:

Motivation

The Default Topology feature as it stands now seems to be broken, there were multiple attempts made to fix it but the recent refactoring seems to have broken it. This feature is useful and some advantages. Also it could be sometimes confusing to use it and difficult to troubleshoot and maintain (since it uses the same port as all other topologies use). 

This KIP proposes a similar functionality but proposes that they use a different port. The requests to this topology do not have to include the {gateway} and {topology} elements. So essentially it will be Default Topology feature but now we can have support for multiple topologies (listening on different ports). Off-course the standard port configured through gateway config will work just the way it always has, so all the existing features will not be affected.

Say, we have three topologies alongside our default 'sandbox' topology, 'eerie', 'ontario' and 'huron'.  These three topologies will listen on 3 separate ports, say, 5443, 6443 and 7443. Default gateway port will not be  affected by these ports and these topologoes can be accessed from the default port (i.e. 8443).

In the above example one should be able to make a successful request to

https://{gateway-host}:5443/webhdfs
https://{gateway-host}:6443/webhdfs
https://{gateway-host}:7443/webhdfs
https://{gateway-host}:8443/gateway/eerie/webhdfs 
https://{gateway-host}:8443/gateway/ontario/webhdfs
https://{gateway-host}:8443/gateway/huron/webhdfs

This would provide interesting possibilities for existing hadoop cli's.

 

Architecture

 

Improvements

  1. Need to address the inability for the hadoop java client to deal with spnego challenges from the DN redirect.