You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

Status

Current state: In-progress

Discussion thread: 

JIRA: 

Motivation

The user demand for UI proxying has been growing which is terrific but at the same time there are some painful truths related to treating UIs just as we do APIs.

APIs generally have an API specific URL and app path therefore the rewrite rules don't need to be very specific in order to match an incoming URL or response contained

URL with a service definition and set of rewrite rules.

UIs, on the other hand, often have no app path and many response contained relative URLs.

Moreover, the use of common application frameworks means that many of the relative URLs are the same across UIs and it becomes ambiguous as to which service definition

and set of rewrite rules are to be engaged.

This has lead to the current state of UI rewrite rules in which every single file in an application needs to have its own rewrite rule to match it to the appropriate resource.

This is unfortuantely very brittle and combining this brittleness with the nature of modern UIs as constantly evolving and changing and frameworks being upgraded, etc - leaves

us with a moving target that will be difficult to track and to provide backward compatibility for older versions.

This KIP is proposing an approach to address the above issues and provide much simpler rewrite rules for UI proxying.

Topologies

One of the approaches that folks have used is to isolate an individual UI in a specific topology.

This has a couple advantages, such as relative URLs will be less likely to collide with those of another application.

Others have gone as far as to break up larger UIs into smaller service definitions in order to make the set of rewrite rules more manageable.

As you look at these deployments however, it starts to breakdown the namespacing intents that topologies originally provided.

Knox's multiple cluster support is based on ability to represent a cluster with a single topology.

By breaking out something like the hbaseui into a separate topology such as hbaseui.xml we break the topology namespace in a way that relegates Knox to

supporting a single hbase UI across all clusters that Knox is used as a proxy for.

Now, one could using a naming convention to combine the hbaseui and default cluster into a topology name to preserve the namespacing. However, this will require some thought

about what the URL is for the deployed resources and makes it difficult to know what they will be from other applications that may reference them - such as Ambari quicklinks.

The use of topology based isolation does have its merits though and we will describe an approach that preserves the cluster deployment namespace in a way that still allows for the

same benefits.

New <ui> Elements

In order to preserve the namespacing intents that we had in mind for topologies and still leverage separate topologies for UIs, we will need to be able to expect a URL into a topology

that represents a given cluster.

By adding a new element to the topology that is specific for UIs, we can leverage a browser redirect out to a UI specific topology.

This single redirect will need to allow a topology level URL to be used in order to avoid the need to add an app context path such as "hbaseui" to every response contained relative URL.

For instance the following should provide a browser redirect from the containing default.xml topology to the hbaseui.xml topology without requiring a service name for each served URL:

<ui>
    <role>HBASEUI</role>
    <url>gateway/default-hbaseui</url>
    <remote-url>actual-url</remote-url>
</ui>

The "default-hbaseui" above is the name of the hbaseui specific topology for the default cluster represented by the default.xml topology.

The name could be anything and maybe it even makes sense to make it a uuid instead of making it easier to bypass the redirect.

The use of the above element at deployment time would also allow for the generation of the UI specific topology called "default-hbaseui.xml" for example.

Default Service for Topologies

In order to eliminate the need to add an artificial app context path to every URL in a UI and be able to use "gateway/default-hbaseui" to access a UI, we will need some notion of the

service intended for that topology. We can add a default service to the topology which will mean that the GatewayFilter matching will need to associate a URL at the topology level

with the default service that has been configured for that topology.

This will also allow us to add a default service to any topology - whether that service is a UI, an API or a hosted application would be immaterial.

<topology default-service="hbaseui">
...
    <service>
        <role>HBASEUI</role>
        <url>actual-url</role>
    </service>
</topology>

 

So, the above use of the *default-service="hbaseui"* would allow a request to gateway/default-hbaseui/ to be matched and directed to the HBASEUI service and subsequently dispatched to the actual-url for the hbase UI.

Impact on Rewrite Rules

<rules>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/root" pattern="*://*:*/**/hbase/webui/">
 <rewrite template="{$serviceUrl[HBASEUI]}/"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/path" pattern="*://*:*/**/hbase/webui/{**}">
 <rewrite template="{$serviceUrl[HBASEUI]}/{**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/query" pattern="*://*:*/**/hbase/webui/{**}?{**}">
 <rewrite template="{$serviceUrl[HBASEUI]}/{**}?{**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/regionserver/home" pattern="*://*:*/**//hbase/webui/regionserver?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/rs-status"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/home" pattern="*://*:*/**//hbase/webui/master?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/master-status"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/logs1" pattern="*://*:*/**/hbase/webui/logs?{scheme}?{host}?{port}?{**}">
 <rewrite template="{scheme}://{host}:{port}/logs/?{**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/masterlogs" pattern="*://*:*/**/hbase/webui/logs">
 <rewrite template="{$serviceUrl[HBASEUI]}/logs/"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/dump" pattern="*://*:*/**/hbase/webui/dump?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/dump"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/masterdump" pattern="*://*:*/**/hbase/webui/dump">
 <rewrite template="{$serviceUrl[HBASEUI]}/dump"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/jmx" pattern="*://*:*/**/hbase/webui/jmx?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/jmx"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/masterjmx" pattern="*://*:*/**/hbase/webui/jmx">
 <rewrite template="{$serviceUrl[HBASEUI]}/jmx"/>
 </rule>
<rule dir="IN" name="HBASEUI/hbase/inbound/zkdump" pattern="*://*:*/**/hbase/webui/zk.jsp">
 <rewrite template="{$serviceUrl[HBASEUI]}/zk.jsp"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/loglevel" pattern="*://*:*/**/hbase/webui/logLevel?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/logLevel"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/regions" pattern="*://*:*/**/hbase/webui/region.jsp?{host}?{port}?name={**}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/region.jsp?name={**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/storefile" pattern="*://*:*/**/hbase/webui/storeFile.jsp?{host}?{port}?name={**}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/storeFile.jsp?name={**}"/>
 </rule>
<rule dir="IN" name="HBASEUI/hbase/inbound/logfile" pattern="*://*:*/**/hbase/webui/{host}/{port}/logs/{**}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/{host}/{port}/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/regionserver/home" pattern="//{host}:{port}/rs-status/">
 <rewrite template="{gateway.url}/hbase/webui/regionserver?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/master/home" pattern="//{host}:{port}/master-status/">
 <rewrite template="{gateway.url}/hbase/webui/master?host={$hostmap(host)}?{port}"/>
 </rule>
<rule dir="OUT" name="HBASEUI/png" pattern="/static/hbase_logo_small.png">
 <rewrite template="{gateway.url}/hbase/webui/static/hbase_logo_small.png"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/css" pattern="/static/css/{**}">
 <rewrite template="{gateway.url}/hbase/webui/static/css/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/js" pattern="/static/js/{**}">
 <rewrite template="{gateway.url}/hbase/webui/static/js/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/static" pattern="/static/{**}">
 <rewrite template="{gateway.url}/hbase/webui/static/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/master" pattern="/master-status">
 <rewrite template="{gateway.url}/hbase/webui/master-status"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/webui/outbound/zkdump" pattern="//{host}:{port}/zk.jsp">
 <rewrite template="{gateway.url}/hbase/webui/zk.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/webui/outbound/zkdump2" pattern="/zk.jsp">
 <rewrite template="{gateway.url}/hbase/webui/zk.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/tables" pattern="//{host}:{port}/tablesDetailed.jsp">
 <rewrite template="{gateway.url}/hbase/webui/tablesDetailed.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/tables2" pattern="/tablesDetailed.jsp">
 <rewrite template="{gateway.url}/hbase/webui/tablesDetailed.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/procedures" pattern="/procedures.jsp">
 <rewrite template="{gateway.url}/hbase/webui/procedures.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/regions" pattern="//{host}:{port}/region.jsp?name={**}">
 <rewrite template="{gateway.url}/hbase/webui/region.jsp?host={$hostmap(host)}?{port}?name={**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logs" pattern="//{host}:{port}/{**}/{**}/logs/">
 <rewrite template="{gateway.url}/hbase/webui/logs?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logs1" pattern="/logs/">
 <rewrite template="{gateway.url}/hbase/webui/logs/"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logs/files" pattern="/{host}/{port}/logs/{**}">
 <rewrite template="{gateway.url}/hbase/webui/{host}/{port}/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logs/files1" pattern="/logs/{**}">
 <rewrite template="{gateway.url}/hbase/webui/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/logs2" pattern="/logs/">
 <rewrite template="{$frontend[url]}/hbase/webui/logs/"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/logs/files2" pattern="/logs/{**}">
 <rewrite template="{$frontend[url]}/hbase/webui/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logLevel" pattern="//{host}:{port}/logLevel">
 <rewrite template="{gateway.url}/hbase/webui/logLevel?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/debug" pattern="//{host}:{port}/dump">
 <rewrite template="{gateway.url}/hbase/webui/dump?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/jmx" pattern="//{host}:{port}/jmx">
 <rewrite template="{gateway.url}/hbase/webui/jmx?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/conf" pattern="//{host}:{port}/conf">
 <rewrite template="{gateway.url}/hbase/webui/conf?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/table" pattern="//{host}:{port}/table.jsp?name={**}">
 <rewrite template="{gateway.url}/hbase/webui/table.jsp?host={$hostmap(host)}?{port}?name={**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/storefile" pattern="//{host}:{port}/storeFile.jsp?name={**}">
 <rewrite template="{gateway.url}/hbase/webui/storeFile.jsp?host={$hostmap(host)}?{port}?name={**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/filter" pattern="?{filter}">
 <rewrite template="{gateway.url}/hbase/webui/master-status?{filter}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/format" pattern="?{format}?{filter}">
 <rewrite template="{gateway.url}/hbase/webui/master-status?{format}?{filter}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/conf2" pattern="/conf">
 <rewrite template="{$frontend[url]}/hbase/webui/conf"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/jmx2" pattern="/jmx">
 <rewrite template="{$frontend[url]}/hbase/webui/jmx"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/debug2" pattern="/dump">
 <rewrite template="{$frontend[url]}/hbase/webui/dump"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/logLevel2" pattern="/logLevel">
 <rewrite template="{$frontend[url]}/hbase/webui/logLevel"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/logs/files2" pattern="/logs/{**}">
 <rewrite template="{$frontend[url]}/hbase/webui/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/logs2" pattern="/logs/">
 <rewrite template="{$frontend[url]}/hbase/webui/logs/"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/tables2" pattern="/tablesDetailed.jsp">
 <rewrite template="{$frontend[url]}/hbase/webui/tablesDetailed.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/master" pattern="/master-status">
 <rewrite template="{$frontend[url]}/hbase/webui/master-status"/>
 </rule>
 <filter name="HBASEUI/hbase/outbound/headers/logs">
 <content type="application/x-http-headers">
 <apply path="Location" rule="HBASEUI/hbase/outbound/headers/logs/location"/>
 </content>
 </filter>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/headers/logs/location">
 <match pattern="{scheme}://{host}:{port}/logs/?{**}"/>
 <rewrite template="{$frontend[url]}/hbase/webui/logs?{scheme}?host={$hostmap(host)}?{port}?{**}"/>
 </rule>
</rules>

The above is from the service definition in the 0.13.0 release for hbaseui.

This proposal intends to change the above to something more like:

<rules>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/root" pattern="*://*:*/**/">
 <rewrite template="{$serviceUrl[HBASEUI]}/"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/path" pattern="*://*:*/**/{**}">
 <rewrite template="{$serviceUrl[HBASEUI]}/{**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/query" pattern="*://*:*/**/{**}?{**}">
 <rewrite template="{$serviceUrl[HBASEUI]}/{**}?{**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/regionserver/home" pattern="*://*:*/**//regionserver?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/rs-status"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/master/home" pattern="*://*:*/**//master?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/master-status"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/logs1" pattern="*://*:*/**/logs?{scheme}?{host}?{port}?{**}">
 <rewrite template="{scheme}://{host}:{port}/logs/?{**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/masterlogs" pattern="*://*:*/**/logs">
 <rewrite template="{$serviceUrl[HBASEUI]}/logs/"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/dump" pattern="*://*:*/**/dump?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/dump"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/masterdump" pattern="*://*:*/**/dump">
 <rewrite template="{$serviceUrl[HBASEUI]}/dump"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/jmx" pattern="*://*:*/**/jmx?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/jmx"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/masterjmx" pattern="*://*:*/**/jmx">
 <rewrite template="{$serviceUrl[HBASEUI]}/jmx"/>
 </rule>
<rule dir="IN" name="HBASEUI/hbase/inbound/zkdump" pattern="*://*:*/**/zk.jsp">
 <rewrite template="{$serviceUrl[HBASEUI]}/zk.jsp"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/loglevel" pattern="*://*:*/**/logLevel?{host}?{port}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/logLevel"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/regions" pattern="*://*:*/**/region.jsp?{host}?{port}?name={**}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/region.jsp?name={**}"/>
 </rule>
 <rule dir="IN" name="HBASEUI/hbase/inbound/storefile" pattern="*://*:*/**/storeFile.jsp?{host}?{port}?name={**}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/storeFile.jsp?name={**}"/>
 </rule>
<rule dir="IN" name="HBASEUI/hbase/inbound/logfile" pattern="*://*:*/**/{host}/{port}/logs/{**}">
 <rewrite template="{$serviceScheme[HBASEUI]}://{host}:{port}/{host}/{port}/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/regionserver/home" pattern="//{host}:{port}/rs-status/">
 <rewrite template="regionserver?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/master/home" pattern="//{host}:{port}/master-status/">
 <rewrite template="master?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/tables" pattern="//{host}:{port}/tablesDetailed.jsp"> 
 <rewrite template="tablesDetailed.jsp"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/regions" pattern="//{host}:{port}/region.jsp?name={**}">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="region.jsp?host={$hostmap(host)}?{port}?name={**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logs" pattern="//{host}:{port}/{**}/{**}/logs/">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="logs?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logs/files" pattern="/{host}/{port}/logs/{**}">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="{host}/{port}/logs/{**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/logLevel" pattern="//{host}:{port}/logLevel">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="logLevel?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/debug" pattern="//{host}:{port}/dump">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="dump?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/jmx" pattern="//{host}:{port}/jmx">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="jmx?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/conf" pattern="//{host}:{port}/conf">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="conf?host={$hostmap(host)}?{port}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/table" pattern="//{host}:{port}/table.jsp?name={**}">    // WHAT IS THIS ONE ABOUT????
 <rewrite template="table.jsp?host={$hostmap(host)}?{port}?name={**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/storefile" pattern="//{host}:{port}/storeFile.jsp?name={**}">
 <rewrite template="storeFile.jsp?host={$hostmap(host)}?{port}?name={**}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/filter" pattern="?{filter}">
 <rewrite template="master-status?{filter}"/>
 </rule>
 <rule dir="OUT" name="HBASEUI/format" pattern="?{format}?{filter}">
 <rewrite template="master-status?{format}?{filter}"/>
 </rule>
 <filter name="HBASEUI/hbase/outbound/headers/logs">
 <content type="application/x-http-headers">
 <apply path="Location" rule="HBASEUI/hbase/outbound/headers/logs/location"/>
 </content>
 </filter>
 <rule dir="OUT" name="HBASEUI/hbase/outbound/headers/logs/location">
 <match pattern="{scheme}://{host}:{port}/logs/?{**}"/>
 <rewrite template="{$frontend[url]}/hbase/webui/logs?{scheme}?host={$hostmap(host)}?{port}?{**}"/>
 </rule>
</rules>

The above has not been tested yet but it will be what is used to start a POC for this functionality.

HBase UI happens to have one of the more complicated scenarios due to the master and regionserver splits which is why there is still a good number of rules required on OUT

rules for disambiguating the same pages from multiple endpoints.

It may be able to be simplified more and we may need to put some things back but this is a starting point at least.

The simplifications above consist of:

  • removing rules that aren't needed for relative URLs anymore
  • removing all of the gateway-url host and port and frontend-url where relative paths will work
  • removing all of the hbase/webui context paths since relatives should work with the default-service feature for the topology

Simpler scenarios where there is only one endpoint - like Ambari - will see much larger simplifications in the rewrite rules.

 

  • No labels