KIP-3 Websocket Support

This KIP discusses the Websocket support for Apache Knox.

Status

Current state: Implemented
JIRA: KNOX-752

Discussion thread: http://mail-archives.apache.org/mod_mbox/knox-dev/201611.mbox/%3CCAMFO%2B3QA%2BMvO%3DNboNsMGD79yeG%3D3rdLsLc4x_Mi%3DnvzES4NxsQ%40mail.gmail.com%3E

Motivation

Websocket is a communication protocol that allows full duplex communication over single TCP connection (for more information see RFC 6455). As more and more web UIs are using websocket for communication with servers it is important that Knox supports proxying of websocket connections so that the UIs which include them can function seamlessly over Knox for e.g. Apache Zeppelin.

KNOX-752 is an initial attempt to enable websocket support for Knox as a result following requirements will be addressed in the initial release:

Proxy UIs using websocket protocol.
Text message support for proxying - As majority of implementations use text over binary.
Use of existing configuration files (topology and service definitions) to configure backend websocket url.
Secure websocket support from browser to Knox (wss://)

Design

The main considerations for design are:

Compatibility with the current configuration (service definitions, topologies)
Reuse of existing components and functionality

We take advantage of Jetty's websocket support (both native and JSR 356 implementations) and leverage it to support websocket connection proxying. We register our websocket handler with Jetty during startup which is then used to intercept websocket connections from the browsers (Http connection upgrade request). After the connection is established it remains open until the browser socket or the backend service socket closes the connection or in an error event.

A pipe is created between the browser socket and the backend service socket served behind Knox. Websocket traffic can then pass through this pipe. There are hooks to inspect the traffic and enforce rewrite rules, the initial implementation does not address these. An attempt is made to relay accurate error messages and events to the browser, which can be improved and customized in future iterations.

Implementation

The above diagram describes, at a high level, how websocket requests are handled.

A browser initiates websocket connection by sending an HTTP connection upgrade request, Knox's websocket handler intercepts this request and returns a secure socket (wss://) to the browser - acting as a websocket server. Knox's websocket handler also establishes another connection with the backend websocker server - acting as a websocket client.

By default websocket functionality is disabled, one can enable it by changing the 'gateway.websocket.feature.enabled' property to 'true' in <KNOX-HOME>/conf/gateway-site.xml file.

	<property>
        <name>gateway.websocket.feature.enabled</name>
        <value>true</value>
        <description>Enable/Disable websocket feature.</description>
    </property>

Service and rewrite rules need to changed accordingly to match the appropriate websocket context (e.g. ws://myhost:9999/ws)

example of <KNOX-HOME>/data/services/{myservice}/{version}/rewrite.xml where myservice = websocket and version = 0.6.0

<rules>
  <rule dir="IN" name="WEBSOCKET/ws/inbound" pattern="*://*:*/**/ws">
    <rewrite template="{$serviceUrl[WEBSOCKET]}/ws"/>
  </rule>
</rules>

example of <KNOX-HOME>/data/services/{myservice}/{version}/service.xml where myservice = websocket and version = 0.6.0

<service role="WEBSOCKET" name="websocket" version="0.6.0">
  <policies>
        <policy role="webappsec"/>
        <policy role="authentication" name="Anonymous"/>
        <policy role="rewrite"/>
        <policy role="authorization"/>
  </policies>
  <routes>
    <route path="/ws">
        <rewrite apply="WEBSOCKET/ws/inbound" to="request.url"/>
    </route>
  </routes>
</service>

Finally, update the topology file at <KNOX-HOME>/conf/{topology}.xml with the backend service url

	<service>
        <role>WEBSOCKET</role>
        <url>ws://myhost:9999/ws</url>
    </service>

A detailed example demonstrating websocket configuration (using docker containers for simplicity) can be found here https://github.com/moresandeep/knox-dev-docker#demo-knox-dev-container–zeppelin

Security Consideration

WebSocket RFC does not define any specific way for servers to authenticate clients it leaves the choice to the application developers. As per the RFC, The WebSocket server can use any client authentication mechanism available to a generic HTTP server, such as cookies, HTTP authentication, or TLS authentication.

The implication of this is that any WebSocket connection opened behind a secured (authenticated) page is not "automatically" secured, application developers have to take necessary steps to secure the WebSocket connection (along with the http page). for e.g. consider a simple WebSocket echo server proxied via Knox. When we make an HTTP request Knox will challenge for credentials

➜  ~ curl -ik https://localhost:8443/gateway/sandbox/echows
HTTP/1.1 401 Unauthorized
Date: Thu, 05 Oct 2017 21:24:45 GMT
WWW-Authenticate: BASIC realm="application"
Content-Length: 0
Server: Jetty(9.2.15.v20160210)

But one could directly use the WebSocket protocol, wss:// to "bypass" Knox's security framework which is based on servlet filters.

➜  ~ wscat -n -c wss://localhost:8443/gateway/sandbox/echows
connected (press CTRL+C to quit)
> hello
< hello

WebSocket connection could be secured in multiple ways, one way is to use the HTTP headers commonly used for authentication as used by web views. In Knox this support will be enabled as part of KNOX-895. It is difficult to customize WebSocket headers from Javascript as a result we are limited by implicit auth (Basic or Cookies) that browsers use. WebSocket servers could be separate from the webservers which makes shared authentication difficult or impossible. KNOX-773 will add support for Basic auth which could ease out some pain but until then use WebSocket with same caution as you would without Knox.

A "ticket" based authentication can be used by applications to handle the authentication problem, similar to what Zeppelin uses as the time or writing. In this method the client side talks to WebSocket server over HTTP, authenticates and obtains a security "ticket". WebSocket server issues this "ticket" and ties it to the user identity (e.g. username) this mapping is then stored in a cache. When the client connects via WebSocket it sends this "ticket" as part of the payload which the WebSocket server uses to verify the user identity and make sure the session is not expired.

Future work

KNOX-772 - Implement binary protocol support for Websocket feature

Currently websocket feature in Knox only supports text messages, to be fully compatible with the websocket specs we need to support binary protocol as well.

KNOX-773 - Secure websocket support on the dispatch side and security enhancements

Currently Knox communicates with browser (or TCP based client) over secure socket however secure communication on the dispatch side is not tested. It is unlikely that it will fail if the certificates for the dispatch (backed) server are provisioned in Knox - again need to check.

Also we need to support all the authentication mechanisms that Knox already supports.

KNOX-774 - Make websocket connections Async and Non-Blocking

Websocket connections could potentially hold up Jetty threads (during handshakes etc.) so it is important to have them non-blocking.

KNOX-775 - Create a generic framework that can be reused and extended by other protocols

Currently Websockets is the only non-http protocol Knox supports, it would help if we could create a generic framework that can be used by all non-http protocols such as Websockets, thrift etc. preventing code duplication and help code and resource re-use.

Space shortcuts

Child pages