Status
Current state: Under discussion
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Kafka Connect provides a REST interface for managing connectors. Below is a list of supported documented endpoints:
Method | Path |
---|---|
GET | /connectors |
POST | |
GET | /connectors/{name} |
GET | /connectors/{name}/config |
PUT | |
GET | /connectors/{name}/status |
GET | /connectors/{name}/tasks |
GET | /connectors/{name}/tasks/{taskid}/status |
PUT | /connectors/{name}/pause |
PUT | /connectors/{name}/resume |
POST | /connectors/{name}/restart |
POST | /connectors/{name}/tasks/{taskId}/restart |
DELETE | /connectors/{name} |
GET | /connector-plugins |
PUT | /connector-plugins/{connector-type}/config/validate |
In addition to above methods, OPTIONS method could be also used.
Responce to OPTIONS request includes auto-generated WADL, e.g.:
curl -i -X OPTIONS http://localhost:8083/connectors/jdbc-sink HTTP/1.1 200 OK Date: Thu, 13 Dec 2018 02:15:30 GMT Content-Type: application/vnd.sun.wadl+xml Allow: HEAD,DELETE,GET,OPTIONS Last-Modified: Wed, 12 Dec 2018 18:15:30 PST Content-Length: 1153 Server: Jetty(9.4.12.v20180830) <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <application xmlns="http://wadl.dev.java.net/2009/02"> <doc xmlns:jersey="http://jersey.java.net/" jersey:generatedBy="Jersey: 2.27 2018-04-10 07:34:57"/> <grammars> <include href="http://localhost:8083/application.wadl/xsd0.xsd"> <doc title="Generated" xml:lang="en"/> </include> </grammars> <resources base="http://localhost:8083/"> <resource path="connectors/jdbc-sink"> <method id="getConnector" name="GET"> <request> <param xmlns:xs="http://www.w3.org/2001/XMLSchema" name="forward" style="query" type="xs:boolean"/> </request> <response> <representation mediaType="application/json"/> </response> </method> <method id="destroyConnector" name="DELETE"> <request> <param xmlns:xs="http://www.w3.org/2001/XMLSchema" name="forward" style="query" type="xs:boolean"/> </request> </method> </resource> </resources> </application>
Having WADL is useful for code generation, applications modeling and visualizations, etc. At the same time, it might also be a security threat.
So users should have a way to disable exposing WADL via Connect REST endpoints.
Public Interfaces
Add a new Kafka Connect Distributed/Standalone worker configuration property to enable/disable exposure of WADL via Connect REST endpoints.
Proposed Changes
Add configuration parameter:
- rest.wadl.enable with default value of 'true', and with importance of 'low'.
Update org.apache.kafka.connect.runtime.rest.RestServer to respect the configuration property described above.
With rest.wadl.enable=true, responce to OPTIONS method should have list of supported HTTP methods only, e.g.:
curl -i -X OPTIONS http://localhost:8083/connectors/jdbc-sink HTTP/1.1 200 OK Date: Thu, 13 Dec 2018 02:52:30 GMT Content-Type: text/plain Allow: HEAD,DELETE,GET,OPTIONS Content-Length: 26 Server: Jetty(9.4.12.v20180830) HEAD, DELETE, GET, OPTIONS
Test plan
Add more unit tests under org.apache.kafka.connect.runtime.rest.RestServerTest, asserting on responce's Content-type.
Compatibility, Deprecation, and Migration Plan
Since default value of the new configuration parameter is 'true', the change should be backward compatible, no action is required for upgrading from older versions.
Rejected Alternatives
- Disable exposure of WADL without option to keep exposing it. Some users might rely upon it.