Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).


Redis Streams is an append-only data structure that helps in various streaming use cases like real-time messaging (RTM), message broker, etc. There was a Jira ticket created in January 2020, which shows quite a high number of people watching progress on that ticket. With the introduction of externalizing connectors from Flink to their own repository, it also makes it interesting to add a Redis Streams connector in its own external connector repository. 

Public Interfaces

The Redis Streams connector will consist of both a Source and Sink. The following interfaces will be used: 

Proposed Changes

The new connector will be based on the Unified Source Interface (FLIP-27) for the Source and the Async Sink (FLIP-171) for the Sink, support both Bounded (Batch) and Unbounded (Streaming) and both DataStream and Table API/SQL.

The Flink community will create a new connector repo, similar to ElasticSearch [1]. The repository name will be:

  • flink-connector-redis-streams


Versioning Strategy

The flink-connector-redis-streams version will be independent of Flink. We will follow the same versioning strategy as Flink in terms of feature freeze windows, release candidates and branching/tagging. We will publish a Flink support matrix in the connector README and also update Flink documentation to reference supported connectors. The initial release of flink-connector-redis-streams will target 3.0.0 and support Flink 1.16.x and upwards. 

Compatibility, Deprecation, and Migration Plan

  • With respect to Flink, this is a new feature, no compatibility, deprecation, and migration plan is expected.

Test Plan

We will add the following tests:

  • Unit tests
  • Integration tests that perform end to end tests against a Redis Streams testcontainer

Rejected Alternatives