Research and development roadmap for Flink Streaming

Fault tolerance guarantees for streaming
 

There has been a lot of development throughout the years towards fault tolerant stateful streaming. It is true that fault tolerance mechanisms can be very expensive in stream processing applications both in storage and latency requirements. Part of our research on Flink is to minimize these costs by taking advantage of semantical properties such as window pre-aggregates as well as investigating different state management approaches. The ultimate goal of fault tolerance in Flink will be to offer exactly-once processing semantics while sacrificing as few resources as possible.

...