Page tree
Skip to end of metadata
Go to start of metadata

This page describes a proposed Flink Improvement Proposal (FLIP) process for proposing a major change to Flink.

To create your own FLIP, click on "Create" on the header and choose "FLIP-Template" other than "Blank page".

Purpose

The purpose of FLIPs is to have a central place to collect and document planned major enhancements to Apache Flink. While JIRA is still the tool to track tasks, bugs, and progress, the FLIPs give an accessible high level overview of the result of design discussions and proposals. Think of FLIPs as collections of major design documents for user-relevant changes.

We want to make Flink a core architectural component for users. We also support a large number of integrations with other tools, systems, and clients. Keeping this kind of usage healthy requires a high level of compatibility between releases — core architectural elements can't break compatibility or shift functionality from release to release. As a result each new major feature or public API has to be done in a way that we can stick with it going forward.

This means when making this kind of change we need to think through what we are doing as best we can prior to release. And as we go forward we need to stick to our decisions as much as possible. All technical decisions have pros and cons so it is important we capture the thought process that lead to a decision or design to avoid flip-flopping needlessly.

Hopefully we can make these proportional in effort to their magnitude — small changes should just need a couple brief paragraphs, whereas large changes need detailed design discussions.

This process also isn't meant to discourage incompatible changes — proposing an incompatible change is totally legitimate. Sometimes we will have made a mistake and the best path forward is a clean break that cleans things up and gives us a good foundation going forward. Rather this is intended to avoid accidentally introducing half thought-out interfaces and protocols that cause needless heartburn when changed. Likewise the definition of "compatible" is itself squishy: small details like which errors are thrown when are clearly part of the contract but may need to change in some circumstances, likewise performance isn't part of the public contract but dramatic changes may break use cases. So we just need to use good judgement about how big the impact of an incompatibility will be and how big the payoff is.

What is considered a "major change" that needs a FLIP?

Any of the following should be considered a major change:

  • Any major new feature, subsystem, or piece of functionality
  • Any change that impacts the public interfaces of the project

What are the "public interfaces" of the project?


All of the following are public interfaces that people build around:

  • DataStream and DataSet API, including classes related to that, such as StreamExecutionEnvironment
  • Classes marked with the @Public annotation
  • On-disk binary formats, such as checkpoints/savepoints
  • User-facing scripts/command-line tools, i.e. bin/flink, Yarn scripts, Mesos scripts
  • Configuration settings
  • Exposed monitoring information


Not all compatibility commitments are the same. We need to spend significantly more time on public APIs as these can break code for users. They cause people to rebuild code and lead to compatibility issues in large multi-dependency projects (which end up requiring multiple incompatible versions). Configuration, monitoring, and command line tools can be faster and looser — changes here will break monitoring dashboards and require a bit of care during upgrades but aren't a huge burden.

For the most part monitoring, command line tool changes, and configs are added with new features so these can be done with a single FLIP.

What should be included in a FLIP?

A FLIP should contain the following sections:

  • Motivation: describe the problem to be solved
  • Proposed Change: describe the new thing you want to do. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences, depending on the scope of the change.
  • New or Changed Public Interfaces: impact to any of the "compatibility commitments" described above. We want to call these out in particular so everyone thinks about them.
  • Migration Plan and Compatibility: if this feature requires additional support for a no-downtime upgrade describe how that will work
  • Rejected Alternatives: What are the other alternatives you considered and why are they worse? The goal of this section is to help people understand why this is the best solution now, and also to prevent churn in the future when old alternatives are reconsidered.

Who should initiate the FLIP?

Anyone can initiate a FLIP but you shouldn't do it unless you have an intention of getting the work done to implement it (otherwise it is silly).

Process

Here is the process for making a FLIP:

  1. Create a page which is a child of this one. Take the next available FLIP number (below) and give your proposal a descriptive heading. e.g. "FLIP 42: Enable Flink Streaming Jobs to stop gracefully". If you don't have the necessary permissions for creating a new page, please ask on the development mailing list.
  2. Fill in the sections as described above
  3. Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] FLIP-{your FLIP number} {your FLIP heading} The discussion should happen on the mailing list not on the wiki since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making.
  4. Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is consensus.
  5. Please update the FLIP wiki page, and the index below, to reflect the current stage of the FLIP after a vote. This acts as the permanent record indicating the result of the FLIP (e.g., Accepted or Rejected). Also report the result of the FLIP vote to the voting thread on the mailing list so the conclusion is clear.

FLIP round-up

Next FLIP Number: 150

Use this number as the identifier for your FLIP and increment this value.

Adopted/Accepted but unreleased FLIPs

FLIPTarget ReleaseLink to Discussion Thread
FLIP-16: Loop Fault ToleranceTBDhttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-13-Consistent-Processing-with-Loops-td14149.html
FLIP-17 Side Inputs for DataStream APITBDhttps://lists.apache.org/thread.html/e5fae9bb51ce37eb1d54fa4ecf9a01a663063cd06357c7b2f280ba5f@%3Cdev.flink.apache.org%3E
FLIP-23: Model Serving  TBDhttps://lists.apache.org/thread.html/968a633aee01463dc844d35ffa9a9c9e46b638158a8d32f1c4b4dea2@%3Cdev.flink.apache.org%3E
FLIP-27: Refactor Source Interface1.11https://lists.apache.org/thread.html/70484d6aa4b8e7121181ed8d5857a94bfb7d5a76334b9c8fcc59700c@%3Cdev.flink.apache.org%3E
FLIP-30: Unified Catalog APIsTBDhttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Integrate-Flink-SQL-well-with-Hive-ecosystem-tt24538.html
FLIP-32: Restructure flink-table for future contributionsTBDhttps://lists.apache.org/thread.html/13c605185f81958ef63c1090f72c82ba979f90f283406f90217dd5db@%3Cdev.flink.apache.org%3E

FLIP-35: Support Chinese Documents and Website

TBDhttps://lists.apache.org/thread.html/1d079e7218b295048c990edcb9891b5935d02eeac0927c89696aad42@%3Cdev.flink.apache.org%3E
FLIP-37: Rework of the Table API Type System1.10

https://lists.apache.org/thread.html/e1eca89862d59f78ba86c9a2596a810fdb1d42bcba7356fbda51e60e@%3Cdev.flink.apache.org%3E

FLIP-39: Flink ML pipeline and ML libs

1.10https://mail-archives.apache.org/mod_mbox/flink-dev/201904.mbox/%3CCAAjCYUWJs6kaf-n4rewm3vY2LJC_sgrr8ExexnR9-yV%3DOhxGjQ%40mail.gmail.com%3E
FLIP-41: Unify Canonical Binary Format for Keyed StateTBDhttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-41-Unify-Keyed-State-Snapshot-Binary-Format-for-Savepoints-td29197.html
FLIP-43: State Processing API1.9

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-42-Savepoint-Connector-td29231.html

FLIP-50: Spill-able Heap Keyed State Backend1.10https://lists.apache.org/thread.html/a10cae910f92936783e59505d7b9fe71c5f66ceea4c1c287c87164ae@%3Cdev.flink.apache.org%3E
FLIP-51: Rework of the Expression Design1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-51-Rework-of-the-Expression-Design-td31653.html
FLIP-55: Introduction of a Table API Java Expression DSL1.11https://lists.apache.org/thread.html/eb5e7b0579e5f1da1e9bf1ab4e4b86dba737946f0261d94d8c30521e@%3Cdev.flink.apache.org%3E
FLIP-56: Dynamic Slot Allocation1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-56-Dynamic-Slot-Allocation-td31960.html
FLIP-65: New type inference for Table API UDFs1.10https://lists.apache.org/thread.html/2b8cfd811a927d64a79f47387b7412c3b98a11fcb9358d3c23ef666c@%3Cdev.flink.apache.org%3E
FLIP-66: Support time attribute in SQL DDL1.10 for MVPhttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-66-Support-time-attribute-in-SQL-DDL-tt32766.html
FLIP 69 - Flink SQL DDL Enhancement1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-69-Flink-SQL-DDL-Enhancement-td33090.html
FLIP-76: Unaligned Checkpoints1.11 for MVPhttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-76-Unaligned-checkpoints-td33651.html
FLIP-79 Flink Function DDL Support1.10 for MVPhttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discussion-FLIP-79-Flink-Function-DDL-Support-td33965.html
FLIP-83: Flink End-to-end Performance Testing Framework1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-83-Flink-End-to-end-Performance-Testing-Framework-td34517.html

FLIP-90: Support SQL 2016-2017 JSON functions in Table API & SQL

1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html

FLIP-93: JDBC catalog and Postgres catalog

1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-93-JDBC-catalog-and-Postgres-catalog-td38208.html
FLIP-95: New TableSource and TableSink interfaces1.11https://lists.apache.org/thread.html/r03cbce8996fd06c9b0406c9ddc0d271bd456f943f313b9261fa061f9%40%3Cdev.flink.apache.org%3E
FLIP-98: Better Back Pressure Detection1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-98-Better-Back-Pressure-Detection-td37893.html

FLIP-99: Make Max Exception Configurable1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-99-Make-Max-Exception-Configurable-td37895.html

FLIP-100: Add Attempt Information1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-100-NEW-Add-Attempt-Information-td38453.html

FLIP-102: Add More Metrics to TaskManager1.12

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-102-Add-More-Metrics-to-TaskManager-td37898.html

FLIP-104: Add More Metrics to Jobmanager1.12

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-104-Add-More-Metrics-to-Jobmanager-td37901.html

FLIP-107: Handling of metadata in SQL connectors1.12

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-107-Reading-table-columns-from-different-parts-of-source-records-td38277.html

Vote thread: https://lists.apache.org/thread.html/r07ea23467f117d7ddbc8f9cbb2a1c99076abba74b2db6f0428bf9d21%40%3Cdev.flink.apache.org%3E

FLIP-128: Enhanced Fan Out for AWS Kinesis Consumers1.12

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-128-Enhanced-Fan-Out-for-AWS-Kinesis-Consumers-td42720.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-128-Enhanced-Fan-Out-for-AWS-Kinesis-Consumers-td42846.html

FLIP-129: Refactor Descriptor API1.12

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-129-Refactor-Descriptor-API-to-register-connector-in-Table-API-tp42995.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-129-Refactor-Descriptor-API-to-register-connector-in-Table-API-td43420.html

FLIP-130 Support Python DataStream API (Stateless part)1.12

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-130-Support-for-Python-DataStream-API-Stateless-Part-td43035.html

Vote thread:  http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-130-Support-for-Python-DataStream-API-Stateless-Part-tt43424.html

FLIP-132 Temporal Table DDL and Temporal Table Join1.12

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-132-Temporal-Table-DDL-tc43483.html

Vote thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-132-Temporal-Table-DDL-tc43632.html#a44091

FLIP-133: Rework PyFlink Documentation

1.11

Discussion thread: https://lists.apache.org/thread.html/redebc9d1281edaa4a1fbf0d8c76a69fcff574b0496e78519840a5a61%40%3Cdev.flink.apache.org%3E

Vote thread: https://lists.apache.org/thread.html/rf21ef9bd45c675b836370165a21feaa5b55e9c93ea94c50eb8e3aae7%40%3Cdev.flink.apache.org%3E

FLIP-137: Support Pandas UDAF in PyFlink1.12

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-137-Support-Pandas-UDAF-in-PyFlink-tt44060.html

Vote thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-137-Support-Pandas-UDAF-in-PyFlink-tc44384.html

FLIP-139: General Python User-Defined Aggregate Function Support on Table API1.12

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-139-General-Python-User-Defined-Aggregate-Function-on-Table-API-td44139.html

Vote thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-139-General-Python-User-Defined-Aggregate-Function-on-Table-API-td44421.html

FLIP-141: Intra-Slot Managed Memory Sharing1.12http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-141-Intra-Slot-Managed-Memory-Sharing-td44146.html
FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)1.12

DISCUSS: https://lists.apache.org/thread.html/r4f24c4312cef7270a1349c39b89fb1184c84065944b43aedf9cfba6a%40%3Cdev.flink.apache.org%3E

VOTE: https://lists.apache.org/thread.html/r09c17db5599b01faba33c2fa112ef0961d72d78788426b7302db4f59%40%3Cdev.flink.apache.org%3E

FLIP-134: Batch execution for the DataStream API1.12

DISCUSS: https://lists.apache.org/thread.html/reb368f095ec13638b95cd5d885a0aa8e69af06d6e982a5f045f50022%40%3Cdev.flink.apache.org%3E

VOTE: https://lists.apache.org/thread.html/r6ffdbe90e98542f61e610e5f5abdba0f7a5acb5f98cd01baa1fc17d0%40%3Cdev.flink.apache.org%3E

FLIP-138: Declarative Resource management1.12

Discuss thread: https://lists.apache.org/thread.html/r7e4edf06d40369c717d74da1485435279a52b4491db6f61d35892d21%40%3Cdev.flink.apache.org%3E

Vote thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-138-Declarative-Resource-Management-tp44375.html

FLIP-142: Disentangle StateBackends from Checkpointing1.12

Discuss thread:

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-142-Disentangle-StateBackends-from-Checkpointing-td44496.html

Vote thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-142-Disentangle-StateBackends-from-Checkpointing-td44679.html

FLIP-144: Native Kubernetes HA for Flink1.12

DISCUSS: https://lists.apache.org/thread.html/r466ad059dda1276fba4fa9a710cbfdfeab6b8a24c4047c6ed5d619e8%40%3Cdev.flink.apache.org%3E

VOTE: https://lists.apache.org/thread.html/r816a1c557fc197769e2974950d13605406d9fce59a4dce51562b8b33%40%3Cdev.flink.apache.org%3E

FLIP-143: Unified Sink API1.12

discuss: https://lists.apache.org/thread.html/rf09dfeeaf35da5ee98afe559b5a6e955c9f03ade0262727f6b5c4c1e%40%3Cdev.flink.apache.org%3E

vote: https://lists.apache.org/thread.html/r0f71d53810c7970430e5952ec69aa59ac54353de6233844421f753d6%40%3Cdev.flink.apache.org%3E

FLIP-145: Support SQL windowing table-valued function1.12

DISCUSS: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-145-Support-SQL-windowing-table-valued-function-td45269.html

VOTE: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-145-Support-SQL-windowing-table-valued-function-td45571.html

FLIP-149: Introduce the upsert-kafka Connector1.12

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-149-Introduce-the-KTable-Connector-td45813.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-149-Introduce-the-upsert-kafka-connector-td45953.html

FLIPs under discussion

FLIPStateLink to Discussion Thread
FLIP-5: Only send data to each taskmanager once for broadcastsDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-5-Only-send-data-to-each-taskmanager-once-for-broadcasts-td12677.html
FLIP-14: crossGroup OperatorDiscuss
FLIP-15: Redesign Iterations (Scoping, Flow Control and Termination)Discusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-13-Consistent-Processing-with-Loops-td14149.html
FLIP-18: Code Generation for improving sorting performanceDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-18-Code-Generation-for-improving-sorting-performance-tc16486.html
FLIP-21: Improve object Copying for Streaming RuntimeDiscusshttps://lists.apache.org/thread.html/d1cd94a97eaba49cb05d8891b133374cd59021d6c5f04e9a19f0a6df@%3Cdev.flink.apache.org%3E
FLIP-22: Eager State DeclarationDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-22-Eager-State-Declaration-td18562.html
FLIP-26: Service AuthorizationDiscuss

FLIP-33: Standardize Connector Metrics

Discusshttps://lists.apache.org/thread.html/65078bad6e04bbbb7578d502e1e5d92026f13fd9648725f5b74ed330@%3Cdev.flink.apache.org%3E
FLIP-36: Support Interactive Programming in FlinkDiscusshttps://lists.apache.org/thread.html/5f4961f1dfe23204631fd6f2b3227724ce9831f462737f51742a52c1@%3Cdev.flink.apache.org%3E
FLIP-40: Flink DriverDiscusshttps://lists.apache.org/thread.html/f5dd0920ffd1361513c1d73352788086773595777cae9d70713fc1ac@%3Cdev.flink.apache.org%3E
FLIP-44: Support Local Aggregation in FlinkDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-44-Support-Local-Aggregation-in-Flink-td29513.html
FLIP-45: Reinforce Job Stop SemanticDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-45-Reinforce-Job-Stop-Semantic-td30161.html
FLIP-46: Graceful Shutdown Handling by UDFsDiscusshttps://lists.apache.org/x/thread.html/dd3b941621645bd78e119c44667c18cfef23f89b8721eeb9f8926fc6@%3Cdev.flink.apache.org%3E
FLIP-47: Checkpoints vs. SavepointsDiscuss https://lists.apache.org/x/thread.html/dc5ab1fad1a38b58aa762303922c04f1310d49dbd3c3ac8264cbe27f@%3Cdev.flink.apache.org%3E
FLIP-54: Evolve ConfigOption and ConfigurationDiscusshttps://lists.apache.org/thread.html/a56c6b52e5f828d4a737602b031e36b5dd6eaa97557306696a8063a9@%3Cdev.flink.apache.org%3E
FLIP-60: Restructure the Table API & SQL documentationDiscusshttps://lists.apache.org/thread.html/0480d17dad32c2df62b3d401385f2140e221b42ee696494a14f73dc5@%3Cdev.flink.apache.org%3E
FLIP-72: Introduce Pulsar ConnectorDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-72-Introduce-Pulsar-Connector-tc33283.html
FLIP-75: Flink Web UI Improvement ProposalDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
FLIP-79: Flink Function DDL SupportDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discussion-FLIP-79-Flink-Function-DDL-Support-td33965.html
FLIP-80: Expression String Serializable and DeserializableDiscusshttp://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISUCSS-FLIP-80-Expression-String-Serializable-and-Deserializable-tt34146.html
FLIP-89: Improve usability of TableConfigDiscuss
FLIP-91: Support SQL Client GatewayDiscusshttp://mail-archives.apache.org/mod_mbox/flink-dev/202001.mbox/%3CCADQYLGsCGDJkfd3L1hAy1y_M2625YkNHJGW82UraGLhzg6p7Ug%40mail.gmail.com%3E
FLIP-100: Add Attempt InformationDiscuss/Vote

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-100-Add-Attempt-Information-td37896.html

FLIP-101: Add Pending Slots DetailDiscuss/Vote

discuss: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

vote: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-101-Add-Pending-Slots-Detail-td37897.html

FLIP-136: Improve interoperability between DataStream and Table API

Discuss

https://lists.apache.org/thread.html/r62b47ec6812ddbafed65ac79e31ca0305099893559f1e5a991dee550%40%3Cdev.flink.apache.org%3E


FLIP-147: Support Checkpoints After Tasks FinishedDiscussdiscuss: https://lists.apache.org/thread.html/r2780b46267af6e98c7427cb98b36de8218f1499ae098044e1f24c527%40%3Cdev.flink.apache.org%3E


Implemented and Released FLIPs

FLIPFirst Release VersionLink to Discussion Thread
FLIP-1: Fine Grained Recovery from Task Failures1.9http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-1-Fine-grained-recovery-from-task-failures-td12510.html
FLIP-2: Extending Window Function Metadata1.3http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-2-Extending-Window-Function-Metadata-td12522.html
FLIP-3: Improving Organization of Documentation1.2http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-3-Organization-of-Documentation-td12560.html
FLIP-4: Enhance Window Evictor1.2http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Enhance-Window-Evictor-in-Flink-td12406.html
1.5

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-6-Flink-Deployment-and-Process-Model-Standalone-Yarn-Mesos-Kubernetes-etc-td12685.html

FLIP-7: Expose metrics to WebInterface1.2
FLIP-8: Rescalable Non-Partitioned State1.2

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-8-Rescalable-Non-Partitioned-State-tt12999.html

FLIP-10: Unify Checkpoints and Savepoints1.2http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-10-Unify-Savepoints-and-Checkpoints-tt13149.html

FLIP-11: Table API Stream Aggregations

1.3http://mail-archives.apache.org/mod_mbox/flink-dev/201609.mbox/%3C4f43ebc4-7c5d-711a-5d9f-3e179bdcd33c%40apache.org%3E
FLIP-12: Asynchronous I/O Design and Implementation1.2
FLIP-13: SideOutputs in Flink1.3http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discuss-FLIP-13-Side-Outputs-in-Flink-td14204.html

FLIP-19: Improved BLOB storage architecture

1.4http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-19-Improved-BLOB-storage-architecture-td18092.html
FLIP-20: Integration of SQL and CEP1.7

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Integrating-Flink-Table-API-amp-SQL-with-CEP-td17964.html

FLIP-24: SQL Client1.5http://mail-archives.apache.org/mod_mbox/flink-dev/201712.mbox/%3C8a9d718b-5dae-0fe2-1da6-a8d557d45582%40apache.org%3E
FLIP-25: TTL for State1.6https://lists.apache.org/thread.html/05d187544229d3cbb46270a9e1f09001e02904b8d3e6dea71593a18f@%3Cdev.flink.apache.org%3E

FLIP-29: Support map/flatMap/aggregate/flatAggregate on TableAPI

1.9http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Table-API-Enhancement-Outline-td25070.html
FLIP-31: Pluggable Shuffle Manager1.9https://lists.apache.org/thread.html/73d7c62a6c6dc7d3fddda79701c71b921e635766737c1b33b84df234@%3Cdev.flink.apache.org%3E
FLIP-34: Terminate/Suspend Job with Savepoint1.9https://lists.apache.org/thread.html/b8d2f3209e7ca7467af6037383ade6c14c35276f7acb2bbbc9a50c0f@%3Cdev.flink.apache.org%3E
FLIP-38: Python Table API1.9https://lists.apache.org/thread.html/08aad344e43a409c318d4bda67bac1fa7beecc6abd8539c53021d310@%3Cdev.flink.apache.org%3E
FLIP-49: Unified Memory Configuration for TaskExecutors1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-49-Unified-Memory-Configuration-for-TaskExecutors-td31436.html
FLIP-52: Remove legacy Program interface.1.10 https://lists.apache.org/thread.html/0dbd0a4adf9ad00d6ad869dffc8820f6ce4c1969e1ea4aafb1dd0aa4@%3Cdev.flink.apache.org%3E
FLIP-53: Fine Grained Operator Resource Management1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-53-Fine-Grained-Resource-Management-td31831.html
FLIP-57: Rework FunctionCatalog1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-57-Rework-FunctionCatalog-td32291.html#a32613

FLIP-58: Flink Python User-Defined Stateless Function for Table

1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Python-User-Defined-Function-for-Table-API-td31673.html
FLIP-59: Enable execution configuration from Configuration object1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-59-Enable-execution-configuration-from-Configuration-object-td32359.html
FLIP-61: Simplify Flink's cluster level RestartStrategy configuration1.10https://lists.apache.org/thread.html/e206390127bcbd9b24d9c41a838faa75157e468e01552ad241e3e24b@%3Cdev.flink.apache.org%3E
FLIP-62: Set default restart delay for FixedDelay- and FailureRateRestartStrategy to 1s1.10https://lists.apache.org/thread.html/9602b342602a0181fcb618581f3b12e692ed2fad98c59fd6c1caeabd@%3Cdev.flink.apache.org%3E
FLIP-63: Rework table partition support1.10

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-63-Rework-table-partition-support-td32770.html

FLIP-68: Extend Core Table System with Pluggable Modules1.10https://www.mail-archive.com/dev@flink.apache.org/msg29894.html
FLIP-73: Introducing Executors for job submission1.10

 https://lists.apache.org/thread.html/dc3a541709f96906b43df4155373af1cd09e08c3f105b0bd0ba3fca2@%3Cdev.flink.apache.org%3E

FLIP-74: Flink JobClient API1.10https://lists.apache.org/x/thread.html/b2e22a45aeb94a8d06b50c4de078f7b23d9ff08b8226918a1a903768@%3Cdev.flink.apache.org%3E
FLIP-77: Introduce ConfigOptions with Data Types1.10https://lists.apache.org/thread.html/6f98eae7879764aa3a3991311d700a5ccdb713dbe345e6ecf514b2d7@%3Cdev.flink.apache.org%3E
FLIP-78: Flink Python UDF Environment and Dependency Management1.10https://lists.apache.org/x/thread.html/3441424fed2af339b8d83c6a8aeecbb4edc23f7859ed1ccf42b389e0@%3Cdev.flink.apache.org%3E
FLIP-73: Introducing Executors for job submission1.10

 https://lists.apache.org/thread.html/dc3a541709f96906b43df4155373af1cd09e08c3f105b0bd0ba3fca2@%3Cdev.flink.apache.org%3E

FLIP-81: Executor-related new ConfigOptions.1.10https://lists.apache.org/thread.html/a4dd8e0c7b79350c59f5afefc1bc583dac99abcf94caaa8c22017974@%3Cdev.flink.apache.org%3E
FLIP-86: Improve Connector Properties1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-86-Improve-Connector-Properties-td34922.html
FLIP 87: Primary key constraints in Table API1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Primary-keys-in-Table-API-td35138.html
FLIP-88: PyFlink User-Defined Function Resource Management1.10http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
FLIP-64: Support for Temporary Objects in Table module1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-64-Support-for-Temporary-Objects-in-Table-module-td32684.html
FLIP-67: Cluster partitions lifecycle1.11

https://mail-archives.apache.org/mod_mbox/flink-dev/201909.mbox/%3C58db5a13-2b27-e853-bbf1-ccbf404abc7e%40apache.org%3E

FLIP-84: Improve & Refactor API of TableEnvironment1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-84-Improve-amp-Refactor-API-of-Table-Module-td34537.html#a37786

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-84-Feedback-Summary-td39261.html

FLIP-85: Flink Application Mode1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-85-Delayed-Job-Graph-Generation-td35759.html#a35767

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-85-Flink-Application-Mode-td38551.html

FLIP-92: Add N-Ary Stream Operator in Flink1.11

discussion http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Add-N-Ary-Stream-Operator-td11341.html

vote http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-92-Add-N-Ary-Stream-Operator-in-Flink-td36539.html

FLIP-96: Support Python ML Pipeline API1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Python-ML-Pipeline-API-td37291.html
FLIP-97: Support Scalar Vectorized Python UDF in PyFlink1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-scalar-vectorized-Python-UDF-in-PyFlink-tt37264.html
FLIP-103: Better TM/JM Log Display1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-103-Better-TM-JM-Log-Display-tp37899.html

FLIP-105: Support to Interpret Changelog in Flink SQL (Introducing Debezium and Canal Format)1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-105-Support-to-Interpret-and-Emit-Changelog-in-Flink-SQL-td37665.html
FLIP-106: Support Python UDF in SQL Function DDL1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-106-Support-Python-UDF-in-SQL-Function-DDL-td38107.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-106-Support-Python-UDF-in-SQL-Function-DDL-td38895.html

FLIP-108: Add GPU support in Flink1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-108-Add-GPU-support-in-Flink-td38286.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-108-Add-GPU-support-in-Flink-td39655.html

FLIP-110: Support LIKE clause in CREATE TABLE1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-110-Support-LIKE-clause-in-CREATE-TABLE-td38378.html
FLIP-111: Docker image unification1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-111-Docker-image-unification-td38444.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-111-Docker-image-unification-td39952.html

FLIP-114: Support Python UDF in SQL Client1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-114-Support-Python-UDF-in-SQL-Client-td38655.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-114-Support-Python-UDF-in-SQL-Client-tp39526.html

FLIP-116: Unified Memory Configuration for Job Managers1.11

http://mail-archives.apache.org/mod_mbox/flink-dev/202003.mbox/%3C4E1A4410-462C-463E-9C9E-24483EA5D4A5%40gmail.com%3E

http://mail-archives.apache.org/mod_mbox/flink-dev/202003.mbox/%3CCAJNyZN7B39y08ukH9xYvNgHhrSX2dCWOSC9GEAqGt%3DfONBXPMw%40mail.gmail.com%3E

FLIP-120: Support conversion between PyFlink Table and Pandas DataFrame1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-120-Support-conversion-between-PyFlink-Table-and-Pandas-DataFrame-tt39611.html

FLIP-121: Support Cython Optimizing Python User Defined Function1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-121-Support-Cython-Optimizing-Python-User-Defined-Function-tt39577.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-121-Support-Cython-Optimizing-Python-User-Defined-Function-tt39979.html

FLIP-122: New Connector Property Keys for New Factory1.11http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-122-New-Connector-Property-Keys-for-New-Factory-td39462.html
FLIP-123: DDL and DML compatibility for Hive connector1.11

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-123-DDL-and-DML-compatibility-for-Hive-connector-td39633.html

Vote thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-123-DDL-and-DML-compatibility-for-Hive-connector-tc40183.html

FLIP-124: Add open/close and Collector to (De)SerializationSchema

1.11

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-124-Add-open-close-and-Collector-to-De-SerializationSchema-td39864.html

FLIP-126: Unify (and separate) Watermark Assigners1.11

DISCUSS: https://lists.apache.org/thread.html/r7988ddfe5ca8d85e666039cf6240e1007a2ca337a52108f684b66d90%40%3Cdev.flink.apache.org%3E

VOTE: https://lists.apache.org/thread.html/r104a5b0febe90b9e828938f25034ef71c962a803a0e20cbb85086064%40%3Cdev.flink.apache.org%3E

FLIP-118 Improve Flink’s ID system1.12

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-118-Improve-Flink-s-ID-system-td39321.html

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-118-Improve-Flink-s-ID-system-td40320.html

FLIP-140: Introduce batch-style execution for bounded keyed streams1.12

Discussion thread: https://lists.apache.org/thread.html/rf88c0ceddf419ba14760a47ae87945d33387abbd03ccbea5f35b8ed4%40%3Cdev.flink.apache.org%3E

Vote thread: https://lists.apache.org/thread.html/r29e718f0922278e1c896cec83a581c95c813ac21b3f20a504794ff5f%40%3Cdev.flink.apache.org%3E

FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)1.12

DISCUSS: https://lists.apache.org/thread.html/r4f24c4312cef7270a1349c39b89fb1184c84065944b43aedf9cfba6a%40%3Cdev.flink.apache.org%3E

VOTE: https://lists.apache.org/thread.html/r09c17db5599b01faba33c2fa112ef0961d72d78788426b7302db4f59%40%3Cdev.flink.apache.org%3E

Discarded FLIPs

FLIPComment
FLIP-9: Trigger DSLDecided to not work on that, at least at the moment.

FLIP-28: Long-term goal of making flink-table Scala-free

This FLIP is part of the bigger vision described in FLIP-32.

  • No labels