Flink 1.14 Burndown (blockers and critical bugs): https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=468
Test stability issues (all versions): https://issues.apache.org/jira/issues/?filter=12343317
Sync meeting
The sync meeting is happening every second Tuesday starting on the 8th of June at 9am CET / 4pm CST.
Everyone can join as a silent listener.
Timeline
Feature Freeze
Monday, 16th of August, end of business CEST.
(has been moved in the bi-weekly on the 20th of July)
Release
Mid September
Highlight features
- ML
- Table API (Sources, Sink, Schema Inference)
- Common source metrics (for new sources)
please feel free to add/suggest.
Features
List of features announced by contributors and committers that are likely to be ready for the feature freeze:n
NOTICE: It's preferred if only new features end up there and not all bugs/tasks separately, so that the page is not over bloated. Of course, unless fixing a bug is a really big or important one equivalent to implementing a completely new feature. A good rule of thumb would be that each entry in the page could (but does not have to) be later on included in a release blog post.
Legend:
symbol | meaning | comment |
---|---|---|
done done | well documented and cross-team tested | |
done | ||
will make it | ||
in danger | ||
very unlikely | ||
won't make it | ||
state unclear | ||
independent | as the artefact could be released independent of Apache Flink |
Summary:
17th of August | 31 | 0 | 1 | 0 | 5 | 0 | 1 | ∑38 | |
24th of August | 10 | 23 | 5 | ∑38 | |||||
31st of August | 19 | 14 | |||||||
7th of September | 27 | 6 |
As we are entering the stabilisation phase we also want to track documentation and testing efforts. As someone who is responsible for a feature please link the corresponding issues or/and PRs.
Runtime | ||||||
---|---|---|---|---|---|---|
Feature Name | Responsible | Note | State | Updated | Documentation | Cross-Team Testing |
MVP | 2021-07-21 | |||||
2021-07-29 | Not needed | |||||
2021-08-17 | Not needed (it is performance optimization) | Not needed | ||||
Unclear about merge with other efforts. | 2021-08-17 | Not needed | Not needed | |||
2021-07-21 | Documentation PR | Not needed | ||||
2021-08-13 | ||||||
2021-07-29 | Documentation PR | |||||
2021-07-21 | Settings documented | Not needed | ||||
We need to complete the RocksDB version bump up to keep pace to other projects, also see KAFKA-8897 | 2021-07-30 | Not needed | Not needed | |||
Won't make it | 2021-07-21 | |||||
Connectors | ||||||
Feature Name | Responsible | Note | State | Updated | Documentation | Cross-Team Testing |
Not in Flink 1.14 | 2021-07-21 | |||||
2021-08-16 | ||||||
2021-07-29 | ||||||
Needs JUnit 5 support | 2021-08-16 | Documentation PR | Not needed | |||
Arvid Heise | 2021-08-16 | |||||
Thomas Weise | 2021-08-16 | Tested during PR review | ||||
Yufan Sheng | 2021-08-18 | |||||
API | ||||||
Feature Name | Responsible | Note | State | Updated | Documentation | Cross-Team Testing |
2021-07-21 | Not needed | Not needed | ||||
SQL | ||||||
Feature Name | Responsible | Note | State | Updated | Documentation | Cross-Team Testing |
2021-07-29 | ||||||
MVP released in 1.13 | 2021-07-29 | |||||
"ChangeLog Upsert Disordering in Flink SQL" should be fixed. | 2021-07-21 | Not Needed | ||||
Caizhi Jingsong Lee | 2021-07-21 | Not Needed | Not Needed | |||
Partially finished in Release 1.13, finish the rest of the work | 2021-08-17 | |||||
Partially finished in Release 1.13, finish the rest of the work | 2021-07-21 | Not needed | Not needed | |||
Partially finished in Release 1.13, finish the rest of the work | 2021-07-21 | Not needed | Not needed | |||
2021-07-21 | Not needed | Not needed | ||||
FLIP-136 | 2021-07-29 | Documentation PR | Not needed | |||
2021-08-17 | Not needed | |||||
2021-07-29 | Not needed | |||||
2021-07-29 | ||||||
API/Python | ||||||
Feature Name | Responsible | Note | State | Updated | Documentation | Cross-Team Testing |
2021-07-21 | Not needed | Not needed (This is an optimization which should already be covered in the existing UT/IT cases) | ||||
2021-07-21 | Documentation PR | |||||
2021-08-16 | ||||||
2021-07-29 | Documentation PR | |||||
2021-07-29 | Documentation PR | |||||
2021-07-29 | Documentation PR | - FLINK-23822Getting issue details... STATUS | ||||
2021-07-29 | Documentation PR | |||||
2021-08-16 | ||||||
Machine Learning | ||||||
Feature Name | Responsible | Note | State | Updated | Documentation | Cross-Team Testing |
FLIP-173 has been proposed and needs some attention. Independent of Flink releases. | 2021-07-21 |
Status / Follow-ups
2021-09-07
No special items to discuss, asides from going through the Kanban board.
2021-08-31
- Branch `release-1.14` has been cut, and RC0 has been created.
- For PRs meant for 1.14, please merge to both master/release-1.14 branches, and set fix-versions to
both1.14.0/1.15.0. - For PRs merged recently (since last weekend), please double-check if they appear in all expected branches.
- For PRs meant for 1.14, please merge to both master/release-1.14 branches, and set fix-versions to
2021-08-24
- Docs & Tests
- Status check
- FLIP-150 Hybrid Source: Does it need cross-team testing? Thomas Weise Stephan Ewen
- FLINK-20731 Pulsar Source: No test ticket. Yufan Sheng Arvid Heise
- Please start progressing the documents early
- Get cross-team testing tickets assigned
- Release managers will try reaching out to people this week
- Can still pick-up voluntarily
- Status check
- Branch cutting time
2021-08-17
- Exceptions after feature freeze?
- Update wiki page
- Feature state
- Documentation
- Cross-Team Testing
- Documentation
- Blocks cross-team feature tests
- Cross-Team Testing
- Add description about how the feature should be tested
- Voluntarily pick-up for 1 week, look for assignees for unassigned tasks afterwards
- Build Stability
- Pushing towards stabilization
- Report issues with FixVersion 1.14.0 and proper Priority - everyone
- Try to find assignees for Blocker and Critical issues - release managers
- Blocker: immediately
- Critical: on syncs
- Keep monitoring the build stability - release managers
- Syncs
- Agenda
- Discussion items (if any)
- Go through all blockers (maybe not in-progress doc & test issues)
- Go through unassigned critical issues
- Schedule: once per week?
- Agenda
- Release branch & RC0
- How do we decide the branch is stabilized?
- Timeline based: 1 week?
2021-08-03
Release blockers (2)
Issue | Component | Assignee | Note |
---|---|---|---|
flink-docker | Downgrade to Critical | ||
Benchmarks | Cause founded, fixing |
Test instabilities (198)
- Despite the increment in number of tickets, the overall build stability is improving
- New issues are being taken care of
- Old issues are left open for observation
Most Frequent Unassigned Issue | Component | Responsibility | Note |
---|---|---|---|
Runtime / Task | Dawid Wysakowicz | Error stack shows some mailbox problem | |
API / DataStream | |||
API / Python | |||
Runtime / Coordination |
Discussion Items
- State changes
- FLINK-23007, FLINK-21091, FLINK-22911 to
- FLINK-22672, FLINK-22914, FLINK-19554 to
- FLIP-187, FLINK-21352, FLINK-14482 to
- FLINK-18578, FLINK-21634 to
- We should urge to push things either to or and heavily focus on
- Confirm feature freeze date
- Cross team tests
- Highlight features follow-up
2021-07-20
Release blockers (2)
Issue | Component | Assignee | Note |
---|---|---|---|
Build system | vote in progress |
Test instabilities (184)
Most Frequent Unassigned Issue | Component | Responsibility | Note |
---|---|---|---|
Kafka |
Discussion Items
- 2 weeks left before feature freeze
- do we want to postpone it? when do we want to have the actual freeze?
- Build Stability
- nightly runs started passing! or failing with 1/2 failures, which is a nice change
- What are the highlight features/improvements?
2021-07-06
Release blockers (2)
Issue | Component | Assignee | Note |
---|---|---|---|
Connectors/Kafka API/Table OperatorCoordinator | |||
Table SQL / Runtime | @Caizhi Weng |
Test instabilities (184)
Most Frequent Unassigned Issue | Component | Responsibility | Note |
---|---|---|---|
Infrastructure | PR in review | ||
Connectors / Kafka | |||
Deployment / YARN |
Discussion Items
- 4 weeks left before feature freeze
- Update status of the tickets
- 36 Features listed, 2 closed, 8 in progress, which makes it 26 open.
- Build Stability
- New instabilities emerging
- Some frequent instabilities are from old branches
- Criterion for closing outdated tickets
2021-06-22
Release blockers (6)
Issue | Component | Assignee | Note |
---|---|---|---|
Connectors/Kafka API/Table OperatorCoordinator | Qingsheng Ren | ||
Connectors/Hive | |||
Connectors/Hive DataStream/F27/Idleness | |||
Scala, Build system, avro glue schema registry | |||
Benchmarks | Probably fixed, need to gather a few more results | ||
API/Table | Shenkgai Feng |
Recent test instabilities (178)
Most Frequent Unassigned Issue | Component | Responsibility |
---|---|---|
Infrastructure | Johannes Moser | |
Connectors/Hive DataStream/F27/Idleness | ||
Connectors/Kafka | ||
Connectors/Kafka API/Table OperatorCoordinator | ||
Infrastructure |
Discussion Items
- Time Plan follow up (User ML thread, Dev ML thread)
- Build Stability follow up
- 174
- Criteria for merging PRs
- Strictly forbidden merging with "unrelated" CI failures?
2021-06-08
Release blockers (6)
Issue | Component | Assignee | Note |
---|---|---|---|
Runtime / Checkpointing | |||
API / State Processor | PR in review | ||
Connectors / Kafka | |||
Connectors / Hive | |||
Runtime / Checkpointing | PR in review | ||
Benchmarks |
Recent test instabilities (187)
Most Frequent Unassigned Issue | Component | Responsibility |
---|---|---|
Connectors / Kafka | ||
Connectors / Kafka | ||
Connectors / ElasticSearch |
Discussion Items
- Time Plan
- Build Stability
- Almost no success nightly build on master.
- Recent successes: 06-02, 04-15, 02-17
- Documentation
- Release advertising
7 Comments
Johannes Moser
Bi-weekly 8th of June
Time Plan:
Build stability:
Documentation & Release advertising
Further topcis
Johannes Moser
Discussed topics:
Release Blockers
Test instabilities
Time plan
Criteria for merging PRs
Dependencies
Johannes Moser
Bi-weekly 20th of July
2 weeks left before feature freeze
do we want to postpone it? when do we want to have the actual freeze?
Will be added to the Flink ML repo, isn't affected by the feature freeze.
Johannes Moser , tries to get someone to give feedback.
1 or 2 weeks would help, isn't required to delay the feature, hard to say how much this would increase the chance to get it into 1.14, anything else.
Team updates:
Build Stability
What are the highlight features/improvements?
Xintong Song
Bi-weekly 3rd of August
Features in danger
Feature freeze date
No objection against the previously decided feature freeze date: August 16th.
Cross-team test
Highlight feature
Stephan Ewen and Johannes Moser will discuss this offline.
Blockers
Build Stability
Johannes Moser
2021-08-17
Exceptions after feature freeze?
Update wiki page (Feature state, Documentation, Cross-Team Testing)
Documentation (Blocks cross-team feature tests)
Cross-Team Testing
Build Stability
Pushing towards stabilization
Syncs
Release branch & RC0
Johannes Moser
Docs & Tests
Status check
Xintong Song will check with Stephan Ewen
Arvid Heise will create a issue and link it.
Please start progressing the documentation tickets eary early
Branch cutting time
Blockers
Branch cutting time > RC0
Critical
Johannes Moser
Weekly Stabilisation Sync 31st of August
Blocker
Documentation
Cross-team testing
RC0 is ready