Table of contents

Jira Boards

Flink 1.20 Burndown: 1.20 Burndown - Agile Board - ASF JIRA (apache.org)

Sync meeting

  • The release sync will start from April 9, 2024, at 9.30am (UTC+2) and 3.30pm (UTC+8). The release sync happens bi-weekly at first, and adjusted to weekly from June 4, 2024, as we approaching the feature freeze date. 
  • Feel free to join on Google Meetand attendees are encouraged to fill out the topics to be discussed at the bottom of this page a day in advance to make it easier for everyone to understand the background of the topic. 

Timeline

  • Feature Freeze
    • June 15, 2024, 00:00 CEST(UTC+2) 
  • Release
    • End of July 2024


Highlight features

please feel free to add/suggest.

Features

List of features announced by contributors and committers that are likely to be ready for the feature freeze:n

NOTICE: It's preferred if only new features end up there and not all bugs/tasks separately, so that the page is not over bloated. Of course, unless fixing a bug is a really big or important one equivalent to implementing a completely new feature. A good rule of thumb would be that each entry in the page could (but does not have to) be later on included in a release blog post.

Legend

State

symbolmeaningcomment
(big grin)validatedthrough cross team testing
(tick)donewell documented with a complete test coverage
(green star)will make itthere is no reason this effort should not go into 1.20
(star)in dangerthere are some concerns the effort could be ready for the feature freeze of 1.20
(red star)very unlikelythere are severe concerns the effort could make it to 1.20
(minus)won't make itit was decided against adding this for the 1.20 release. working on the effort has been stopped.
(question)state unclear
(blue star)independentas the artifact could be released independent of Apache Flink

X-Team verification

symbolmeaning
(tick)done
(blue star)not required

Feature Stage

Please align with the list on the Apache Flink Roadmap (https://flink.apache.org/roadmap.html).

  • MVP: Have a look, consider whether this can help you in the future.
  • Beta: You can benefit from this, but you should carefully evaluate the feature.
  • Ready and Evolving: Ready to use in production, but be aware you may need to make some adjustments to your application and setup in the future, when you upgrade Flink.
  • Stable: Unrestricted use in production
  • Reaching End-of-Life: Stable, still feel free to use, but think about alternatives. Not a good match for new long-lived projects.
  • Deprecated: Start looking for alternatives now 

Summary

Numbers are based on the items in the list below, not on the tickets 


(big grin)(tick) (green star)(star)(red star)(minus)(question)(blue star)Remaining weeks
2024-04-23

51



68
2024-05-07

62



86
2024-05-21
172



104
2024-06-04
38
2


132
2024-06-11
68

2

161
2024-06-18
13


3

160

List

Feel free to add categories.

Runtime
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified
Shuffle









Yuxin Tan 

(tick)
2024-06-12(tick)(blue star)(tick)











Coordination









Junrui Lee Zhu Zhu 

(tick)
2024-05-31(tick)(tick)(tick)











State & Checkpoint











(tick)
2024-06-18(tick)(tick)(tick)

FLINK-31238 - Getting issue details... STATUS



(minus)
2024-06-04




(minus)
2024-06-04


FLINK-26050 - Getting issue details... STATUS



(tick)99%2024-06-18(tick)To be updated(tick)

REST

































Web UI











RocMarshal 

(tick)
2024-05-16(tick)(blue star)(tick)











SQL
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified
Jane Chan 

(tick)
2024-06-18(tick)(tick)(tick)

FLIP-376: Add DISTRIBUTED BY clause

FLINK-33494 - Getting issue details... STATUS

Timo Walther 

(tick)
2024-06-18(tick)(tick)(blue star)

ron liu@Feng Jin


MVP for 1.20 completed, docs are preparing(tick)95%2024-06-11(tick)(tick)(tick)

ron liu @Feng Jin



(tick)100%2024-06-11(tick)(tick)(tick)
Connectors









Sergey Nuyanzin 

(tick)100%2024-06-13(tick)(blue star)(blue star)
Xia Sun Zhu Zhu 

(tick)
2024-05-22(tick)(blue star)(tick)


(tick)
2024-05-26(tick)(tick)(tick)
Resource Management































Runtime

FLIP-398: Improve Serialization Configuration And Usage In Flink

Add built-in Map/List/Set serializers

FLINK-34037 - Getting issue details... STATUS

Zhanghao Chen 

(minus)
2024-06-12


FLIP-443: Interruptible timers firing

FLINK-20217 - Getting issue details... STATUS



(tick)
2024-06-14(tick)(blue star)(blue star)











Status / Follow-ups

04/09/24

  • Feature freeze date – June 15 all set?
    • Discussed on ML and we are all aligned on the date 👍🏼
    • Robert Metzger feature list looks pretty light currently
    • TODOUfuk Celebi to ping Confluent teams to add features to Wiki
  • Flink 1.19 retro
    • TODO Ufuk Celebi to reach out to release managers of Flink 1.19 and ask for feedback
    • TODO Robert Metzger to ask about mailing list monitoring
  • Daily work divisions
    • JIRA ticket (Mainly focus on new blockers, progress of existing blockers each week)
    • CI (Failures reported in #build channel and not reported)
    • Benchmark (Check if there's regression in the #flink-dev-benchmarks channel)
    • Mailing list monitoring
  • Blockers of 1.20
    • FLINK-35040 - Getting issue details... STATUS
      • It seems Java 11 regresses, and Java 8 and Java 17 are fine.
      • Robert Metzger Any relevant change recently? 
      • Rui Fan applied for a Jenkins account to check for a JVM version upgrade.
    • FLINK-35009 - Getting issue details... STATUS
      • Since Transformation  is an Internal API indeed, and this only breaks the kafka-connector test code. I tend to this should not be a blocker at all.
    • FLINK-34997 - Getting issue details... STATUS
      • This one is an unstable test case, seems to be that docker-compose binaries are sometimes missing in the Azure VMs. 


04/23/24

  • Blockers of 1.20
    •   FLINK-35040 - Getting issue details... STATUS
      • It seems the performance regresses after FLINK-34955 - Getting issue details... STATUS is merged.
      • The performance will be recovered after reverting common-ios version.
      • 2 concerns:
        • I didn't find any common-ios code cost the time during run the benchmark on my Local. (async-profiler wall mode)
        • FLINK-34955 - Getting issue details... STATUS seems necessary and fixes some bugs.
      • TODO: Try generate the flame graph on benchmark server Rui Fan 
    • FLINK-35041 - Getting issue details... STATUS
      • I can't reproduce this by running IncrementalRemoteKeyedStateHandleTest itself, but I can reliably reproduce it (within about a minute) by running the entire org.apache.flink.runtime.state package in the mode "Repeat until stopped". (Comments from Ryan)
      • I have reached out Feifan Wang  to take a look.
    • FLINK-35009 - Getting issue details... STATUS
      • Can anyone reach out Martijn Visser and ask him about this? I'm inclined to think we can downgrade it.
    • FLINK-34997 - Getting issue details... STATUS
    • FLINK-35215 - Getting issue details... STATUS
      • It seems FLINK-34954 - Getting issue details... STATUS causes it.


05/07/24

  • Blockers of 1.20
    •   FLINK-35040 - Getting issue details... STATUS
      • TODO: Try generate the flame graph on benchmark server Rui Fan 
    • FLINK-35041 - Getting issue details... STATUS
      • I can't reproduce this by running IncrementalRemoteKeyedStateHandleTest itself, but I can reliably reproduce it (within about a minute) by running the entire org.apache.flink.runtime.state package in the mode "Repeat until stopped". (Comments from Ryan)
      • I have reached out Feifan Wang  to take a look.
      • Rui Fan reach out Feifan Wang 
    • FLINK-35215 - Getting issue details... STATUS
  • Rui Fan to send a reminder about:
    • The feature freeze date in 6 weeks
    • Check whether the Public or Public Evolving api related to flink 2.0 deprecation has been done


05/21/24

  • Blockers of 1.20
    •   FLINK-35040 - Getting issue details... STATUS
      • Generated the flame graph on benchmark server Zakelly Lan  Rui Fan 
      • But we didn't see any common-ios package code is called. And it doesn't influence other dependency.
      • Update it to dev mail list to collect some valuable suggestion.
      • We will accept this regression if we don't have any suggestion.
        • It only happens on benchmark server with jdk11
    • FLINK-35041 - Getting issue details... STATUS
      • it's solved.
    • FLINK-35215 - Getting issue details... STATUS

06/04/24

  • Blockers of 1.20
    • closed since the last sync: 
      • FLINK-35040 - Getting issue details... STATUS
        • We accepted this regression
      • FLINK-35215 - Getting issue details... STATUS
        • The performance has been recovered.
    • unclosed:
      • FLINK-35423 - Getting issue details... STATUS
        • Have reached out Sergey for updating the progress.
  • The development progress
    • Only left 2 weeks for feature freeze, do we need to remind developers in dev SUMMARY mail list?
    • And ask developers whether we need to extend it because of some awesome feature?
    • Update the percentage of progress.
  • CI pipeline triggered by pull request seems sort of unstable. https://github.com/apache/flink/pull/24883
  • According to the last discussion, this call will be scheduled once a week.

06/11/24

  • No blockers of 1.20
  • June 15 is the planned feature freeze day, total 16, done 8, will make it 6, very unlikely 2;
    • There are only three days left to feature freeze
    • We need to check all the features one by one
    • Highlight in the SUMMARY mail that it is very close to the feature freeze date.
    • Whether to delay feature freeze day
  • When to cut release-1.20 branch? 
    • 1~2 weeks after feature freeze.
  • Start cross team release testing after feature freeze Weijie Guo 

06/18/24

  • Blockers:
    • FLINK-35629 - Getting issue details... STATUS
      • It just happened in the past few days, not sure it's caused by flink side or benchmark side?
    • FLINK-35587 - Getting issue details... STATUS
      • 10 TB TPC-DS blocking shuffle sometimes raise due to unexpected credit. Maybe we should upgrade it to blocker.
  • Feature Freeze
    • We have 12 completed feature before feature freeze, and 3 features were moved to the next release.
    • Zakelly Lan want to merge  FLINK-32086 - Getting issue details... STATUS and replied in the feature freeze thread. Any objections?
    • Matthias Pohl wants to merge non-crucial (but rather cosmetic) FLINK-35539 - Getting issue details... STATUS proposed by zhen wang . Any objections?
  • Release Testing
    • Weijie Guo and Rui Fan created all cross-team testing instructions issue: FLINK-35602 - Getting issue details... STATUS
      • We should remind feature contributors to complete the documentation and testing instruction.
    • We'll start release testing after release-1.20 branch cut.
  • Cut Branch
    • Cut 1.20 branch this Friday?
    • If we have unresolved blockers, should it be delayed?
  • Release Note
    • We should check all related jira/ticket to see if it need a release note. In particular, the deprecation stuff for 2.0.
    • Would anyone like to help with this? → Ufuk Celebi  to take a look
  • Release Sync Schedule
    • We decide to change the schedule of release sync to 9.30am (UTC+2) and 3.30pm (UTC+8).

06/25/24

  • Blockers
    • Congrats, no known blocker for now.
  • Cut Branch
  • Cross-team release testing FLINK-34285 - [Umbrella] Test Flink Release 1.20  
    • total 13 flip/features, 8 confirmed,  5 wait for response
    • testing instructions ready: 8  (1 assigned)
  • RC0
    • Any suggestions for the time, maybe this Friday or next Monday?

07/02/24

  • Blockers
    • Congrats, no known blocker for now.
  • Release Candidate #0
    • It was announced last Friday and receive no feedback.
  • Documentation
    • We should check all documentation for the features listed in this wiki page and mark it as `Documented`.
  • Cross-team release testing FLINK-34285 - [Umbrella] Test Flink Release 1.20 Rui Fan  
    • total 13 flip/features, all of them are confirmed
    • 10 testing instructions ready, 8 assigned, 2 wait for volunteer
    • Weijie Guo  plans to prepare RC1 the Monday after next, so it is better to finish all the testing work by the end of next week.
  • Release Note Ufuk Celebi 
    • We should go through all features, because some feature don't have FLIP.
    • We'd better complete all the release notes by the end of this week.
  • RC1
    • Weijie Guo  plan to vote on RC1 the Monday after next

07/09/24

  • Blockers
    • FLINK-35784 - Getting issue details... STATUS Founded via release testing(good catch).
      • Zakelly has filed a PR to fix this, it will be merged after CI green.
  • Release Candidate #0
    • There are no known issues for now.
    • Thanks to the people from Confluent for helping with the validation!
  • Documentation
    • Weijie Guo has checked all documentation for the features listed in this wiki page and updated the status.
  • Cross-team release testing FLINK-34285 - [Umbrella] Test Flink Release 1.20 Rui Fan  
    • We only left two testing to be done:
    • We expect finish all the testing work by the end of this week.
  • Release Note Weijie Guo Rui  
    • We have gone through all features, because some feature don't have FLIP.
    • All release notes have been added now and Weijie Guo will draft a release note documentation(i.e. flink-1.20.md) for the 1.20 release.
  • RC1
    • Weijie Guo plan to vote on RC1 next Monday if we don't have any new blocker.

07/16/24

  • Blockers
    • Congrats, no known blocker for now.
    • Known issue: Javadocs are broken. Working on it: FLINK-35154 - Getting issue details... STATUS
  • Cross-team release testing
    • All done!
  • RC1
    • Weijie Guo has started the voting on RC1 in the dev mail list.
    • Encourage PMCs to test and vote.
  • Release
    • Once this candidate is accepted, Weijie Guo  will do the rest of the work as soon as possible. If necessary, we can talk about the subsequent work divisions in the next meeting.


07/23/24

  • Blockers
    • Congrats, no known blocker for now.
    • FLINK-35872 - Getting issue details... STATUS is solved
    • Known issue:
      • Javadocs are broken. Working on it: FLINK-35154 - Getting issue details... STATUS
        • We have updated the `maven-javadoc-plugin` to 3.7.0 and Samrat verified it locally.
        • But the Documentation Build CI Pipeline is broken:    FLINK-35863 - Getting issue details... STATUS (It's not clear that this has anything to do with plugin updating)
  • RC1 is canceled : FLINK-35872 - Getting issue details... STATUS  
  • Create RC2 recently Weijie Guo 
  • Release
    • Once this candidate is accepted, Weijie Guo  will do the rest of the work as soon as possible. If necessary, we can talk about the subsequent work divisions in the next meeting.
  • No labels