Table of contents

Jira Boards

Flink 1.19 Burndown: 1.19 Burndown - Agile Board - ASF JIRA (apache.org)

Sync meeting

  • The release sync will start from Nov 14th, 2023, at 9am (CET, UTC+1) and 4pm (UTC+8). The release sync happens bi-weekly at first, and will be adjusted to weekly as we approaching the feature freeze date. 
  • Feel free to join on Google Meet, and attendees are encouraged to fill out the topics to be discussed at the bottom of this page a day in advance to make it easier for everyone to understand the background of the topic. 

Timeline

  • Feature Freeze
    • Jan 26, 2024, 00:00 CET(UTC+1) 
  • Release
    • End of Feb 2024


Highlight features

please feel free to add/suggest.

Features

List of features announced by contributors and committers that are likely to be ready for the feature freeze:n

NOTICE: It's preferred if only new features end up there and not all bugs/tasks separately, so that the page is not over bloated. Of course, unless fixing a bug is a really big or important one equivalent to implementing a completely new feature. A good rule of thumb would be that each entry in the page could (but does not have to) be later on included in a release blog post.

Legend

State

symbolmeaningcomment
(big grin)validatedthrough cross team testing
(tick)donewell documented with a complete test coverage
(green star)will make itthere is no reason this effort should not go into 1.19
(star)in dangerthere are some concerns the effort could be ready for the feature freeze of 1.19
(red star)very unlikelythere are severe concerns the effort could make it to 1.19
(minus)won't make itit was decided against adding this for the 1.19 release. working on the effort has been stopped.
(question)state unclear
(blue star)independentas the artifact could be released independent of Apache Flink

X-Team verification

symbolmeaning
(tick)done
(blue star)not required

Feature Stage

Please align with the list on the Apache Flink Roadmap (https://flink.apache.org/roadmap.html).

  • MVP: Have a look, consider whether this can help you in the future.
  • Beta: You can benefit from this, but you should carefully evaluate the feature.
  • Ready and Evolving: Ready to use in production, but be aware you may need to make some adjustments to your application and setup in the future, when you upgrade Flink.
  • Stable: Unrestricted use in production
  • Reaching End-of-Life: Stable, still feel free to use, but think about alternatives. Not a good match for new long-lived projects.
  • Deprecated: Start looking for alternatives now 

Summary

Numbers are based on the items in the list below, not on the tickets 


(big grin)(tick) (green star)(star)(red star)(minus)(question)(blue star)∑Remaining weeks
2023-11-28
211




219
2023-12-12
311




237
2023-12-26
415




275
2024-01-09
415

1

273
2024-01-23
157214

291
2024-01-30
240214

310
2024-02-06









List

Feel free to add categories.

Runtime
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verifiedVolunteers for testing
Shuffle


































Coordination










FLIP-379: Dynamic source parallelism inference for batch jobs

Xia Sun

Zhu Zhu 



(tick)
2024-01-23



FLINK-31757 - Getting issue details... STATUS

RocMarshal 

Rui Fan Yangze Guo Zhu Zhu 

Accepted
(minus)
2024-01-22



FLINK-33735 - Getting issue details... STATUS

Rui Fan 

Zhu Zhu RocMarshal 

DoneAll features are merge, it only left a little test and doc, so marking it done first.(tick)
2024-01-22



FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

Piotr Nowojski 


Done
(tick)
2024-01-18(tick)


FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

Piotr Nowojski 
Done


(tick)
2024-01-24(tick)


FLIP-386: Support adding custom metrics in Recovery Spans

Piotr Nowojski 


Done


(tick)
2024-01-24(tick)


FLIP-383: Support Job Recovery for Batch Jobs

Lijie Wang 

Junrui Lee 

Zhu Zhu 

Accepted
(minus)
2024-01-22



State & Checkpoint










FLINK-30613 - Getting issue details... STATUS

Hangxiang Yu




(tick)
2024-1-15



FLINK-34190 - Getting issue details... STATUS

Zakelly Lan 




(tick)
2024-1-24



REST




































Configuration










FLIP-366: Support standard YAML for FLINK configuration

Junrui Lee 

Zhu Zhu 



(tick)
2024-01-26



FLINK-33712 - FLIP-391: Deprecate RuntimeContext#getExecutionConfig

Junrui Lee 

Zhu Zhu 



(tick)
2024-01-16



FLINK-33581 - FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

Junrui Lee 

Zhu Zhu 



(tick)
2024-01-16



FLINK-34079 - Getting issue details... STATUS

Xuannan Su Rui Fan 


Done
(tick)
2024-01-22



SQL

Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

FLINK-33397 - Getting issue details... STATUS

Jane Chan




(tick)
2024-01-30



FLINK-33203 - Getting issue details... STATUS

@Yangze Guo




(tick)





FLIP-376: Add DISTRIBUTED BY clause

Timo Walther 




(minus)
2024-01-30



FLIP-392: Deprecate the Legacy Group Window Aggregation

FLINK-24024 & FLINK-20281

xuyang 



no new flip needed, continue the work on FLINK-24024 & FLINK-20281(tick)
2024-01-26

FLINK-24024: (tick)

FLINK-20281:  N/A



FLIP-393: Make QueryOperations SQL serializable

Dawid Wysakowicz 


Done
(tick)
2024-01-18(tick)N/A

FLIP-387: Support named parameters for functions and call procedures

@Feng Jin

xuyang 
need documentation(tick)
2024-01-30



FLINK-31830 - Getting issue details... STATUS

Jane Chan



Under discussion(minus)
2024-01-18



FLIP-415: Introduce a new join operator to support minibatch

xu_shuai_ 

lsy


need documentation(tick)
2024-01-30

(tick)

FLIP-400: AsyncScalarFunction for asynchronous scalar function support

Alan Sheinberg 

Timo Walther 
Needs documentation(tick)
2024-02-05(tick)


Connectors










FLINK-31275 - Getting issue details... STATUS

Fang Yong

Qingsheng Ren




(minus)
2024-01-22



FLINK-26203 - Getting issue details... STATUS

Leonard Xu




Pulsar connector has formal release already(tick)
2024-01-23



FLINK-33261 - Getting issue details... STATUS

Zhanghao Chen 

Dewei Su 

Benchao Li 


only left for changes in kafka connector(tick)
2024-01-22



FLINK-25857 - Getting issue details... STATUS

FLIP-371: Provide initialization context for Committer creation in TwoPhaseCommittingSink

Peter Vary 

Márton Balassi 

Done
(tick)
2024-01-18



FLINK-33972 - Getting issue details... STATUS
FLIP-372: Enhance and synchronize Sink API to match the Source API

Peter Vary 

Tzu-Li (Gordon) Tai 

Mark it done since main part of the new api been merged (remaining tests and doc)
(tick)
2024-01-23



FLINK-33528 - Getting issue details... STATUS

Peter Vary 

Marton Balázs 



(minus)





Resource Management


































Runtime


FLINK-33625 - Getting issue details... STATUS

Rui Fan RocMarshal Done
(tick)
2023-12-04



FLINK-33315 - Getting issue details... STATUS

A series of optimizations to reduce the memory and cpu usage for large StreamOperator

Rui Fan 

Piotr Nowojski Weihua Hu RocMarshal 



(tick)
2023-11-23















Misc












FLINK-33325 - Getting issue details... STATUS

Yu Chen

Yun Tang 

DONE, only docs left
(tick)
2024-01-23



FLINK-33230 - Getting issue details... STATUS

(will create a FLIP)

Yu Chen 

Yun Tang 



(minus)
2024-01-23



Status / Follow-ups

11/14/2023

  • Is Google Meet OK for everyone?  - ok
  • The release sync meeting time, currently Europe has gone into winter time(CET, UTC+1), is 9am ok? - ok
  • Daily work divisions
    • JIRA ticket (Mainly focus on new blockers, progress of existing blockers each week) lincoln lee 
    • CI (Failures reported in #build channel and not reported) Martijn Visser 
    • Benchmark (Check if there's regression in the #flink-dev-benchmarks channel) Yun Tang 
    • Mailing list monitoring Jing Ge 
    • This division doesn't mean that other RMs are now allowed to do the work. All RMs should take responsibilities on all of these daily issues if they are available. 
  • Feature freezing date. - Jan 26, 2024, 00:00 CET(UTC+1) 
  • Retrospective of 1.18

11/28/2023

  • Blockers
  • GitHub Actions Migration discussion in dev ML (FLIP-396)
  • flink-shaded driven by Sergey Nuyanzin 
  • As suggested by Xintong Song, we encourage attendees to fill out the topics a day in advance  to make it easier for everyone to understand the background of the topic. 

12/12/2023

12/26/2023

  • Current status of 1.19 Features lincoln lee - updated
    • 5 weeks remaining, total 27, done 4, will make it 15
  • Existing blocker issues lincoln lee
    • FLINK-33641 - Getting issue details... STATUS has been fixed
  • Skip this sync meeting? - no response until 12/25, skipped finally

01/09/2024

  • Current status of 1.19 Features lincoln lee - updated
    • 3 weeks remaining, total 27, done 4, will make it 15, won't make it 1
  • Existing blocker issues lincoln lee

01/23/2024

01/30/2024

02/06/2024

  • Blockers:
  • Cut release-1.19 branch
  • fyi: FLINK-34360 - Getting issue details... STATUS GitHub Actions e2e tests seem to have disk space issues causing multiple test failure (not present in Azure) 
  • Cross-team release testing FLINK-34285 - Getting issue details... STATUS  
    • total 26 flip/features, 15 confirmed,  7 not required,  4 wait for response(authors: Dawid Wysakowicz, Peter Vary, Alan Sheinberg)
    • testing instructions ready: 11  (1 completed, 3 assigned,  7 unassigned),  wait for doc ready:  2
    • more completed flips found, will update the testing ticket (done, 6 Instructions tickets added)

02/13/2024

  • Not verified Jira issues:
  • CI instability issues:
    • FLINK-34273 - Getting issue details... STATUS
    • FLINK-34202 - Getting issue details... STATUS
    • Nightly build for 1.19 doesn't seem to be picked
  • Cross-team release testing FLINK-34285 - Getting issue details... STATUS  
    • wait for response:  there's some authentication problem in current kanban dashboard,  use filter on Jira search 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels is EMPTY AND status = OPEN ORDER BY updatedDate'  
    • wait for doc ready:  
    • testing tickets: Jira filter condition 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels = release-testing ORDER BY updatedDate'

02/20/2024

  • Not verified Jira issues:
  • CI instability issues:
  • Cross-team release testing FLINK-34285 - Getting issue details... STATUS  
    • wait for response:  there's some authentication problem in current kanban dashboard,  use filter on Jira search 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels is EMPTY AND status = OPEN ORDER BY updatedDate'  
    • wait for doc ready:  
    • testing tickets: Jira filter condition 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels = release-testing ORDER BY updatedDate'
      • 15 total, 8 closed, 4 unassigned, 3 testing
  • When to create rc? –- after release testing finished and no blockers

02/27/2024

03/05/2024

03/12/2024

03/19/2024

  • Finalize & Promote Release
    • also followup work: start a discussion for 1.17 end of life 
  • Release progress
    • Wiki Page Updates
      • swith to use token for uploading python
      • no need to upload signatures to PyPI
    • Flink-docker
      • publish docker image: should specify 'docker.io' for login if encounter "Error: writing blob: initiating layer upload to /v2/apache/flink/blobs/uploads/ in registry-1.docker.io: requested access to the resource is denied"
    • Promote release: add a necessary sub task FLINK-34716 - Getting issue details... STATUS
  • No labels