Proposed Agenda

Attendees

  • Vikram Koka

  • Shahar Epstein

  • Jarek Potiuk

  • Jed Cunningham

  • Sean Ghaeli

  • Michal Modras

  • Tamara Fingerlin

  • Jens Scheffler

  • Pavan Kumar

  • Ephraim Anierobi

  • Naseem Shah

  • Rahul Vats

  • Kenten Danas

  • Ash Berlin-Taylor

  • Phani Kumar

  • Shubham Raj

  • Cedrik Neumann

  • Christos Bisias

  • Kaxil Naik

  • Elad Kalif

Summary

  • Catch-up on action items from last call: None
  • Development updates
    • AIP-72 Task Execution Interface aka Task SDK update (Kaxil Naik)
      • Kaxil shared TP had eliminated the Base Operator completely from airflow.models, and that the serialized base / mapped operators were being used exclusively
      • Kaxil also said that the DAG object removal from models is in progress and that he moved to have the big DAG serialization PR out this week, so that all the Task information would be in the serialized DAG version without relying on the shared code.
      • This would then enable the server-client separation and independent deployment upgrades 
      • He said that Ephraim was migrating the airflow exceptions code to appropriate SDK/Core locations 
      • The one blocker for full separation was the shared config parse module. 
    • Benchmark (AIP-59 Performance tests framework) update (Michal Modras)
      • Michal shared the initial benchmark data comparing Airflow 2.10.5 vs. 3.0.0 which was run on a GKE cluster with 4 nodes.
      • Initial performance results for 2 baseline scenarios were positive with Airflow 3.x being significantly faster than Airflow 2.10
      • Michal said that the team would be testing with more scenarios and would publish the testing framework before Airflow Summit 
      • The team was positively surprised by the results and there was significant interest in leveraging the framework as part of RCs and as part of pre-merge checks for complex PRs such as pessimistic vs. optimistic scheduling. 
    • AIP-86 Deadline Alerts update (Dennis, Ramit)
      • Deferred to next dev call
    • Airflow 3.1 test plan update (Rahul)
      • Rahul shared the updated 3.1 test plan and covered the thinking behind it.
      • He explained that the key focus areas were around upgrade / downgrade testing, Human in the loop (since the development there was nearly complete), and the Task SDK decoupling. 
      • Rahul also said that the Task SDK decoupling was of critical priority and needed focus. 
      • Jarek asked about testing of internationalization, which led to the discussion below.
  • Discussion topics
    • UI Internationalization release process
      • Following up the 3.1 release testing plan, the team discussed the key steps to releasing the UI internationalization work. 
      • Jens spoke about the earlier discussed policy for this which included a reminder to the dev list about 2 weeks before RC, which would serve as a notice to the language owners to make sure that the translations are up to date before the release. 
      • Shahar volunteered to coordinate between the release manager (Kaxil) and the language owners for the 3.1 release. 
    • Airflow 3 adoption blockers (issues with label priority:upgrade_to_airflow3)
      • There was a good discussion of all the open (around a dozen) issues which were tagged as Airflow 3 upgrade blockers. 
  • Action items
    • Shahar: Sync up with Kaxil and send out note to language owners about translation updates and validation
    • Rahul: Update test plan and raise issues with contributors as needed 
    • Michal: Share benchmark framework when ready
    • Vikram: Ping Rajesh, Dennis, et al. about sync SLA callbacks