This page summarizes the "CI/CD knowledge Tranfer" metting that took place in San Francisco, Friday 13th. 9am - 1pm in Westin hotel.
The meeting was organized by Jarek Potiuk in order to kick-start the project of knowledge transfer and reduce a "bus factor" for the CI/CD infrastructure of Airflow.
So far, most of the work in Airflow CI/CD was done be Jarek (especially when it comes to GitHub Actions workflows) but having more people involved is necesary so the CI/CD will continue to sustainably work - especially that we are approaching a busy period of work connected to Airflow 3, where we expect changes to make things restructuring and refactorings we plan for Airflow 3 work.
Participants:
- Jarek Potiuk, Jens Scheffler, Elad Kalif, Rom Sharon, Shahar Epstein
- Not participating, FYI: Hussein Awala Buğra Öztürk Amogh Desai
The room we took was an extended rental of Airflow Summit, and we used the fact that the people were already in the same place / hotel. We used Bloomberg's targeted sponsorship only to pay for breakfast (and some overhead, not the basic rental).
The things we discussed and explained:
- Handling regular maintenance and review of "main" status for the CI builds including fixing two failures: https://github.com/apache/airflow/pull/42227, https://github.com/apache/airflow/pull/42221
- CI/CD setup of ours as documented in https://github.com/apache/airflow/blob/main/dev/breeze/doc/ci/01_ci_environment.md including permissions (followed by https://github.com/apache/airflow/blob/main/dev/breeze/doc/ci/06_diagrams.md)
- Selective checks implementation: https://github.com/apache/airflow/blob/main/dev/breeze/doc/ci/04_selective_checks.md
- Particularly - labels used https://github.com/apache/airflow/blob/main/dev/breeze/doc/ci/04_selective_checks.md#changing-behaviours-of-the-ci-runs-by-setting-labels
- The links to the current status of ARC for our infrastructure https://github.com/apache/airflow-ci-infra - where we should switch to new workers and images when the ARC work is completeted
Particularly we reviewed, classified and prioritized work that are currently planned for our CI / CD: https://github.com/orgs/apache/projects/403 - including adding new ideas and proposals.
Total Cost of the meeting: USD 924.25