Test cases AIP-66: DAG Bundles & Parsing

TC-001

Ensure DAG run using a specific version of DAG bundle.

1. Create a DAG bundle with multiple versions.
2. Trigger a DAG run. 
3. Modify DAG code and create a new version.   
4. Verify that DAG executes on the original DAG bundle version.   
5. Clear tasks on the original bundle version.

DAG should execute on the DAG bundle version at the time of DAG run creation.

Positive

TC-002

Ensure DAGs can opt to continue the existing behavior of running on the latest DAG code.

1. Update DAG to opt to continue using the latest code.   
2. Trigger DAG run.   
3. Modify DAG code and create a new version.   
4. Verify that DAG executes on the latest DAG bundle version.

DAG should execute on the latest DAG bundle version.

Positive

TC-003

Validate that local disk backend operates under "latest only" paradigm.

1. Configure local disk DAG bundle backend.   
2. Modify the DAG code.   
3. Trigger a DAG run. 
4. Confirm execution uses the latest DAG bundle version.

DAG runs should always execute on the latest DAG code.

Positive

TC-004

Validate Git backend retrieves DAG bundles based on commit hashes.

1. Create and version a DAG bundle in Git.   
2. Configure Git-based DAG bundle backend.   
3. Trigger DAG run with a specific commit hash.   
4. Verify that the correct DAG bundle version is used.

Correct DAG bundle version is retrieved based on commit hash.

Positive

TC-005

Verify manifest controls flexible DAG parsing behavior.

1. Create a manifest with parsing rules for directories and DAG files.   
2. Deploy the manifest in the DAG bundle.   
3. Monitor DAG parsing behavior.

DAGs should be parsed according to intervals and paths defined in the manifest.

Positive

TC-006

Validate scheduler behavior with latest DAG bundle versions.

1. Configure multiple DAG bundle versions.   
2. Trigger new DAG runs.   
3. Ensure scheduler uses the latest DAG bundle for new tasks.

Scheduler should use the latest DAG bundle version for new DAG runs.

Positive

TC-007

Ensure worker retrieves and uses correct DAG bundle version for task execution.

1. Trigger tasks using different versions of DAG bundles.   
2. Verify that the scheduler specifies the correct DAG bundle version for each task.   
3. Ensure the worker fetches and uses the specified version of the DAG bundle.   
4. Check if the worker can cache the bundle locally for multiple tasks when applicable.

Worker should receive instructions from the executor about the correct DAG bundle version and fetch it successfully. The worker may cache and reuse bundles for multiple tasks when necessary.

Positive

TC-008

Validate DAG bundle backends can be configured via UI, API, and CLI.

1. Configure a DAG bundle backend using Airflow UI.   
2. Configure a DAG bundle backend using API.   
3. Configure a DAG bundle backend using CLI.

Configuration should succeed via UI, API, and CLI, and backend should function as expected.

Positive

TC-009

Verify `.airflowignore` is still supported.

1. Create a DAG bundle with a `.airflowignore` file.   
2. Add files to be ignored.   
3. Deploy the bundle and trigger DAG parsing.   
4. Ensure that files in `.airflowignore` are not parsed.

Files listed in `.airflowignore` should be ignored.

Positive

TC-010

Test `bundle_fetch_interval` from the manifest.

1. Create a manifest with a `bundle_fetch_interval` (e.g., 5 minutes).   
2. Deploy the bundle.   
3. Modify DAG code.   
4. Verify DAG bundle fetching at the specified interval.

DAG bundle fetches should occur at the specified interval.

Positive

TC-011

Ensure backward compatibility with existing DAG folders.

1. Deploy Airflow without configuring DAG bundle backends.   
2. Add DAGs to the default `dags_folder`.   
3. Monitor DAG behavior.

Airflow should work with existing DAG folders without issues.

Positive

TC-012

Handle backend unavailability.

1. Configure a DAG bundle backend (e.g., Git). 
2. Simulate backend unavailability (e.g., disconnect Git repo).   
3. Trigger a DAG run.

Airflow should log errors and handle the unavailability gracefully.

Negative

TC-013

Ensure local caching of DAG bundles on workers.

1. Configure a worker for multiple tasks from the same bundle version.   
2. Ensure bundle caching.   
3. Trigger multiple tasks.   
4. Verify bundle reuse from the cache.

Worker should cache and reuse DAG bundles across tasks.

Positive

TC-014

Configure multiple DAG bundle backends.

1. Configure two different DAG bundle backends (e.g., Git and blob storage).   
2. Trigger DAG runs from both backends.   
3. Monitor for backend conflicts.

Airflow should handle multiple DAG bundle backends smoothly.

Positive

TC-015

Test long-running DAG with DAG bundle version change.

1. Trigger a long-running DAG using an older version.   
2. Modify DAG code mid-run.   
3. Ensure ongoing tasks use the original bundle.

Ongoing tasks should use the original DAG bundle version.

Positive

TC-016

Validate parsing behavior with varied scan intervals.

1. Create a manifest with different `scan_interval` and `min_file_parse_interval` for directories.   
2. Deploy the manifest.   
3. Monitor parsing for each directory.

Parsing should follow manifest-specified intervals.

Positive

TC-017

Handle misconfigured manifest file.

1. Create a manifest with invalid configurations (e.g., incorrect file paths).   
2. Deploy the manifest.   
3. Monitor parsing.

Airflow should log errors but continue functioning without crashing.

Negative

TC-018

Test parsing of DAG files on change.

1. Configure a manifest with `min_file_parse_interval: on change`.   
2. Modify a DAG file.   
3. Verify that only the changed file is re-parsed.

Airflow should re-parse only changed files.

Positive

TC-019

Manual refresh of DAG bundles via UI/API/CLI.

1. Trigger a DAG run.   
2. Modify DAG bundle content.   
3. Use UI/API/CLI to refresh the DAG bundle.   
4. Verify changes.

DAG bundle refresh should apply changes successfully.

Positive

TC-020

Handle large temp disk usage when caching bundles.

1. Configure a worker to cache multiple DAG bundles.   
2. Monitor temp disk usage.   
3. Ensure temp disk doesn’t exceed limits.

Airflow should not exceed temp disk limits.

Edge

TC-021

Handle scenario where the scheduler/executor requests a non-existent DAG bundle version.

1. Configure a DAG run with a version of a DAG bundle that does not exist (e.g., incorrect version in the configuration).   
2. Trigger the DAG run.   
3. Monitor the scheduler and executor as they send the request to the worker.   
4. Observe worker behavior when it attempts to fetch the non-existent DAG bundle version.

The worker should log an error when it cannot retrieve the non-existent DAG bundle version. The task execution should fail gracefully, and Airflow should log appropriate error messages.

Negative

TC-022

Test high-frequency parsing configuration.

1. Set `scan_interval` and `min_file_parse_interval` to very low values (e.g., 1 second). 
 2. Monitor resource usage and DAG parsing behavior.

Airflow should handle high-frequency parsing without crashing.

Edge

TC-023

Handle corrupted DAG bundle retrieval.

1. Simulate corrupted DAG bundle in the backend.   
2. Trigger DAG run that tries to use the corrupted bundle.   
3. Monitor task behavior.

Airflow should fail gracefully and log an appropriate error.

Negative