Add maintenance window UI for Deployment Managers

[stephen-bracken]

Description

When running a large, multi team airflow deployment with multiple stakeholder teams and lots of dags, finding time to schedule routine maintenance can be difficult, and we have to rely on agreements from the teams that Airflow will not be running at agreed times e.g. Saturday at 11am-12pm.

Use case/motivation

I would like a screen added to the UI that calculates periods where no dags are running, i.e. 'maintenance windows' so that deployment managers of Airflow can calculate when maintenance periods should be scheduled. This is also useful for other engineers to calculate when systems Airflow is dependent on should have 'safe' downtime.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[shahar1]

Description

When running a large, multi team airflow deployment with multiple stakeholder teams and lots of dags, finding time to schedule routine maintenance can be difficult, and we have to rely on agreements from the teams that Airflow will not be running at agreed times e.g. Saturday at 11am-12pm.

Use case/motivation

I would like a screen added to the UI that calculates periods where no dags are running, i.e. 'maintenance windows' so that deployment managers of Airflow can calculate when maintenance periods should be scheduled. This is also useful for other engineers to calculate when systems Airflow is dependent on should have 'safe' downtime.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

Great idea!
Keep in mind that practically there could be edge cases where there's no such window in practice (e.g., Dags that run once a minute),
but:

1. It would be helpful to be aware of that, and maybe pre-calculate the "potential damage" (i.e., what Dags would be impacted due to the migration).
2. As an extension of this feature, maybe we could create a mechanism to prioritize tasks that have been on hold due to the maintenance window.

[stephen-bracken]

Great idea! Keep in mind that practically there could be edge cases where there's no such window in practice (e.g., Dags that run once a minute), but:

1. It would be helpful to be aware of that, and maybe pre-calculate the "potential damage" (i.e., what Dags would be impacted due to the migration).

2. As an extension of this feature, maybe we could create a mechanism to prioritize tasks that have been on hold due to the maintenance window.

Agree with 1. To expand on this, we could guestimate the window by adding the last or median dag run time to the next scheduled time? for 2. I can see how this would be useful, but it would need to be handled with care.