How do I stop an airflow DAG?

airflow airflow-scheduler
Greg Dougherty picture Greg Dougherty · Feb 28, 2018 · Viewed 11k times · Source

Running Airflow 1.9.0 with python 2.7. How do I gracefully stop a DAG?

In this case, I have a DAG that's running a file upload with bad code that causes everything to take 4 times as long, and I'd really prefer not to have to wait a day for it to finally time out (timeout is set to 10 hours).

The DAG looks for a tar file. When it finds one, it goes through every file in the tar, looking for files to process, and processes them.

I could not find any way to stop the DAG. I tried clicking on the "Running" circle in the "DAG Runs" column (the one to the right). It let me select the process and mark it as "failed". But it didn't stop running.

I tried clicking on the "Running" circle in the "Recent Tasks" column (the one to the left). It let me select processes, but trying to set them to filed (or to success), generated an exception in Airflow

Answer

Aclwitt picture Aclwitt · Jan 15, 2019

Browse -> DAG Runs -> Checkbox -> With Selected "Delete"

Related questions

Airflow 1.9.0 is queuing but not launching tasks

Airflow is randomly not running queued tasks some tasks dont even get queued status. I keep seeing below in the scheduler logs [2018-02-28 02:24:58,780] {jobs.py:1077} INFO - No tasks to consider for execution. I do see tasks in database …

Read More
Airflow tasks get stuck at "queued" status and never gets running

I'm using Airflow v1.8.1 and run all components (worker, web, flower, scheduler) on kubernetes & Docker. I use Celery Executor with Redis and my tasks are looks like: (start) -> (do_work_for_product1) ├ -> (do_work_for_…

Read More
Why is a task stuck and not executed in airflow?

I'm trying out airflow with the BigQueryOperator. I thought I would use google composer later on, but I want it running locally first. I have airflow up and running an BashOperator works fine, I can also run airflow test <…

Read More