How to run one airflow task and all its dependencies?

python-3.6 airflow airflow-scheduler
itzjustricky picture itzjustricky · Mar 6, 2017 · Viewed 7.4k times · Source

I suspected that

airflow run dag_id task_id execution_date

would run all upstream tasks, but it does not. It will simply fail when it sees that not all dependent tasks are run. How can I run a specific task and all its dependencies? I am guessing this is not possible because of an airflow design decision, but is there a way to get around this?

Answer

Priyank Mehta picture Priyank Mehta · Mar 7, 2017

You can run a task independently by using -i/-I/-A flags along with the run command.

But yes the design of airflow does not permit running a specific task and all its dependencies.

You can backfill the dag by removing non-related tasks from the DAG for testing purpose

Related questions

SyntaxError: unexpected EOF while parsing

I am getting error while running this part of the code. tried some of the existing solutions, none of them helped elec_and_weather = pd.read_csv(r'C:\HOUR.csv', parse_dates=True,index_col=0) # Add historic DEMAND to each …

Read More
How can I convert a .py to .exe for Python?

I'm trying to convert a fairly simple Python program to an executable and couldn't find what I was looking for, so I have a few questions (I'm running Python 3.6): The methods of doing this that I have found so far …

Read More
ModuleNotFoundError: What does it mean __main__ is not a package?

I am trying to run a module from the console. The structure of my directory is this: I am trying to run the module p_03_using_bisection_search.py, from the problem_set_02 directory using: $ python3 p_03_using_bisection_search.…

Read More