Use case of dummy operator

airflow airflow-operator
Nabin picture Nabin · Aug 13, 2019 · Viewed 10.1k times · Source

I was learning apache airflow and found that there is an operator called DummyOperator. I googled about its use case, but couldn't find anything that I can understand. Can anyone here please discuss its use case?

Answer

Yong Wang picture Yong Wang · Aug 14, 2019

Operator that does literally nothing. It can be used to group tasks in a DAG.

https://airflow.apache.org/_api/airflow/operators/dummy_operator/index.html

as far as I know, at least to two case:

  • test purpose. in dags, the dummy operation just between upstream and downstream, later, you can replace the true operator.
  • Workflow purpose: BranchPythonOperator work with DummyOperator. If you want to skip some tasks, keep in mind that you can’t have an empty path, if so make a dummy task. https://airflow.apache.org/concepts.html#workflows

Related questions

How do I restart airflow webserver?

I am using airflow for my data pipeline project. I have configured my project in airflow and start the airflow server as a backend process using following command airflow webserver -p 8080 -D True Server running successfully in backend. Now I …

Read More
execution_date in airflow: need to access as a variable

I am really a newbie in this forum. But I have been playing with airflow, for sometime, for our company. Sorry if this question sounds really dumb. I am writing a pipeline using bunch of BashOperators. Basically, for each Task, …

Read More
Airflow - How to pass xcom variable into Python function

I need to reference a variable that's returned by a BashOperator. In my task_archive_s3_file, I need to get the filename from get_s3_file. The task simply prints {{ ti.xcom_pull(task_ids=submit_file_to_spark) }} …

Read More