Apache Airflow is a workflow management platform to programmatically author, schedule, and monitor workflows as directed acyclic graphs (DAGs) of tasks.
I would like to create a conditional task in Airflow as described in the schema below. The expected scenario is …
python conditional-statements airflowProblem Is there any way in Airflow to create a workflow such that the number of tasks B.* is unknown …
python workflow airflowWhat is the way to pass parameter into dependent tasks in Airflow? I have a lot of bashes files, and …
bash airflowSay you have an airflow DAG that doesn't make sense to backfill, meaning that, after it's run once, running it …
python scheduled-tasks airflowHi everyone, I need to schedule my python files(which contains data extraction from sql and some joins) using airflow. …
python apache-spark scheduling reload airflowI followed online tutorial to set up Email SMTP server in airflow.cfg as below: [email] email_backend = airflow.utils.…
python smtp airflowI've been trying to use Airflow to schedule a DAG. One of the DAG includes a task which loads data …
python-3.x airflowI am using Airflow to schedule batch jobs. I have one DAG (A) that runs every night and another DAG (…
python etl airflowI am using the PythonOperator to call a function that parallelizes data engineering process as an Airflow task. This is …
python airflowI have a DAG in airflow and for now it is running each hour (@hourly). Is it possible to have …
airflow