Airflow: Log file isn't local, Unsupported remote log location

javed picture javed · Aug 3, 2017 · Viewed 11.2k times · Source

I am not able see the logs attached to the tasks from the Airflow UI: enter image description here

Log related settings in airflow.cfg file are:

  • remote_base_log_folder =
  • base_log_folder = /home/my_projects/ksaprice_project/airflow/logs
  • worker_log_server_port = 8793
  • child_process_log_directory = /home/my_projects/ksaprice_project/airflow/logs/scheduler

Although I am setting remote_base_log_folter it is trying to fetch the log from http://:8793/log/tutorial/print_date/2017-08-02T00:00:00 - I don't understand this behavior. According to the settings the workers should store the logs at /home/my_projects/ksaprice_project/airflow/logs and they should be fetched from the same location instead of remote.

Update task_instance table content: enter image description here

Answer

Jaguar picture Jaguar · Nov 4, 2017

I also faced the same problem.

Setting below variables in airflow.cfg worked for me. Use {hostname} as machine's FQDN {hostname} instead of localhost.

endpoint_url = http://{hostname}:8080

base_url = http://{hostname}:8080

Best of luck!