Airflow Task Logs Not Displaying in UI on Mac M1, Returns 404

[dengshu2]

Content:
I've set up Apache Airflow using Docker compose on my Mac (M1 chip). The installation and setup process went well, and I can successfully access the Airflow Webserver and Scheduler. However, when I try to view the task logs through the Airflow UI, the log display is blank.

I'm using the following versions:

• Airflow version: 2.6.2
• OS: macOS with M1 chip

In my Docker compose setup, I've mounted a local directory to the Airflow logs directory in the container, and also set up the directories as follows:

- ${AIRFLOW_PROJ_DIR:-.}/logs:/opt/airflow/logs
airflow_version=$$(AIRFLOW__LOGGING__LOGGING_LEVEL=INFO && gosu airflow airflow version)
mkdir -p /sources/logs /sources/dags /sources/plugins
chown -R "${AIRFLOW_UID}:0" /sources/{logs,dags,plugins}

When I check my local log directory (/Users/dengshu/Documents/airflow/logs), the logs seem to be present. However, in the Airflow UI, when I try to access the logs, it returns a 404 error. The request details are as follows:

172.18.0.1 - - [26/Aug/2023:09:25:44 +0000] "GET /api/v1/dags/my_first_dag/dagRuns/manual__2023-08-26T08:59:32.277324+00:00/taskInstances/first_task/logs/1?full_content=false HTTP/1.1" 404 190 "http://localhost:8080/dags/my_first_dag/grid?tab=logs&dag_run_id=manual__2023-08-26T08%3A59%3A32.277324%2B00%3A00&task_id=first_task" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36"

Has anyone else experienced this issue? I'd appreciate any help or guidance on how to resolve this.

[potiuk]

You need to look at what webserver logs show you in when you try to access the logs. Webserver should show you some errors when you try to access it, they should give you clue.

[dengshu2]

Thank you for your insightful response. I appreciate your assistance in helping me understand the issue at hand.
From your explanation, it seems that the problem may stem from a timezone issue, which could be causing the instance file path to be incorrect.
As per my current requirements, could you please guide me on how to address this issue using docker-compose or any other alternative methods? I'm looking for a solution that can effectively ensure the correct timezone is set, thereby preventing any potential file path errors.
Thank you once again for your time and help. I eagerly look forward to your suggestions.

[potiuk]

As per my current requirements, could you please guide me on how to address this issue using docker-compose or any other alternative methods? I'm looking for a solution that can effectively ensure the correct timezone is set, thereby preventing any potential file path errors.

I might try if you provide results of your investigation. What timezone? What error you see? what is your timezone in your host system? what is the timezone in your docker compose /image ? What is the discrepancy you see?

The more information you provide from your investigation and logs, the bigger chance someone (maybe be, maybe someone else will be able to help you with your problem).

[dengshu2]

I managed to solve the issue with the logs not displaying. The solution involved executing the setup from the official Docker Compose documentation. I believe the problem may have originated from my previous modifications to the Docker Compose file, in which I had removed the airflow worker and airflow flower services. Reverting these changes and following the official setup instructions seems to have resolved the issue.