Ingress dies [ConnectionResetError: Cannot write to closing transport] #5248
Labels
Comments
|
Same issue occurred again this evening. Now I have set up an automation for HA to send me a push notification when it starts so I can investigate the system shortly afterward and restart the supervisor. |
|
home-assistant/core#124173 is the fix on the core side for this |
|
The issue occurred again this afternoon, however the supervisor log looks a bit different this time. In this case there was no error Here is the HA log from around the same period: Even though the logs appear quite different, the symptoms that I experienced were exactly the same. Ingress was still down, and sshing to the host and running The problems that HA encounters which cause the supervisor to restart it could be many, however the main issue is that the supervisor fails to re-establish a connection to HA after the restart because it has encountered some problem of it's own. Can you tell from these new logs whether you expect your fix to work for this situation as well? |
|
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. |
|
Since upgrading to 2024.8.3 I have not seen the random restarts of Home Assistant which seem to be the main cause of this issue with the supervisor. The system has been running stable for about 1 month now which is a good sign! Although there is still likely some problem with the Supervisor I'll close this issue for now and reopen again if the problem re-occurs. |
|
Although the occasions when this occurs has reduced, the problem is still there. This is how I understand the sequence in the logs:
Why does the supervisor continually restart the observer after the supervisor loses contact with home assistant, rather than restart itself? Isn't the supervisor supposed to manage itself too? There seems to be a error in the logic of the decision making from supervisor here. I just run I just learned about the observer webportal available at http://homeassistant.local:4357/, which I will inspect next time this happens. I also set the supervisor.plugins.observer logging level to debug Supervisor log: HA log: |
|
This occurred again while I was at the computer this time looking at home assistant. I did not observe any problems with home assistant on my computer but I get a push notification to my phone when HA starts, and that's how I knew some problem had occurred. This time I checked http://homeassistant.local:4357/but it did not provide me with any useful information. It said everything was healthy, even though the supervisor had lost connection to HA. Here is the supervisor log: The HA log shows no errors around the time the supervisor started getting timeouts from the core api. The log right now has a large amount of spam in it because I set it to debug so I won't bother posting it. I will will change it back info because debug is not helpful to investigate this issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the issue you are experiencing
For a very long time on my setup (maybe 1 year+) ingress has been dying 2-3 times per month. The problem has migrated with my setup from a RPI4 4GB with SD card, to RPI4 4GB with SSD, to an Intel N100 with 16GB.
After ingress from the supervisor dies, HA still appears to run correctly, but all add-ons become unavailable. The only way to resolve the problem is to ssh to the host and run
ha supervisor restartor to restart the host OS from the GUI.The problem seems to begin with the supervisor encountering
ConnectionResetError: Cannot write to closing transport.This error occurs 22 times in the log file, and then HA is restarted. However when HA comes up again the supervisor seems to fail to establish contact with it as the supervisor log continually repeats the error
Timeout on call http://172.30.32.1:8123/api/core/statesupervisor-2024-08-14.log
From what I can tell in the HA log, everything appears ok except for one odd thing that I can't explain which is that this message from HA:
s6-rc: info: service legacy-services: stoppingat18:48:58.033occurs before the supervisor log message:[supervisor.docker.manager] Restarting homeassistantat18:49:35.445. This oddity is quite a bit after the initial problems that the supervisor encountered, so I'm inclined not to read too much into it.ha-2024-08-14.log
What type of installation are you running?
Home Assistant OS
Which operating system are you running on?
Home Assistant Operating System
Steps to reproduce the issue
Anything in the Supervisor logs that might be useful for us?
[31m2024-08-13 18:44:54.294 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request resp = await request_handler(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 543, in _handle resp = await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 114, in impl return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 189, in block_bad_requests return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 205, in system_validation return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 224, in token_validation return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/supervisor/supervisor/api/middleware/security.py", line 283, in core_proxy return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/supervisor/supervisor/api/ingress.py", line 161, in handler return await self._handle_websocket(request, addon, path, session_data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/supervisor/supervisor/api/ingress.py", line 190, in _handle_websocket await ws_server.prepare(request) File "/usr/local/lib/python3.12/site-packages/aiohttp/web_ws.py", line 152, in prepare payload_writer = await super().prepare(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_response.py", line 417, in prepare return await self._start(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/web_response.py", line 425, in _start await self._write_headers() File "/usr/local/lib/python3.12/site-packages/aiohttp/web_response.py", line 505, in _write_headers await writer.write_headers(status_line, self._headers) File "/usr/local/lib/python3.12/site-packages/aiohttp/http_writer.py", line 130, in write_headers self._write(buf) File "/usr/local/lib/python3.12/site-packages/aiohttp/http_writer.py", line 75, in _write raise ConnectionResetError("Cannot write to closing transport") ConnectionResetError: Cannot write to closing transport�[0mSystem Health information
There are currently no repairs availableSupervisor diagnostics
config_entry-hassio-a7a964f4381b8f813b47cd58b792c395.json
Additional information
No response
The text was updated successfully, but these errors were encountered: