[BUG] Memory leak (lwip layer ?) when hit with SSL request (https://ip:80)
#172
Comments
|
Another coredump error I sometimes get is this one: |
|
Hello, Your stack size and queue size are huge: this is possible you run out of memory. Also, please do not post encoded stack traces like that: this is impossible to read and decrypt them. There is nothing we can do so I will close. Please reopen if:
Note: we are all very busy and maintaining this OSS project on our free time so please make sure when you open an issue, there is inside enough info for us to do something ;-) Thanks. |
you do not seem to use latest versions. please make sure to upgrade also before opening an issue ;-) |
|
Many thanks for your prompt reply, highly appreciated!
I did try reducing this before posting this issue, but it didn't influence the behaviour for HTTPS responses. Moreover, I need to transfer files of ~500 kB, and if I reduce the queue size and stack size, the ESP doesn't survive a stresstest (wdt triggers).
I'm using a debug build with the decoder enabled, hence the for the '#0 0x403897b4 in _prvTaskExitError at /home/runner/work/' messages. For the life of me I've never seen any more advanced debugging messages. I just did some Googling and as far as I can see I've configured things correctly. Could you please point me in the right direction? That would help me out immensely.
While I understand you are very busy, and would like more advanced debugging data, I'm not sure this statement is true. I do see the behaviour in various implementations and am quite sure my code is OK since it runs fine otherwise. Was it ever confirmed that repetitively sending HTTPS-requests to the HTTP server does not cause exceptions?
Your effort is highly appreciated! I hope you can point me in the direction of how to get more advanced exception decoding, so I can support as much as possible.
Thanks for the pointer, I recently pulled changes from the libs and assumed they were the latest versions. Will look into this as well. |
best thing you could do is:
to decode exceptions you can place that in your pio file:
Please re-open the issue once you have:
thanks! |
|
@vortigont : FYI. this might be related to #165 but not sure considering the little info we have.... |
|
I already had After updating AsyncTCP to v3.3.0 and ESPAsyncWebServer to v3.4.1, the error did change a bit: |
|
I'm happy to provide more information, but at the moment that doesn't seem possible (at least for me). However, triggering the problem is as easy as opening any webpage served by an ESP running your library through a browser (Firefox in my case) using https://ESP:80 a few times in a row. |
|
I am wondering if it could be because you are on an old arduino version (espressif32@6.9.0). could you try switching to latest arduino version ?
Since Arduino 3, you need to use pioarduino since PlatformIO dropped support for Arduino. Also, which ESP board are you using ? |
|
I didn't know that support was dropped, will take a look to migrate things. I'm using an ESP32-C3-MINI chipset btw. |
|
Yes, sadly. The community is quite unhappy about that. FYI ESP32-C3-MINI is single core, so you cannot have ESPAsyncWS and ASyncTCP of course have more sense on dual-core.... If you are on single core, there is no point in using an asynchronous web server (although it should work). You could directly use Arduino WebServer ;-) Also, do you have this issue if you use other forks of AsyncTCP and ESPAsyncWS, or the original repos from me-no-dev ?
|
ugh that SSL-everywhere madness... SSL is a binary protocol and can't be properly identified by current websever's code. We might try to improve it probably, but it would be an overhead code for a thing that should no be there anyway. |
|
This morning I spent some time migrating everything to the latest platform (which required some code changes, but ala). Now, the good news is that the code runs like it used to. However, the bad news is that the code runs like it used to. It's perfectly stable until you do https://esp:80. This will crash it quite reliably. I know I can set this parameter, however the people that use my hardware do not... They just open port forwarding on the router and have it crash the ESP every time they attempt to open the web page as the browsers attempt SSL first. |
|
I already explained that AsyncTCP does not support SSL. Why are you still connecting with https to your esp ? If you need SSL, use PsychicHttp, not ESPAsyncWS! |
|
I believe there is a misunderstanding. My intention is NOT to use SSL. The intention is to avoid crashing when my users attempt to (accidentally) do so :) |
|
BTW, I can't reproduce it with curl, it just closes the connection with 404, no crash. So it's not a 100% case and might depend on how webserver is set for your project. |
|
curl also doesn't cause a crash here, please test with Firefox instead. |
|
Tried to run the SimpleServer sample in the project: |
|
I did some more testing:
With the latest libraries, my coredump is as follows: Note the message |
|
I was finally able to reproduce with FF, hitting refresh a couple f time with I think we could add a feature inside |
|
v3.4.3 released! |
|
No worries, it's great you picked it up. There is a memory leak now though :-P |
oh crap where ? |
|
no idea; every time I do an F5 with HTTPS about 2kb is lost. |
|
perhaps _client->close() is leaky? |
|
I am wondering if this is the created request object which is never sent... I will have a look. edit Oh! thanks a lot for the support btw 😃 |
|
I can reproduce and I fear this memory leak was caused by something else because as we can see, the disconnect handler is closed, which is deleting the request object. @vortigont did some change recently in AsyncTCP queues so I will check with a previous version of AsyncTCP |
|
Ok so this is not caused by AsyncTCP changes. The request object is correctly deleted: https://192.168.4.1:80/ (1 hit creates 2 or more requests) http://192.168.4.1/ (several hits) |
|
@vortigont : fyi I also checked line 1104 of AsyncTCP I suspect the issue to be before ESPAsyncWS because the data from a new client arrives, a new request is created ( All that within the call of Then, This one is a hard one lol! |
|
it's not that scary actually |
|
I am wondering if it is browser dependent. For exemple, Firefox seems to retry. Also, even if globally, issuing several refresh does decrease the heap, sometimes, I see the feee heap value that is going up again. But not to the point where it was at start. Searching on Google, I saw some older issues regarding SO_LINGER not properly implemented: set to 0 by default, which should make the esp-idf layer discard the packets when connection closes, but those issues were reporting that the code is not cleaning, and users had to activate SO_LINGER and set the cleanup time to 0 in order to force an immediate buffer release once the connection closes. But whether this issue is still there or not, we do not have control over that. on Arduino Core. Strangely, esp-idf http server seems to have control over that: |
|
Out of curiosity, I tested with AsyncTCPSock, which is based on Hitting refresh consecutively also does decreases the heap. So I added this to activate so_linger with a 0 delay And the free heap still decreases. But a little less than with AsyncTCP. So its not that. |
|
my 5 cents that it's lwip's internal allocations and garbage collection. It should keep tcp struct lingering for some time by the standard. It would be much easy to use |
mathieucarbou
changed the title
https://ip:80)
mathieucarbou
added
bug
help wanted
|
many thanks, works perfectly |
Description
Running PlatformIO with espressif32@6.9.0, mathieucarbou AsyncTCP and ESPAsyncWebServer. Custom settings:
-D CONFIG_ASYNC_TCP_PRIORITY=10
-D CONFIG_ASYNC_TCP_QUEUE_SIZE=256
-D CONFIG_ASYNC_TCP_STACK_SIZE=8192
-D CONFIG_ASYNC_TCP_RUNNING_CORE=1
When a request is made through HTTPS to the HTTP port, the web server (or AsyncTCP, perhaps) crashes fairly reliably, instead of just returning a request error. Sometimes it requires a few CTRL-F5 attempts for the core panic to occur.
Stack trace
How can I avoid this? Some of my users are using port forwarding with random port numbers, which results in problems if you just enter the web address without adding http:// explicitly since the browser attempts HTTPS first for non-standard ports. Apart from that, crashing because of such requests should never occur imho.
The text was updated successfully, but these errors were encountered: