athenad: fix memory leak by closing Response objects #34101
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
deanlee
force-pushed
the
athenad_fix_requests_memory_leak
branch
2 times, most recently
from
45f799b to
e99fab9
Compare
Joshawa1
approved these changes
Nov 25, 2024
Joshawa1
approved these changes
Nov 25, 2024
|
can the test be switched to a with statement? some static analysis for unclosed files/requests/contexts would be great, can you check if ruff has a rule for that? |
deanlee
force-pushed
the
athenad_fix_requests_memory_leak
branch
from
e99fab9 to
d1f097c
Compare
|
checked, ruff doesn't support detecting unclosed files/requests/contexts |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes a common memory leak when using the requests library, where unclosed HTTP Response objects prevent garbage collection and cause memory leaks. if Response objects are not explicitly closed, requests retains references to them, causing resource exhaustion over time.
Since we can't use awithstatement due tomock_put = mocker.patch('requests.put')intest_athenad.py,this fix explicitly closes the Response object after use.This ensures proper memory management and prevents resource exhaustion, particularly during large or repeated uploads.see psf/requests#4601 (comment) for details
resolve: #34079
Note: We should also review other areas where the requests library is used, such as the uploader, to ensure that Response objects are properly closed and to prevent potential memory leaks.