Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Public user page is slow for users with "many" requests #8204

Open
laurentS opened this issue Apr 16, 2024 · 1 comment · May be fixed by #8463
Open

Public user page is slow for users with "many" requests #8204

laurentS opened this issue Apr 16, 2024 · 1 comment · May be fixed by #8463
Labels
f:user-profiles improvement Improves existing functionality (UI tweaks, refactoring, performance, etc) performance x:france

Comments

@laurentS
Copy link
Contributor

Ma Dada experienced several OOM crashes in the past few days, which I traced down to the /user/<username> page for one user with over 40k requests.

Specifically, when loading the public user page, this block calls User.safe_previous_names which generates about 7 SQL queries per FOI request authored by the user.
In our extreme case, this meant the https://madada.fr/user/open_knowledge_france page would take 110-150 seconds to load on a 12 core/64GB RAM machine, and it often crashed on the production server.

Loading the user page on WDTK for a random user with 650+ requests takes ~10s, which seems like a waste as the conclusion of all these queries is "no previous names" 😅 Put another way, as bots visit all user pages, and the total number of requests is over 1M, my guestimate is that this method wastes about 4.2 hrs of CPU time for each crawl of your users 💩 (the method result does not seem to get cached either, so this could possibly be used to DoS a site)

I have commented out the section in our template for now, but I'll try to improve the code to keep the functionality for the few cases where it is used.

The method does not appear to be used anywhere else.

@garethrees garethrees added f:user-profiles improvement Improves existing functionality (UI tweaks, refactoring, performance, etc) x:france labels Apr 17, 2024
@WilliamWDTK
Copy link
Collaborator

Just to note that one of the volunteers on WDTK has got in touch to report a slow-loading profile.

garethrees added a commit that referenced this issue Nov 22, 2024
Causes slow profile page loads. More we can clean up around this, but
this fixes the immediate user-facing issue.

Fixes #8204
@garethrees garethrees linked a pull request Nov 22, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
f:user-profiles improvement Improves existing functionality (UI tweaks, refactoring, performance, etc) performance x:france
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants