Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

engine/cleanupmgr: move alert log operations to job queue #4211

Open
wants to merge 41 commits into
base: master
Choose a base branch
from

Conversation

mastercactapus
Copy link
Member

Description:
The alert log cleanup manually scans the alert_logs table for entries belonging to alerts that no longer exist. Because of the volume, there is no ON DELETE CASCADE trigger (it would make it impossible to delete some services, for example); this is the garbage collection operation to compensate.

This PR moves the alert log cleanup operation to the job queue system.

  • The look-for-work component breaks the alert logs table into blocks of 100k, and schedules a job for each block
  • Each of these jobs will scan/cleanup 5k rows at a time until complete
  • This operation will run once per week, or if restarting GoAlert, up to once per day.
  • The priority is set to the lowest of all cleanup operations.

# Conflicts:
#	engine/cleanupmanager/db.go
#	engine/cleanupmanager/queries.sql
#	gadb/queries.sql.go
# Conflicts:
#	engine/cleanupmanager/queries.sql
#	engine/cleanupmanager/update.go
@mastercactapus mastercactapus marked this pull request as ready for review December 18, 2024 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant