Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove emitting SD event for receiving URI data update #998

Merged
merged 2 commits into from
Apr 25, 2024

Conversation

bohhyang
Copy link
Contributor

Summary

We recently added emitting SD status update receipt event for receiving URI data updates (previously only add/remove URI will emit the event). Turns out that increased the event volume a lot ---- apps like seas do URI data updates very frequently (weight, uri specific properties, etc.) and have both large fleet size and large audience group (hence huge fanout). Considering that having the events for add/remove URI is good enough for analyzing SD end-to-end propagation latency, and including the URI data updates could cause unmanageable pressure on the data analysis pipeline, such as Kusto query, inlogs, etc, we could exclude the URI data updates from emitting events.

Testing Done

Updated existing tests.

Copy link
Collaborator

@brycezhongqing brycezhongqing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

One thought from my side is:
If we want to keep update event for URI to analyse later, we may could do sample like only emit first 10 elements.

@bohhyang bohhyang merged commit a646a52 into master Apr 25, 2024
2 checks passed
@bohhyang bohhyang deleted the boyang/excludeUriDataChange branch April 25, 2024 15:35
@bohhyang
Copy link
Contributor Author

One thought from my side is: If we want to keep update event for URI to analyse later, we may could do sample like only emit first 10 elements.

The volume of Up/down status change is already huge and being sampled in query analysis actually. Adding Uri data update was just to be accurate but the volume could become unmanageable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants