Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(inputs.http_listener_v2): Save the query parameters as tags #16073

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

tomasrojas26
Copy link

Summary

Add a new option to save the query parameters as tags. It's very useful to consume metrics from a client that only can send some informational parameters in the URI. Example:

curl -X POST http://local.domain:port/path?tag1=value1&tag2=value2 --data 'metrics in any format'

Checklist

  • No AI generated code was used in this PR

@telegraf-tiger telegraf-tiger bot added feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Oct 24, 2024
Copy link
Member

@DStrand1 DStrand1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I have a couple comments, additionally could you run make docs?

if h.QueryTag {
for key, values := range queryParams {
if len(values) > 0 {
m.AddTag(key, values[0]) // Only use the first value of the query parameter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there cases where we would need more than the first value?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think it's not. It was a quick way to parse the value because it's an array, but now that I looked at it more carefully I consider that it could be possible to include another loop to parse it. Do you think this approach is more accurate?

if h.QueryTag {
	queryParams := req.URL.Query()
	for key, values := range queryParams {
		if len(values) > 0 {
			for _, value := range values {
				m.AddTag(key, value)
			}
		}
	}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it does make more sense to be fully accurate, though I worry about this being specifically a tag, and cardinality issues it causes. If the query values change with any regularity this could cause issues. Do you think it makes sense to have this as a field instead?

Copy link
Author

@tomasrojas26 tomasrojas26 Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you're right. It's a really good point. The case I'm dealing with has static data, that's why I solve it with tags. But thinking more globally, there could be dynamic values, and as a consequence, it could generate an impact on the cardinality.

Maybe we could give the option to choose what parameters are tags, like you currently do with the option http_header_tags. Then another option to choose some parameters as fields. What do you think about?

It's a little bit complex and it adds some processing time to the plugin, but you give to the user the option to enable or not with the fields that they need. In the other hand, choosing all/some paremeters as fields, the user has to include a converter processor to choose their tags, which it could be possible as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could give the option to choose what parameters are tags, like you currently do with the option http_header_tags. Then another option to choose some parameters as fields. What do you think about?

I think this is a good solution, @srebhan what do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having http_query_parameter_tags might be good. This way the user has control over the cardinality. However, if multiple values are present, how about having one tag, sort the multiple values and string-join them?

@DStrand1 DStrand1 self-assigned this Oct 24, 2024
@srebhan
Copy link
Member

srebhan commented Dec 4, 2024

@tomasrojas26 any update?

@srebhan srebhan added the waiting for response waiting for response from contributor label Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins waiting for response waiting for response from contributor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants