Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security assistant] Fix invoke_assistant_success telemetry schema #204280

Merged
merged 2 commits into from
Dec 16, 2024
Merged

[Security assistant] Fix invoke_assistant_success telemetry schema #204280

merged 2 commits into from
Dec 16, 2024

Conversation

stephmilovic
Copy link
Contributor

@stephmilovic stephmilovic commented Dec 13, 2024 •
edited by kibanamachine
Loading

Summary

For the invoke_assistant_success event, I had the toolsInvoked schema defined incorrectly. The data still reached the telemetry server and is searchable, but the result was a warning in the console:

Error in handler TelemetryTracer, handleChainEnd: Error: Failed to validate payload coming from "Event Type 'invoke_assistant_success'":
	- []: excess key 'toolsInvoked' found

Correcting the schema clears the error

To test

  1. Clear Kibana server logs in console
  2. Send a message
  3. Search Kibana server logs for toolsInvoked
  4. No error log should be present. There should only be one instance of toolsInvoked like this: 
    [DEBUG][plugins.elasticAssistant.defaultAssistantGraph.telemetryTracer] Invoke invoke_assistant_success telemetry:
{
  "actionTypeId": ".gen-ai",
  "assistantStreamingEnabled": true,
  "isEnabledKnowledgeBase": false,
  "durationMs": 2102,
  "toolsInvoked": {},
  "isOssModel": false
}

@stephmilovic stephmilovic added release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development Team:Security Generative AI Security Generative AI labels Dec 13, 2024
@stephmilovic stephmilovic requested review from a team as code owners December 13, 2024 20:46
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@stephmilovic stephmilovic changed the title fix telemetry schema [Security assistant] Fix invoke_assistant_success telemetry schema Dec 13, 2024
@stephmilovic stephmilovic mentioned this pull request Dec 13, 2024
Merged
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

e40pud
e40pud approved these changes Dec 14, 2024
View reviewed changes
Copy link
Contributor

@e40pud e40pud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked locally and do not see the error anymore. Thanks for the fix!

@stephmilovic
Copy link
Contributor Author

@elasticmachine merge upstream

@stephmilovic stephmilovic enabled auto-merge (squash) December 16, 2024 15:15
@stephmilovic stephmilovic merged commit 86fe148 into elastic:main Dec 16, 2024
8 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.16, 8.17, 8.x

https://github.com/elastic/kibana/actions/runs/12357529333

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 16, 2024
@stephmilovic
[Security assistant] Fix invoke_assistant_success telemetry schema (…
0850111 
…elastic#204280)

(cherry picked from commit 86fe148)
@kibanamachine kibanamachine mentioned this pull request Dec 16, 2024
Merged
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
❌ 8.16 Backport failed because of merge conflicts
❌ 8.17 Backport failed because of merge conflicts
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 204280

Questions ?

Please refer to the Backport tool documentation

@stephmilovic stephmilovic mentioned this pull request Dec 16, 2024
Merged
stephmilovic added a commit to stephmilovic/kibana that referenced this pull request Dec 16, 2024
@stephmilovic
[Security assistant] Fix invoke_assistant_success telemetry schema (…
aad3221 
…elastic#204280)

(cherry picked from commit 86fe148)

# Conflicts:
#	x-pack/plugins/security_solution/server/assistant/tools/index.ts
@stephmilovic stephmilovic mentioned this pull request Dec 16, 2024
Merged
@stephmilovic
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
✅ 8.17
✅ 8.16

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

stephmilovic added a commit to stephmilovic/kibana that referenced this pull request Dec 16, 2024
@stephmilovic
[Security assistant] Fix invoke_assistant_success telemetry schema (…
8b18c75 
…elastic#204280)

(cherry picked from commit 86fe148)

# Conflicts:
#	x-pack/plugins/security_solution/server/assistant/tools/index.ts
stephmilovic added a commit that referenced this pull request Dec 16, 2024
@stephmilovic
[8.17] [Security assistant] Fix invoke_assistant_success telemetry…
f4904ea 
… schema (#204280) (#204453)

# Backport

This will backport the following commits from `main` to `8.17`:
- [[Security assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)](#204280)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Steph
Milovic","email":"stephanie.milovic@elastic.co"},"sourceCommit":{"committedDate":"2024-12-16T17:00:59Z","message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:
SecuritySolution","backport:prev-major","Team:Security Generative
AI"],"number":204280,"url":"https://github.com/elastic/kibana/pull/204280","mergeCommit":{"message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204280","number":204280,"mergeCommit":{"message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c"}},{"url":"https://github.com/elastic/kibana/pull/204451","number":204451,"branch":"8.x","state":"OPEN"}]}]
BACKPORT-->
kibanamachine added a commit that referenced this pull request Dec 16, 2024
@kibanamachine @stephmilovic
[8.x] [Security assistant] Fix &#x60;invoke_assistant_success &#x60; …
2fe297d 
…telemetry schema (#204280) (#204451)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security assistant] Fix &#x60;invoke_assistant_success &#x60;
telemetry schema
(#204280)](#204280)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Steph
Milovic","email":"stephanie.milovic@elastic.co"},"sourceCommit":{"committedDate":"2024-12-16T17:00:59Z","message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:
SecuritySolution","backport:prev-major","Team:Security Generative
AI"],"title":"[Security assistant] Fix `invoke_assistant_success `
telemetry
schema","number":204280,"url":"https://github.com/elastic/kibana/pull/204280","mergeCommit":{"message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204280","number":204280,"mergeCommit":{"message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c"}}]}]
BACKPORT-->

Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
@kibanamachine
Copy link
Contributor

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Dec 17, 2024
stephmilovic added a commit that referenced this pull request Dec 17, 2024
@stephmilovic @elasticmachine
[8.16] [Security assistant] Fix invoke_assistant_success telemetry…
2699f17 
… schema (#204280) (#204454)

# Backport

This will backport the following commits from `main` to `8.16`:
- [[Security assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)](#204280)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Steph
Milovic","email":"stephanie.milovic@elastic.co"},"sourceCommit":{"committedDate":"2024-12-16T17:00:59Z","message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:
SecuritySolution","backport:prev-major","Team:Security Generative
AI"],"number":204280,"url":"https://github.com/elastic/kibana/pull/204280","mergeCommit":{"message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204280","number":204280,"mergeCommit":{"message":"[Security
assistant] Fix `invoke_assistant_success ` telemetry schema
(#204280)","sha":"86fe148e963f2afaaa02cc1160a58bb959904a5c"}},{"url":"https://github.com/elastic/kibana/pull/204451","number":204451,"branch":"8.x","state":"OPEN"}]}]
BACKPORT-->

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@kibanamachine kibanamachine added v8.16.2 and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Dec 17, 2024
JoseLuisGJ pushed a commit to JoseLuisGJ/kibana that referenced this pull request Dec 19, 2024
@stephmilovic @JoseLuisGJ
[Security assistant] Fix invoke_assistant_success telemetry schema (…
ff6b914 
…elastic#204280)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@e40pud e40pud e40pud approved these changes

Assignees
No one assigned
Labels
backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development release_note:skip Skip the PR/issue when compiling release notes Team:Security Generative AI Security Generative AI Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.16.2 v8.17.1 v8.18.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@stephmilovic @elasticmachine @kibanamachine @e40pud