[Security Solution][SIEM migrations] Onboarding UI improvements #204320

semd · 2024-12-14T16:28:21Z

Summary

Part of: https://github.com/elastic/security-team/issues/10667

Improvements

Implementation of the Onboarding card to create migrations using the flyout
Migration complete summary panel implemented
Migration ready panel improved to detect missing resources
Migration processing improved
Migration missing resources panel implemented
All migration panels and refactored to be reusable by translation table using the
RuleMigrationDataInputWrapper implemented to reuse the Flyout from the translation table
Request poll interval increased from 5 to 10 seconds due to event loop usage.

Note

This feature needs siemMigrationsEnabled experimental flag enabled to work.

Screenshots

Lookups input

Translation "complete" panel

Translation "created" panel (w/ and w/o missing macros)

Translation processing (preparing)

Translation processing (translating)

…ns/onboarding_ui

…boarding_ui

… request polling

…_ui' into 10667/siem_migrations/onboarding_ui_flyout

…grations/onboarding_ui_flyout

…ns/onboarding_ui_flyout

…-fix'

…ns/onboarding_ui_flyout' into 10667/siem_migrations/onboarding_ui_flyout

elasticmachine · 2024-12-16T20:08:53Z

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

elasticmachine · 2024-12-16T20:08:55Z

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine · 2024-12-16T22:01:24Z

💚 Build Succeeded

Buildkite Build
Commit: 823d5fb

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	6410	6430	+20

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	14.7MB	14.8MB	+16.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	88.0KB	88.0KB	+1.0B

Unknown metric groups

async chunk count

id	before	after	diff
`securitySolution`	105	106	+1

References to deprecated APIs

id	before	after	diff
`securitySolution`	491	489	-2

History

cc @semd

e40pud · 2024-12-16T22:47:57Z

...migrations/rules/task/agent/sub_graphs/translate_rule/nodes/translate_rule/translate_rule.ts

@@ -63,9 +63,9 @@ export const getTranslateRuleNode = ({
  };
 };

-const getTranslationResult = (esqlQuery: string): SiemMigrationRuleTranslationResult => {
+const getTranslationResult = (esqlQuery: string): RuleTranslationResult => {


It looks like this method is equivalent of another function in x-pack/plugins/security_solution/server/lib/siem_migrations/rules/task/agent/sub_graphs/translate_rule/nodes/ecs_mapping/ecs_mapping.ts. Maybe we could re-use this code

Yes, noted. I am creating a list of things to do on the agent side, we can do it in a separate PR.

e40pud

Tested these changes locally - everything looks great!

angorayc

nit:

It cannot be unmarked after marking the lookup as empty
Would be nice to indicate that the lookup file is a CSV (as previous uploaded files are JSON)
Delete an added migration.

...utions/security/plugins/security_solution/public/common/components/panel_text/panel_text.tsx

angorayc · 2024-12-17T11:37:06Z

...boarding/components/onboarding_body/cards/siem_migrations/ai_connector/ai_connector_card.tsx

-  const connectors = checkCompleteMetadata?.connectors;
-  const canExecuteConnectors = checkCompleteMetadata?.canExecuteConnectors;
-  const canCreateConnectors = checkCompleteMetadata?.canCreateConnectors;
+  if (!checkCompleteMetadata) {


semd · 2024-12-17T12:53:52Z

@angorayc Thanks a lot for the review, about the questions:

It cannot be unmarked after marking the lookup as empty

After marking a lookup as empty, users can still copy the name and upload the file, if it's provided it's unmarked:

Would be nice to indicate that the lookup file is a CSV (as previous uploaded files are JSON)

We assume it's a CSV for now, but we'll need to support both formats for the MVP (8.18/9.0). I will add specific parsers in the following PRs

Delete an added migration.

That won't be possible in the MVP.

elasticmachine · 2024-12-17T14:42:04Z

💚 Build Succeeded

Buildkite Build
Commit: 4aaa575

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	6413	6432	+19

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	14.7MB	14.8MB	+15.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	88.0KB	88.0KB	+1.0B

Unknown metric groups

async chunk count

id	before	after	diff
`securitySolution`	105	106	+1

References to deprecated APIs

id	before	after	diff
`securitySolution`	491	489	-2

History

💚 Build #261099 succeeded 823d5fb
💔 Build #261085 failed b4c855b
💔 Build #260955 failed 5864495
💔 Build #260594 failed 9c4b257

cc @semd

kibanamachine · 2024-12-17T14:42:20Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12375309176

kibanamachine · 2024-12-17T14:47:22Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - Update data archives for SO integration tests (#204561) - SKA: Relocate script v5.1 (#204563) - [ML] Transforms: Support wildcards in the alerting rule flyout (#204226) - [Search] Deprecate canDeployEntSearch config value (#204272) - [Fleet] Display next steps and actions in agentless integrations flyout (#203824) - [devx] Create script to stage files by CODEOWNER (#203940) - [ai][assistant] Refactor o11y to use new Assistant logo and beacon (#204289) - [Security Solution] Preview navigation in document details flyout (#204292) - Fix generation of dynamic mapping for object with specific subfield (#204104) - Sustainable Kibana Architecture: Move modules owned by `@elastic/kibana-cloud-security-posture` (#202862)

Manual backport

To create the backport manually run:

node scripts/backport --pr 204320

Questions ?

Please refer to the Backport tool documentation

semd · 2024-12-18T11:05:09Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…tic#204320) ## Summary Part of: elastic/security-team#10667 #### Improvements - Implementation of the Onboarding card to create migrations using the flyout - Migration complete summary panel implemented - Migration ready panel improved to detect missing resources - Migration processing improved - Migration missing resources panel implemented - All migration panels and refactored to be reusable by translation table using the - `RuleMigrationDataInputWrapper` implemented to reuse the Flyout from the translation table - Request poll interval increased from 5 to 10 seconds due to event loop usage. > [!NOTE] > This feature needs `siemMigrationsEnabled` experimental flag enabled to work. ## Screenshots Lookups input ![Lookups](https://github.com/user-attachments/assets/73f91e10-7252-44d1-ab0d-89880c78a2b3) Translation "complete" panel ![Translation summary](https://github.com/user-attachments/assets/6fbb451d-c7b3-4a23-a2df-083c91948cbd) Translation "created" panel (w/ and w/o missing macros) ![Ready panels](https://github.com/user-attachments/assets/f8334af2-ccc1-473c-8548-772a9d656aba) Translation processing (preparing) ![preparing panel](https://github.com/user-attachments/assets/0156caba-c6c9-43c1-881a-8bf631f3a8ab) Translation processing (translating) ![translating panel](https://github.com/user-attachments/assets/db523e4b-4858-482f-bfe9-1e36f715fa20) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 303eee8)

kibanamachine · 2024-12-19T14:48:35Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…tic#204320) ## Summary Part of: elastic/security-team#10667 #### Improvements - Implementation of the Onboarding card to create migrations using the flyout - Migration complete summary panel implemented - Migration ready panel improved to detect missing resources - Migration processing improved - Migration missing resources panel implemented - All migration panels and refactored to be reusable by translation table using the - `RuleMigrationDataInputWrapper` implemented to reuse the Flyout from the translation table - Request poll interval increased from 5 to 10 seconds due to event loop usage. > [!NOTE] > This feature needs `siemMigrationsEnabled` experimental flag enabled to work. ## Screenshots Lookups input ![Lookups](https://github.com/user-attachments/assets/73f91e10-7252-44d1-ab0d-89880c78a2b3) Translation "complete" panel ![Translation summary](https://github.com/user-attachments/assets/6fbb451d-c7b3-4a23-a2df-083c91948cbd) Translation "created" panel (w/ and w/o missing macros) ![Ready panels](https://github.com/user-attachments/assets/f8334af2-ccc1-473c-8548-772a9d656aba) Translation processing (preparing) ![preparing panel](https://github.com/user-attachments/assets/0156caba-c6c9-43c1-881a-8bf631f3a8ab) Translation processing (translating) ![translating panel](https://github.com/user-attachments/assets/db523e4b-4858-482f-bfe9-1e36f715fa20) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

kibanamachine · 2024-12-20T15:48:43Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

semd and others added 30 commits November 22, 2024 15:53

add support for topics on the onboarding page

60fbe6f

reorganize directories inside components

3985477

solve conflict

03b5034

Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…

376f2d6

…ns/onboarding_ui

Merge remote-tracking branch 'upstream' into 10667/siem_migrations/on…

318d79e

…boarding_ui

resources schema id

8557994

update and add tests

da3938c

resources type fix

132c67b

add connector selection and auto-resume stopped migrations within the…

94c56c8

… request polling

Merge remote-tracking branch 'origin/10667/siem_migrations/onboarding…

a0702c9

…_ui' into 10667/siem_migrations/onboarding_ui_flyout

Merge branch 'main' into 10667/siem_migrations/onboarding_ui

1d1f1da

initial implementation

cee50d7

extract common styles into single places

5a9a06a

format code

f2beaf3

Merge branch '10667/siem_migrations/onboarding_ui' into 10667/siem_mi…

9f305c9

…grations/onboarding_ui_flyout

conflict solve

4ce992d

cleaning

8b60935

solve conflict

f09344b

upload panel styles

ef6c17d

flyout scafolding

5cb839d

solve conflicts

ecadb11

add upload input

d79dc47

basic workflow working

a486424

Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…

0ce93d9

…ns/onboarding_ui_flyout

no connector selected callout

b02a569

solve conflicts

216a6dc

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

0d6ae1f

…-fix'

add langsmith trace options

6e63ffa

Merge remote-tracking branch 'refs/remotes/origin/10667/siem_migratio…

86210fc

…ns/onboarding_ui_flyout' into 10667/siem_migrations/onboarding_ui_flyout

lint fixes

ac6e675

semd requested review from a team as code owners December 16, 2024 20:08

fix translations

823d5fb

e40pud reviewed Dec 17, 2024

View reviewed changes

e40pud approved these changes Dec 17, 2024

View reviewed changes

semd added 2 commits December 17, 2024 11:59

fix relocation conflicts

beff67b

theme aware vis colors

6e3b4e2

angorayc approved these changes Dec 17, 2024

View reviewed changes

vis dark mode

4a50aa9

semd added 2 commits December 17, 2024 13:59

address PR comments

39e61e2

allow mark as empty when provided

4aaa575

semd enabled auto-merge (squash) December 17, 2024 13:02

semd merged commit 303eee8 into elastic:main Dec 17, 2024
8 checks passed

kibanamachine mentioned this pull request Dec 17, 2024

[EDR Workflows] CrowdStrike RunScript: Log Actions and UI Output #204044

Merged

4 tasks

semd mentioned this pull request Dec 18, 2024

[8.x] [Security Solution][SIEM migrations] Onboarding UI improvements (#204320) #204719

Open

kibanamachine mentioned this pull request Dec 18, 2024

Fix context.pageName by fixing missing executionContext and add enableExecutionContextTracking flag #204547

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Dec 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][SIEM migrations] Onboarding UI improvements #204320

[Security Solution][SIEM migrations] Onboarding UI improvements #204320

semd commented Dec 14, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Dec 16, 2024

elasticmachine commented Dec 16, 2024

elasticmachine commented Dec 16, 2024

async chunk count

References to deprecated APIs

e40pud Dec 16, 2024

semd Dec 17, 2024

e40pud left a comment

angorayc left a comment •

edited

Loading

angorayc Dec 17, 2024

semd commented Dec 17, 2024

elasticmachine commented Dec 17, 2024

async chunk count

References to deprecated APIs

kibanamachine commented Dec 17, 2024

kibanamachine commented Dec 17, 2024

semd commented Dec 18, 2024

kibanamachine commented Dec 19, 2024

kibanamachine commented Dec 20, 2024

[Security Solution][SIEM migrations] Onboarding UI improvements #204320

[Security Solution][SIEM migrations] Onboarding UI improvements #204320

Conversation

semd commented Dec 14, 2024 • edited by kibanamachine Loading

Summary

Improvements

Screenshots

elasticmachine commented Dec 16, 2024

elasticmachine commented Dec 16, 2024

elasticmachine commented Dec 16, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

async chunk count

References to deprecated APIs

History

e40pud Dec 16, 2024

Choose a reason for hiding this comment

semd Dec 17, 2024

Choose a reason for hiding this comment

e40pud left a comment

Choose a reason for hiding this comment

angorayc left a comment • edited Loading

Choose a reason for hiding this comment

angorayc Dec 17, 2024

Choose a reason for hiding this comment

semd commented Dec 17, 2024

elasticmachine commented Dec 17, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

async chunk count

References to deprecated APIs

History

kibanamachine commented Dec 17, 2024

kibanamachine commented Dec 17, 2024

💔 All backports failed

Manual backport

Questions ?

semd commented Dec 18, 2024

💚 All backports created successfully

Questions ?

kibanamachine commented Dec 19, 2024

kibanamachine commented Dec 20, 2024

semd commented Dec 14, 2024 •

edited by kibanamachine

Loading

angorayc left a comment •

edited

Loading