Skip to content

Commit

Permalink
Serverless API integration tests (elastic#192219)
Browse files Browse the repository at this point in the history
Tests for serverless

- copies over and modifies all tests from stateful to work in
serverless. ~~deployment agnostic tests do not yet support enterprise
license for stateful, so are tests don't yet qualify as being deployment
agnostic~~. Given how difficult it is to see differences from the
stateful tests, I've added PR comments where I've changed something that
might be of interest.
- changes to `createObservabilityAIAssistantApiClient` to use supertest
without basic auth and accept headers for serverless and use roles
- removes creating persisted users when tests start and [use
roles](https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/README.md#roles-based-testing)
within tests. its not possible to create custom users with the
serverless test framework at the moment. See
elastic#192711

Skipped tests
- knowledge base tests elastic#192886
- any test suite that uses the LLM proxy has been skipped on MKI
elastic#192751
- all tests that depend on the config.modelId skipped in MKI
elastic#192757

TODO:

- [x] move over remaining tests
- [x]  test in MKI environment before merging
- [x] create issues for skipped tests
- [ ] this will not run on MKI (after merging) unless we ping the
appex-qa team to add it to the pipeline. this is due to creating a
separate config. ask appex-qa team to add our config.

Followup / related issues to be tracked in a newly created issue:

- [ ] elastic#192757
- [ ] elastic#192886
- [ ] elastic#192751
- [ ] elastic#192701
- [ ] elastic#192497
- [ ] elastic#192711
- [ ] elastic#192718
- [ ] serverless functional tests
- [ ] inquire with ml-ui-team to have the ability to delete system
indices which we do after uninstalling tiny elser with .ml indices
  • Loading branch information
neptunian committed Oct 1, 2024
1 parent 00bc980 commit e9aed95
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions .buildkite/ftr_oblt_serverless_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ disabled:
- x-pack/test_serverless/api_integration/test_suites/observability/config.feature_flags.ts
- x-pack/test_serverless/api_integration/test_suites/observability/common_configs/config.group1.ts
- x-pack/test_serverless/api_integration/test_suites/observability/fleet/config.ts
- x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/config.ts
- x-pack/test_serverless/functional/test_suites/observability/config.ts
- x-pack/test_serverless/functional/test_suites/observability/config.examples.ts
- x-pack/test_serverless/functional/test_suites/observability/config.feature_flags.ts
Expand Down

0 comments on commit e9aed95

Please sign in to comment.