The following guide will walk through the necessary instruction about driving an installation of RHTAP locally or in Openshift CI.
Requirements for installing RHTAP in E2E mode:
- An OpenShift 4.12 or higher Environment (If you are using CRC/OpenShift Local please also review optional-codeready-containers-post-bootstrap-configuration)
- A machine from which to run the install (usually your laptop) with required tools:
- A properly setup Go workspace using Go 1.19 is required
- The OpenShift Command Line Tool (oc) Use the version corresponding to the Openshift version
- yq
- jq
- git
- Tokens
- Github Token with the following permissions
repo
delete_repo
workflow
- Valid quay token where to push RHTAP components images generated by the e2e framework
- Github Token with the following permissions
-
Before deploying RHTAP in E2E mode you need to login to your OpenShift cluster with OpenShift Command Line Tool as
admin
(by defaultkubeadmin
):oc login -u <user> -p <password> --server=<oc_api_url>
-
Export required (and recommended) environment variables from default.env. Copy the file (
cp default.env user.env
), edit the required variables and source it (source user.env
). -
Install dependencies:
# Install dependencies
$ go mod tidy
# or go mod tidy -compat=1.21
- By default the installation script will use the
redhat-appstudio-qe
GitHub organization for pushing changes toinfra-deployments
repository.
It is recommended to use your fork of infra-deployments repo in your GitHub org instead - you can change the GitHub organization with environment variable export MY_GITHUB_ORG=<name-of-your-github-org>
.
Some tests could require you have a Github App created in order to test Component builds via Pipelines as Code. Such tests are rhtap-demo, build, and status-reporting-to-pullrequest.
In this case, before you bootstrap a cluster, make sure you created a Github App for your GitHub account. Fill in following details:
- GitHub App name: unique app name
- Homepage URL: some dummy URL (like https://example.com)
- Webhook: mark as active and put some dummy URL to the Webhook URL field
- Permissions and Subscribe to events: refer to the guide in PaC documentation
- Where can this GitHub App be installed>: Any account
Hit create, make a note of the App ID and navigate to the Private keys section where you generate a private key that gets downloaded automatically. Then export following environment variables (or if you're using .env file, update values of following variables):
export E2E_PAC_GITHUB_APP_ID='<YOUR_APP_ID>'
export E2E_PAC_GITHUB_APP_PRIVATE_KEY=$(base64 < /PATH/TO/YOUR/DOWNLOADED/PRIVATE_KEY.pem)
Navigate back to your GitHub App, select Install App and select your GitHub org (the one that you're using in `MY_GITHUB_ORG` env var). Feel free to install it to all repositories of that organization or the forked repositories currently used by rhtap-demo and build tests
For bootstrapping a cluster, run the following command:
make local/cluster/prepare
More information about how to deploy RHTAP are in the infra-deployments repository.
Most of the tests could require you to have specific container image repo's created (if you're using your own container image org/user account (QUAY_E2E_ORGANIZATION
) or your own GitHub organization (MY_GITHUB_ORG
).
In that case, before you run the test, make sure you have created
test-images
repo in quay.io, i.e.quay.io/<QUAY_E2E_ORGANIZATION>/test-images
and make it public (this repo will be used for pushing container images produced by tests)- also make sure that the docker config, that is encoded in the value of
QUAY_TOKEN
environment variable, contains a correct credentials required to push totest-images
repo. And make sure the robot account or user account has the write permissions set fortest-images
repo which is required by the tests to push the generated artifacts.
- also make sure that the docker config, that is encoded in the value of
- fork following GitHub repositories to your org (specified in
MY_GITHUB_ORG
env var)- https://github.com/redhat-appstudio-qe/devfile-sample-hello-world (for running build-service tests)
- https://github.com/redhat-appstudio-qe/hacbs-test-project (for rhtap-demo test)
- https://github.com/redhat-appstudio-qe/strategy-configs (for rhtap-demo test)
- https://github.com/redhat-appstudio-qe/hacbs-test-project-integration (for status-reporting-to-pullrequest test)
Note: All Environments used in all e2e-tests are in default.env file. In case you need to run a specific tests, not all environments are necessary to be defined.
You can use the following make target to build and run the tests:
make local/test/e2e
Or build and run the tests without scripts:
-
Install dependencies and build the tests:
# Install dependencies $ go mod tidy # Create `e2e-appstudio` binary in bin folder. Please add the binary to the path or just execute `./bin/e2e-appstudio` $ make build
-
Run the e2e tests: The
e2e-appstudio
command is the root command that executes all test functionality. To obtain all available flags for the binary please use--help
flags. All ginkgo flags and go tests are available ine2e-appstudio
binary.`./bin/e2e-appstudio`
Note: The binary must be updated by running make build
every time there are new changes in the tests.
The instructions for every test suite can be found in the tests folder, e.g. has Readme.md. You can also specify which tests you want to run using labels or Ginkgo Focus.
The e2e tests are executed against almost all AppStudio repositories.
Sometimes when we have changes in AppStudio we are introducing some breaking changes and the e2e will fail. To prevent this the e2e framework installation in openshift-ci support a new feature of pairing the PR opened against an AppStudio repository with a corresponding PR opened against e2e-tests (based on branch names). Before the e2e framework will be executed in openshift-ci, the logic automatically tries to pair a PR opened in some repo with a branch of the same name that potentially could exists in the developer's fork of the e2e repository
For example, if a developer with GH account cooljohn
opens a PR (for application-service repo) from a branch new-feature
, then the logic checks if there is a PR open in e2e-tests from the new-feature
branch in the cooljohn/e2e-tests
fork. If such a PR is found, the CI will install the e2e framework from that branch.
Once the service PR passes CI with its pairing and is merged, the next step is to get the references in infra-deployments updated. Many services have automation to create PRs in infra-deployments to update their component to the latest version. However, this recently merged latest version will likely need the still unmerged e2e-tests update. The solution is to manually create a PR to infra-deployments that bumps the component version with the same branch name. This will allow the infra-deployments CI to run with your branch of e2e-tests, allowing the CI to pass. Finally, once the infra-deployments CI passes and PR merges, you should be able to rerun your e2e-tests PR and have it pass and merge.
To continue on the previous example, once the application-service PR merges, cooljohn
should open a pull request to infra-deployments using the branch new-feature
to bump the application-service version to the one recently created by the application-service PR merging. This lets the infra-deployments CI run with the required
e2e-tests changes from cooljohn
's fork.
Pairing PRs is handled automatically by running this command from a root directory of this repository:
make ci/test/e2e