Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure Conformance API tests to match Spec #6978

Closed
wants to merge 4 commits into from

Conversation

coryrc
Copy link
Contributor

@coryrc coryrc commented Feb 23, 2020

Fixes #6006 by making TestCustomResourceLimits to be a conformance
test and moving e2e parts out.

For #6226 fixes it for TestContainerErrorMsg and
TestContainerExitingMsg, though could be other tests checking errors
incorrectly.

Addresses #6264 though continuing work is needed.

Don't mix E2E and conformance:
Strip E2E portions of conformance out into tests in the test/e2e
directory. Commonality goes into test/scenarios or test/v1 depending
on specificity of the common functionality.

Rename to more align with runtime test naming (Must/Should).

NONE

/cc @dgerd

@googlebot googlebot added the cla: yes Indicates the PR's author has signed the CLA. label Feb 23, 2020
@knative-prow-robot knative-prow-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Feb 23, 2020
@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: coryrc
To complete the pull request process, please assign markusthoemmes
You can assign the PR to them by writing /assign @markusthoemmes in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Contributor

@knative-prow-robot knative-prow-robot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coryrc: 7 warnings.

In response to this:

Fixes #6006 by making TestCustomResourceLimits to be a conformance
test and moving e2e parts out.

For #6226 fixes it for TestContainerErrorMsg and
TestContainerExitingMsg, though could be other tests checking errors
incorrectly.

Addresses #6264 though continuing work is needed.

Don't mix E2E and conformance:
Strip E2E portions of conformance out into tests in the test/e2e
directory. Commonality goes into test/scenarios or test/v1 depending
on specificity of the common functionality.

Rename to more align with runtime test naming (Must/Should).

NONE

/cc @dgerd

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

test/v1/condition.go Show resolved Hide resolved
test/scenarios/container_failures.go Show resolved Hide resolved
test/scenarios/container_failures.go Outdated Show resolved Hide resolved
test/scenarios/container_failures.go Outdated Show resolved Hide resolved
test/scenarios/container_failures.go Show resolved Hide resolved
test/scenarios/container_failures.go Show resolved Hide resolved
test/v1/constants.go Show resolved Hide resolved
@knative-prow-robot knative-prow-robot added the area/test-and-release It flags unit/e2e/conformance/perf test issues for product features label Feb 23, 2020
@coryrc
Copy link
Contributor Author

coryrc commented Feb 28, 2020

/retest

@knative-prow-robot knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 28, 2020
Fixes knative#6006 by making TestCustomResourceLimits to be a conformance
test and moving e2e parts out.

For knative#6226 fixes it for TestContainerErrorMsg and
TestContainerExitingMsg, though could be other tests checking errors
incorrectly.

Addresses knative#6264 though continuing work is needed.

Don't mix E2E and conformance:
Strip E2E portions of conformance out into tests in the test/e2e
directory. Commonality goes into test/scenarios or test/v1 depending
on specificity of the common functionality.

Rename to more align with runtime test naming (Must/Should).
@knative-prow-robot knative-prow-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 28, 2020
@knative-prow-robot knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 27, 2020
@knative-prow-robot
Copy link
Contributor

@coryrc: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@markusthoemmes
Copy link
Contributor

@coryrc any news on this one?

@coryrc
Copy link
Contributor Author

coryrc commented May 4, 2020

@markusthoemmes If someone who can approve cares, I'm willing to put in the work to get this in.

@markusthoemmes
Copy link
Contributor

/assign @dprotaso

I guess it's up for the API WG to decide what to do with this.

@dprotaso
Copy link
Member

@markusthoemmes If someone who can approve cares, I'm willing to put in the work to get this in.

I think there's value in this work. If you need timely approval feel free to ping me on slack.

At minimum if you're timed constrained on working on these changes do you at least mind re-opening and the relevant issues you care about?

@coryrc
Copy link
Contributor Author

coryrc commented May 28, 2020

Comments in #6264
I will be unavailable to work on this for a while. Perhaps it can serve as a model for others.

@coryrc coryrc closed this May 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features cla: yes Indicates the PR's author has signed the CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tighter Acceptance Criteria in TestCustomResources
5 participants