Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests(unit): retry failures and upload failure artifacts #15378

Merged
merged 5 commits into from
Aug 15, 2023

Conversation

adamraine
Copy link
Member

@adamraine adamraine commented Aug 14, 2023

Unit failures are only happening in CI, failure artifacts might give us more insight. Retry unit tests 5 times to unblock CI.

CI failures added to backlog: #15382

@adamraine adamraine changed the title WIP: upload unit test failure artifacts tests(unit): retry failures and upload failure artifacts Aug 15, 2023
@adamraine adamraine marked this pull request as ready for review August 15, 2023 21:11
@adamraine adamraine requested a review from a team as a code owner August 15, 2023 21:11
@adamraine adamraine requested review from brendankenny and removed request for a team August 15, 2023 21:11
@@ -56,7 +56,7 @@
"unit-viewer": "yarn mocha --testMatch viewer/**/*-test.js",
"unit-flow": "bash flow-report/test/run-flow-report-tests.sh",
"unit": "yarn unit-flow && yarn mocha",
"unit:ci": "NODE_OPTIONS=--max-old-space-size=8192 npm run unit --forbid-only",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh. we need to support this in the script. can you not remove ?

Copy link
Member Author

@adamraine adamraine Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --forbid-only flag doesn't work when we use it like this. This PR adds it to run-mocha-tests.js and gives it a default value of true in CI instead.

Copy link
Collaborator

@connorjclark connorjclark Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean it doesn't work when used like this?

I know it doesn't work at all w/o this PR or #15383 , but I don't understand your meaning here

Copy link
Member Author

@adamraine adamraine Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two reasons:

  • Before this PR we didn't pipe the --forbid-only option into mocha
  • We would need to do npm run unit -- --forbid-ony to pass the flag all the way down

Instead of messing with the flags in the package scripts, I would prefer to just look at process.env.CI. We already do this for --parallel in run-mocha-tests.js:

// Although much faster, mocha's parallel test runner defers printing errors until
// all tests have finished. This may be undesired for local development, so enable
// parallel mode by default only in CI.
// Also, good to default to false locally because that avoids missing cross-file
// test contamination by chance of mocha splitting up the work in a way that hides it.
default: Boolean(process.env.CI),

@adamraine adamraine merged commit f8750cf into main Aug 15, 2023
27 checks passed
@adamraine adamraine deleted the upload-unit-failures branch August 15, 2023 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants