Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch from karma to web-test-runner #704

Merged
merged 12 commits into from
May 29, 2024
Merged

Switch from karma to web-test-runner #704

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
86c4ac7
Switch from karma to web-test-runner
fcollonval May 20, 2024
dded51c
Fix installing playwright browser
fcollonval May 20, 2024
7ba2cfa
Update doc and fix installing chromium
fcollonval May 20, 2024
91dbe24
Align browser name with playwright naming
fcollonval May 20, 2024
1217e9c
Fix inconsistency in commands package
fcollonval May 20, 2024
7ccc7bd
Add note on playwright cache
fcollonval May 24, 2024
7f6d202
Add note to install pw browsers
fcollonval May 24, 2024
a86de5f
Lint contributing.md
fcollonval May 24, 2024
53ba8be
Recommend npx rather than yarn to install web browsers
fcollonval May 24, 2024
69794dd
[skip ci] Apply suggestions from code review
fcollonval May 25, 2024
c43a568
Add playwright as root dev dependency
fcollonval May 25, 2024
e96e2e3
More robust tests
fcollonval May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,38 @@ on:
branches: [main]
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
# Use environment variable to tell Playwright where to install browser binaries so they can be cached for future workflow runs
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/pw-browsers
fcollonval marked this conversation as resolved.
Show resolved Hide resolved

jobs:
test:
name: JS
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
browser: [chrome-headless, firefox-headless, webkit-headless]
browser: [chromium, firefox, webkit]
exclude:
# Playwright webkit binary is not compatible with ubuntu
- os: ubuntu-latest
browser: webkit-headless
browser: webkit
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Set up browser cache
uses: actions/cache@v4
with:
path: |
${{ github.workspace }}/pw-browsers
key: ${{ matrix.os }}-${{ matrix.browser }}-${{ hashFiles('./yarn.lock') }}

- name: Install dependencies
shell: bash
run: |
Expand All @@ -34,7 +49,7 @@ jobs:
run: |
set -eux
yarn build
yarn build:test
npx playwright install ${{ matrix.browser }}
gabalafou marked this conversation as resolved.
Show resolved Hide resolved
fcollonval marked this conversation as resolved.
Show resolved Hide resolved

- name: Run Tests
run: |
Expand All @@ -43,9 +58,6 @@ jobs:
build:
runs-on: ubuntu-latest

env:
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/pw-browsers

steps:
- uses: actions/checkout@v4
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
Expand All @@ -55,7 +67,7 @@ jobs:
with:
path: |
${{ github.workspace }}/pw-browsers
key: ubuntu-latest-${{ hashFiles('./yarn.lock') }}
key: ubuntu-latest-chromium-${{ hashFiles('./yarn.lock') }}
gabalafou marked this conversation as resolved.
Show resolved Hide resolved

- name: Install dependencies
shell: bash
Expand Down
17 changes: 15 additions & 2 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,28 @@ yarn build:src

## Tests

The tests are written using karma to simulate a browser environment.
The tests are written using [web-test-runner](https://modern-web.dev/docs/test-runner/overview/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably in the section above we need to add a line for how to install the Playwright browser binaries, otherwise yarn test will not work.

We also need to specify which version of Yarn to use, because yarn playwright install worked for me in yarn v1 and v4 but not v3.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is weird that it does not work with v3.

Copy link
Contributor

@gabalafou gabalafou May 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I wonder if what I was seeing actually had more to do with my other comment (about putting @playwright/test in the top-level package.json)

to test in browser environment (provided by [playwright](https://playwright.dev/)).
fcollonval marked this conversation as resolved.
Show resolved Hide resolved

To run the tests, run:

```bash
yarn build:test
yarn test # optionally test:chrome, test:firefox, or test:webkit
yarn test # optionally test:chromium, test:firefox, or test:webkit
```

You can run the tests manually to debug them by going inside one package (e.g. `packages/application`)
and then executing `yarn test:debug`. This should open your browser
in which you will be able to select the test file to execute.
The outcome of the tests are displayed in the web browser console and you
can use the web browser debug capability to debug the tests.

> [!NOTE]
> The test files are transpiled and bundled before execution. Therefore
> when debugging in the browser, you must use the file `tests/lib/bundle.test.js`.
> Then you will need to apply the fix to the Typescript test file and re-run
> the test command (to trigger the bundling) to check the test is fixed.

## Examples

Lumino examples are in the `examples/` folder. To build and run an example:
Expand Down
2 changes: 1 addition & 1 deletion examples/example-dockpanel-amd/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "^1.29.0"
"@playwright/test": "^1.44.0"
fcollonval marked this conversation as resolved.
Show resolved Hide resolved
}
}
2 changes: 1 addition & 1 deletion examples/example-dockpanel-iife/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "^1.29.0"
"@playwright/test": "^1.44.0"
}
}
2 changes: 1 addition & 1 deletion examples/example-nested-dockpanel-amd/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "^1.29.0"
"@playwright/test": "^1.44.0"
}
}
11 changes: 4 additions & 7 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"clean": "lerna run clean",
"clean:examples": "lerna run clean --scope \"@lumino/example-*\"",
"clean:src": "lerna run clean --scope \"@lumino/!(test-|example-)*\"",
"clean:tests": "lerna run clean:tests",
"clean:test": "lerna run clean:test",
"predocs": "lerna run build:src",
"docs": "typedoc",
"eslint": "eslint --ext .js,.jsx,.ts,.tsx --cache --fix .",
Expand All @@ -31,14 +31,11 @@
"prettier:check": "prettier --check \"**/*{.ts,.tsx,.js,.jsx,.css,.json,.md}\"",
"publish": "npm run clean && npm run build:dist && node scripts/tag-versions.js && lerna publish --yes -m \"Publish\" from-package",
"remove:dependency": "remove-dependency",
"test": "lerna run test --scope \"@lumino/!(example-)*\"",
"test:chrome": "lerna run test:chrome",
"test:chrome-headless": "lerna run test:chrome-headless",
"test": "lerna run test --concurrency 1 --scope \"@lumino/!(example-)*\"",
"test:chromium": "lerna run test:chromium --concurrency 1",
"test:examples": "lerna run test --scope \"@lumino/example-*\"",
"test:firefox": "lerna run test:firefox",
"test:firefox-headless": "lerna run test:firefox-headless",
"test:firefox": "lerna run test:firefox --concurrency 1",
"test:webkit": "lerna run test:webkit --concurrency 1",
"test:webkit-headless": "lerna run test:webkit-headless --concurrency 1",
"update:dependency": "update-dependency --lerna",
"update:versions": "lerna version --no-push --no-git-tag-version",
"prepare": "husky install"
Expand Down
23 changes: 8 additions & 15 deletions packages/algorithm/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,14 @@
"build": "npm run build:src && rollup -c",
"build:src": "tsc --build",
"build:test": "npm run clean:test && tsc --build tests && cd tests && rollup -c",
"clean": "rimraf ./lib && rimraf *.tsbuildinfo && rimraf ./types && rimraf ./dist",
"clean": "rimraf ./lib *.tsbuildinfo ./types ./dist",
"clean:test": "rimraf tests/lib tests/tsconfig.tsbuildinfo",
"minimize": "terser dist/index.js -c -m --source-map \"content='dist/index.js.map',url='index.min.js.map'\" -o dist/index.min.js",
"test": "npm run test:firefox-headless",
"test:chrome": "cd tests && karma start --browsers=Chrome",
"test:chrome-headless": "cd tests && karma start --browsers=ChromeHeadless",
"test:firefox": "cd tests && karma start --browsers=Firefox",
"test:firefox-headless": "cd tests && yarn karma start --browsers=FirefoxHeadless",
"test:webkit": "cd tests && karma start --browsers=Webkit",
"test:webkit-headless": "cd tests && karma start --browsers=WebkitHeadless",
"test": "npm run build:test && web-test-runner tests/lib/bundle.test.js --node-resolve --playwright",
"test:debug": "npm run test -- --manual --open",
"test:chromium": "npm run test -- --browsers chromium",
"test:firefox": "npm run test -- --browsers firefox",
"test:webkit": "npm run test -- --browsers webkit",
"watch": "tsc --build --watch"
},
"devDependencies": {
Expand All @@ -46,15 +44,10 @@
"@rollup/plugin-node-resolve": "^15.0.1",
"@types/chai": "^4.3.4",
"@types/mocha": "^2.2.39",
"@web/test-runner": "^0.18.2",
"@web/test-runner-playwright": "^0.11.0",
"chai": "^4.3.4",
"karma": "^6.3.4",
"karma-chrome-launcher": "^3.1.0",
"karma-firefox-launcher": "^2.1.1",
"karma-mocha": "^2.0.1",
"karma-mocha-reporter": "^2.2.5",
"karma-webkit-launcher": "^1.0.2",
"mocha": "^9.0.3",
"playwright": "^1.35.0",
"postcss": "^8.4.24",
"rimraf": "^5.0.1",
"rollup": "^3.25.1",
Expand Down
24 changes: 0 additions & 24 deletions packages/algorithm/tests/karma.conf.js
View file
Open in desktop

This file was deleted.

23 changes: 8 additions & 15 deletions packages/application/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,14 @@
"build": "npm run build:src && rollup -c",
"build:src": "tsc --build",
"build:test": "npm run clean:test && tsc --build tests && cd tests && rollup -c",
"clean": "rimraf ./lib && rimraf *.tsbuildinfo && rimraf ./types && rimraf ./dist",
"clean": "rimraf ./lib *.tsbuildinfo ./types ./dist",
"clean:test": "rimraf tests/lib tests/tsconfig.tsbuildinfo",
"minimize": "terser dist/index.js -c -m --source-map \"content='dist/index.js.map',url='index.min.js.map'\" -o dist/index.min.js",
"test": "npm run test:firefox-headless",
"test:chrome": "cd tests && karma start --browsers=Chrome",
"test:chrome-headless": "cd tests && karma start --browsers=ChromeHeadless",
"test:firefox": "cd tests && karma start --browsers=Firefox",
"test:firefox-headless": "cd tests && karma start --browsers=FirefoxHeadless",
"test:webkit": "cd tests && karma start --browsers=Webkit",
"test:webkit-headless": "cd tests && karma start --browsers=WebkitHeadless",
"test": "npm run build:test && web-test-runner tests/lib/bundle.test.js --node-resolve --playwright",
"test:debug": "npm run test -- --manual --open",
"test:chromium": "npm run test -- --browsers chromium",
"test:firefox": "npm run test -- --browsers firefox",
"test:webkit": "npm run test -- --browsers webkit",
"watch": "tsc --build --watch"
},
"dependencies": {
Expand All @@ -51,15 +49,10 @@
"@rollup/plugin-node-resolve": "^15.0.1",
"@types/chai": "^3.4.35",
"@types/mocha": "^2.2.39",
"@web/test-runner": "^0.18.2",
"@web/test-runner-playwright": "^0.11.0",
"chai": "^4.3.4",
"karma": "^6.3.4",
"karma-chrome-launcher": "^3.1.0",
"karma-firefox-launcher": "^2.1.1",
"karma-mocha": "^2.0.1",
"karma-mocha-reporter": "^2.2.5",
"karma-webkit-launcher": "^1.0.2",
"mocha": "^9.0.3",
"playwright": "^1.35.0",
"postcss": "^8.4.24",
"rimraf": "^5.0.1",
"rollup": "^3.25.1",
Expand Down
24 changes: 0 additions & 24 deletions packages/application/tests/karma.conf.js
View file
Open in desktop

This file was deleted.

23 changes: 8 additions & 15 deletions packages/collections/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,14 @@
"build": "npm run build:src && rollup -c",
"build:src": "tsc --build",
"build:test": "npm run clean:test && tsc --build tests && cd tests && rollup -c",
"clean": "rimraf ./lib && rimraf *.tsbuildinfo && rimraf ./types && rimraf ./dist",
"clean": "rimraf ./lib *.tsbuildinfo ./types ./dist",
"clean:test": "rimraf tests/lib tests/tsconfig.tsbuildinfo",
"minimize": "terser dist/index.js -c -m --source-map \"content='dist/index.js.map',url='index.min.js.map'\" -o dist/index.min.js",
"test": "npm run test:firefox-headless",
"test:chrome": "cd tests && karma start --browsers=Chrome",
"test:chrome-headless": "cd tests && karma start --browsers=ChromeHeadless",
"test:firefox": "cd tests && karma start --browsers=Firefox",
"test:firefox-headless": "cd tests && karma start --browsers=FirefoxHeadless",
"test:webkit": "cd tests && karma start --browsers=Webkit",
"test:webkit-headless": "cd tests && karma start --browsers=WebkitHeadless",
"test": "npm run build:test && web-test-runner tests/lib/bundle.test.js --node-resolve --playwright",
"test:debug": "npm run test -- --manual --open",
"test:chromium": "npm run test -- --browsers chromium",
"test:firefox": "npm run test -- --browsers firefox",
"test:webkit": "npm run test -- --browsers webkit",
"watch": "tsc --build --watch"
},
"dependencies": {
Expand All @@ -49,15 +47,10 @@
"@rollup/plugin-node-resolve": "^15.0.1",
"@types/chai": "^3.4.35",
"@types/mocha": "^2.2.39",
"@web/test-runner": "^0.18.2",
"@web/test-runner-playwright": "^0.11.0",
"chai": "^4.3.4",
"karma": "^6.3.4",
"karma-chrome-launcher": "^3.1.0",
"karma-firefox-launcher": "^2.1.1",
"karma-mocha": "^2.0.1",
"karma-mocha-reporter": "^2.2.5",
"karma-webkit-launcher": "^1.0.2",
"mocha": "^9.0.3",
"playwright": "^1.35.0",
"postcss": "^8.4.24",
"rimraf": "^5.0.1",
"rollup": "^3.25.1",
Expand Down
24 changes: 0 additions & 24 deletions packages/collections/tests/karma.conf.js
View file
Open in desktop

This file was deleted.

30 changes: 8 additions & 22 deletions packages/commands/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,23 +27,14 @@
"build": "npm run build:src && rollup -c",
"build:src": "tsc --build",
"build:test": "npm run clean:test && tsc --build tests && cd tests && rollup -c",
"clean": "rimraf ./lib && rimraf *.tsbuildinfo && rimraf ./types && rimraf ./dist",
"clean": "rimraf ./lib *.tsbuildinfo ./types ./dist",
"clean:test": "rimraf tests/lib tests/tsconfig.tsbuildinfo",
"minimize": "terser dist/index.js -c -m --source-map \"content='dist/index.js.map',url='index.min.js.map'\" -o dist/index.min.js",
"test": "npm run test:firefox-headless",
"test:chrome": "npm run test:nobrowser -- --browsers=Chrome",
"test:chrome-headless": "npm run test:nobrowser -- --browsers=ChromeHeadless",
"test:debug": "npm run test:debug:firefox",
"test:debug:chrome": "npm run test:debug:nobrowser -- --browsers=Chrome",
"test:debug:chrome-headless": "npm run test:debug:nobrowser -- --browsers=ChromeHeadless",
"test:debug:firefox": "npm run test:debug:nobrowser -- --browsers=Firefox",
"test:debug:firefox-headless": "npm run test:debug:nobrowser -- --browsers=FirefoxHeadless",
"test:debug:nobrowser": "cd tests && karma start --singleRun=false --debug=true --browserNoActivityTimeout=10000000 --browserDisconnectTimeout=10000000",
"test:firefox": "npm run test:nobrowser -- --browsers=Firefox",
"test:firefox-headless": "npm run test:nobrowser -- --browsers=FirefoxHeadless",
"test:webkit": "cd tests && karma start --browsers=Webkit",
"test:webkit-headless": "cd tests && karma start --browsers=WebkitHeadless",
"test:nobrowser": "cd tests && karma start",
"test": "npm run build:test && web-test-runner tests/lib/bundle.test.js --node-resolve --playwright",
"test:debug": "npm run test -- --manual --open",
"test:chromium": "npm run test -- --browsers chromium",
"test:firefox": "npm run test -- --browsers firefox",
"test:webkit": "npm run test -- --browsers webkit",
"watch": "tsc --build --watch"
},
"dependencies": {
Expand All @@ -62,15 +53,10 @@
"@rollup/plugin-node-resolve": "^15.0.1",
"@types/chai": "^3.4.35",
"@types/mocha": "^2.2.39",
"@web/test-runner": "^0.18.2",
"@web/test-runner-playwright": "^0.11.0",
"chai": "^4.3.4",
"karma": "^6.3.4",
"karma-chrome-launcher": "^3.1.0",
"karma-firefox-launcher": "^2.1.1",
"karma-mocha": "^2.0.1",
"karma-mocha-reporter": "^2.2.5",
"karma-webkit-launcher": "^1.0.2",
"mocha": "^9.0.3",
"playwright": "^1.35.0",
"postcss": "^8.4.24",
"rimraf": "^5.0.1",
"rollup": "^3.25.1",
Expand Down
Loading
Loading