Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[testdriver.js] Internal testdriver messages exposed to onmessage handlers #48326

Closed
jonathan-j-lee opened this issue Sep 24, 2024 · 1 comment · Fixed by #48433
Closed

[testdriver.js] Internal testdriver messages exposed to onmessage handlers #48326

jonathan-j-lee opened this issue Sep 24, 2024 · 1 comment · Fixed by #48433

Comments

@jonathan-j-lee
Copy link
Contributor

In cross-origin cases, the WebDriver-based testdriver.js implementation uses postMessage({type: "testdriver-{command,complete}", ...}) to communicate with the wptrunner harness. However, these internal messages are exposed to all message event handlers on the receiving end. This can trip up test-registered handlers that don't expect extra incoming messages, such as:

window.addEventListener('message', t.step_func(messageEvent => {
// The failure message of no device chosen is expected. The point here is
// to validate not failing because of a sandboxed iframe.
assert_true(messageEvent.data.includes('NotFoundError'));

... which fails because the handler only expects strings postMessage()ed from its child frame.

Maybe these messages should be filtered out in a high-level step_func-like API, so that every test doesn't need its own ad-hoc check (which also leaks testdriver implementation details)?

CC @jgraham @WeizhongX @cbiesinger (who fixed many such cases for fedcm/)

aarongable pushed a commit to chromium/chromium that referenced this issue Sep 24, 2024
Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweih@chromium.org>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1359563}
chromium-wpt-export-bot pushed a commit that referenced this issue Sep 24, 2024
Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: #48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweih@chromium.org>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1359563}
chromium-wpt-export-bot pushed a commit that referenced this issue Sep 24, 2024
Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: #48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweih@chromium.org>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1359563}
i3roly pushed a commit to i3roly/firefox-dynasty that referenced this issue Sep 27, 2024
…dboxed_iframe.https.window.js`, a=testonly

Automatic update from web-platform-tests
[wpt] Ignore testdriver messages in `sandboxed_iframe.https.window.js`

Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweih@chromium.org>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1359563}

--

wpt-commits: 50433bccd99443bf1c59165a0c9a5b88a3c5c3ae
wpt-pr: 48328
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this issue Sep 27, 2024
…dboxed_iframe.https.window.js`, a=testonly

Automatic update from web-platform-tests
[wpt] Ignore testdriver messages in `sandboxed_iframe.https.window.js`

Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweih@chromium.org>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1359563}

--

wpt-commits: 50433bccd99443bf1c59165a0c9a5b88a3c5c3ae
wpt-pr: 48328
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified-and-comments-removed that referenced this issue Oct 1, 2024
…dboxed_iframe.https.window.js`, a=testonly

Automatic update from web-platform-tests
[wpt] Ignore testdriver messages in `sandboxed_iframe.https.window.js`

Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweihchromium.org>
Commit-Queue: Jonathan Lee <jonathanjleegoogle.com>
Cr-Commit-Position: refs/heads/main{#1359563}

--

wpt-commits: 50433bccd99443bf1c59165a0c9a5b88a3c5c3ae
wpt-pr: 48328

UltraBlame original commit: cd1e3e547399549474aa7278bee1e2065b165d43
gecko-dev-updater pushed a commit to marco-c/gecko-dev-comments-removed that referenced this issue Oct 1, 2024
…dboxed_iframe.https.window.js`, a=testonly

Automatic update from web-platform-tests
[wpt] Ignore testdriver messages in `sandboxed_iframe.https.window.js`

Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweihchromium.org>
Commit-Queue: Jonathan Lee <jonathanjleegoogle.com>
Cr-Commit-Position: refs/heads/main{#1359563}

--

wpt-commits: 50433bccd99443bf1c59165a0c9a5b88a3c5c3ae
wpt-pr: 48328

UltraBlame original commit: cd1e3e547399549474aa7278bee1e2065b165d43
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this issue Oct 1, 2024
…dboxed_iframe.https.window.js`, a=testonly

Automatic update from web-platform-tests
[wpt] Ignore testdriver messages in `sandboxed_iframe.https.window.js`

Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweihchromium.org>
Commit-Queue: Jonathan Lee <jonathanjleegoogle.com>
Cr-Commit-Position: refs/heads/main{#1359563}

--

wpt-commits: 50433bccd99443bf1c59165a0c9a5b88a3c5c3ae
wpt-pr: 48328

UltraBlame original commit: cd1e3e547399549474aa7278bee1e2065b165d43
@jonathan-j-lee
Copy link
Contributor Author

jonathan-j-lee commented Oct 2, 2024

Looks like this also affects webusb/requestDevice/sandboxed_iframe.https.window.html and digital-credentials/. Updating each test to ignore testdriver messages won't scale (not to mention it's fragile and ugly).

Maybe we should put __wptrunner_message_queue in a different browsing context cross-origin with all test pages, which postMessage() testdriver events to only that test origin?

Edit: Never mind, looks like an easy fix with stopImmediatePropagation() (#48433).

@jonathan-j-lee jonathan-j-lee self-assigned this Oct 3, 2024
jamienicol pushed a commit to jamienicol/gecko that referenced this issue Oct 3, 2024
…dboxed_iframe.https.window.js`, a=testonly

Automatic update from web-platform-tests
[wpt] Ignore testdriver messages in `sandboxed_iframe.https.window.js`

Fix two issues causing the test to fail spuriously upstream when run
with WebDriver [0]:
a. testdriver.js uses regular `postMessage()`/`onmessage` to communicate
   testdriver commands between the parent and child frames.
   Unfortunately, until [1] is resolved, tests that use `onmessage` need
   to be aware of this implementation detail and ignore the internal
   messages.
b. The parent needs `testdriver(-vendor).js` too so that testdriver
   commands from the child frame are correctly pumped out [2] to the
   harness, which will send the WebDriver command.

Content shell currently passes the test because its testdriver is backed
by internal JS bindings that directly hook into the browser.

[0]: https://wpt.fyi/results/serial/requestPort/sandboxed_iframe.https.window.html?run_id=5088854063448064
[1]: web-platform-tests/wpt#48326
[2]: https://github.com/web-platform-tests/wpt/blob/8ae822f0/tools/wptrunner/wptrunner/testdriver-extra.js#L22

Bug: None
Change-Id: Iddb130f92c1877e389a46cb89a35c21bbfd1b625
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5888384
Reviewed-by: Jack Hsieh <chengweih@chromium.org>
Commit-Queue: Jonathan Lee <jonathanjlee@google.com>
Cr-Commit-Position: refs/heads/main@{#1359563}

--

wpt-commits: 50433bccd99443bf1c59165a0c9a5b88a3c5c3ae
wpt-pr: 48328
jonathan-j-lee added a commit to jonathan-j-lee/wpt that referenced this issue Oct 3, 2024
jonathan-j-lee added a commit that referenced this issue Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant