Commit graph

815 commits

Author SHA1 Message Date
Andrey Lushnikov faf9d901c1
chore: fix test to work on Windows with auto.crlf = false (#10296) 2021-11-13 00:29:51 -08:00
Dmitry Gozman 6a46711347
fix(chromium): race between loadingFailed and requestPaused (#10289) 2021-11-12 19:06:53 -08:00
Dmitry Gozman 4bb123d4b7
fix(test runner): route more errors to reporter (#10263)
For example, top-level errors in files or global setup issues.
2021-11-12 06:47:41 -08:00
Andrey Lushnikov 80da0f7b69
test: test gardening (#10258) 2021-11-12 00:00:13 -08:00
Dmitry Gozman 7eec66d0f9
chore(test runner): route runner errors through Reporter.onError (#10257) 2021-11-11 16:48:08 -08:00
Dmitry Gozman f38f611478
chore: add npm run lint-tests (#10252) 2021-11-11 13:27:50 -08:00
Dmitry Gozman 4c93417e8a
fix(github reporter): handle global onError (#10256) 2021-11-11 13:25:38 -08:00
Yury Semikhatsky fbb3c88f3c
fix(fetch): smarter JSON.stringify for application/json requests (#10245) 2021-11-11 11:12:24 -08:00
Max Schmitt 3a64da7a54
fix: do not show taskkill.exe output (#10244) 2021-11-11 18:32:22 +01:00
Max Schmitt ac629afd46
chore: fix API name capitalisation with APIRequestContext (#10240) 2021-11-11 08:46:17 -08:00
Dmitry Gozman 9622704a8a
fix(test runner): update TestInfo.duration before running afterEach hooks (#10228) 2021-11-10 16:02:27 -08:00
Dmitry Gozman 9ec3e7cd52
fix(click): account for detached elements and iframe overlays (#10206) 2021-11-10 12:14:06 -08:00
Pavel Feldman 03fee2f593
feat(trace-viewer): add metainfo tab (#10205) 2021-11-09 22:13:51 -08:00
Pavel Feldman 0d0f1690cd
fix(trace-viewer): do not rely upon request mode, it is inconsistent (#10198) 2021-11-09 15:12:37 -08:00
Dmitry Gozman 75efeb1e08
fix: resolve ip using grid/api/testsession endpoint (#10196)
For Selenium 4, we use se:cdp ws proxy, pointing it to the hub url.
For Selenium 3, we use grid api to try and get the target node ip.
2021-11-09 14:41:13 -08:00
Pavel Feldman 4e90eb9406
api(frame-locator): allow nth, first, last (#10195) 2021-11-09 14:14:20 -08:00
Max Schmitt d25b0f70bc
chore: api testing test nits (#10180) 2021-11-09 14:11:42 -08:00
Andrey Lushnikov cce359ea83
test: fix fixture teardown (#10174) 2021-11-09 11:12:18 -08:00
github-actions[bot] 5b604acedb
feat(chromium): roll to r939194 (#10165)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2021-11-08 20:05:39 -08:00
Pavel Feldman 806b5706a5
fix(trace-viewer): redirect, time and missing snapshot bugfixes (#10055) 2021-11-08 18:03:10 -08:00
Pavel Feldman 7b64161a37
feat(test-runner): allow specifying fine-grained trace options (#10147) 2021-11-08 15:39:58 -08:00
Dmitry Gozman 3cc839e013
fix(route): catch all Route methods when page closes (#10140)
This fixes a common scenario where you setup a route,
and the page closes (e.g. test ends) while we are aborting/continuing
some requests that are not instrumental to the test itself.
2021-11-08 15:13:15 -08:00
Joel Einbinder 0cad0de3e3
fix(test runner): better error message when importing typescript from esmodule (#10061) 2021-11-08 16:25:40 -05:00
Dmitry Gozman 0a104bc500
docs: clarify that slow tests are about files (#10134) 2021-11-08 12:54:18 -08:00
Pavel Feldman 4553d76fce
api(frame-locator): introduce frame locators (#10102) 2021-11-08 09:58:24 -08:00
Andrey Lushnikov 51a7567907
codegen: use import instead of require for codegen tests (#10104) 2021-11-05 19:01:54 -07:00
Dmitry Gozman 61ff52704c
feat(input): perform hit target check during input (#9546)
This replaces previous `checkHitTarget` heuristic that took place before the action
with a new `setupHitTargetInterceptor` that works during the action:
- Before the action we set up capturing listeners on the window.
- During the action we ensure that event target is the element we expect to interact with.
- After the action we clear the listeners.

This should catch the "layout shift" issues where things move
between action point calculation and the actual action.

Possible issues:
- **Risk:** `{ trial: true }` might dispatch move events like `mousemove` or `pointerout`,
because we do actually move the mouse but prevent all other events.
- **Timing**: The timing of "hit target check" has moved, so this may affect different web pages
in different ways, for example expose more races. In this case, we should retry the click as before.
- **No risk**: There is still a possibility of mis-targeting with iframes shifting around,
because we only intercept in the target frame. This behavior does not change.

There is an opt-out environment variable PLAYWRIGHT_NO_LAYOUT_SHIFT_CHECK that reverts to previous behavior.
2021-11-05 17:31:28 -07:00
Yury Semikhatsky 15a8b29961
test: goto finishes on window.stop() (#10070) 2021-11-05 16:51:22 -07:00
Pavel Feldman f3fd3ebc37
chore(frame-selector): add more tests, use frame logic in element handle (#10097) 2021-11-05 16:36:01 -07:00
Andrey Lushnikov fd2bbe0c1a
tests: disable video tests on Firefox + Tracing (#10096) 2021-11-05 15:15:46 -07:00
Max Schmitt 9dbf225afc
test: fix connectOverCDP with proxy on Windows (#10092) 2021-11-05 19:49:39 +01:00
Pavel Feldman f19864890f
feat(iframe): make iframe selectors work w/ element handles (#10063) 2021-11-05 11:06:04 -07:00
Max Schmitt 84d2ee3929
chore: fix connectOverCDP on Windows when proxy is used (#10080) 2021-11-05 17:38:13 +01:00
Dmitry Gozman 3c1aaa5338
fix(trace): do not call tracing.stopChunk() twice (#10054) 2021-11-04 21:08:42 -07:00
Yury Semikhatsky 83657a1df4
test: skip interception tests in electron (#10056) 2021-11-04 13:36:04 -07:00
Pavel Feldman 78e99249a3
feat(frame-selector): intial implementation (#10018) 2021-11-04 13:28:35 -07:00
github-actions[bot] 9b4d9460eb
feat(chromium): roll to r936784 (#9986) 2021-11-04 12:35:31 -07:00
Dmitry Gozman 2cfc997614
test: detach frame while clicking race (#10029) 2021-11-03 19:41:31 -07:00
Andrey Lushnikov 1a577ac344
chore: introduce docker integration dogfood (#9998)
This patch:
- rolls stable-test-runner to Nov 2, 2021 tip-of-tree
- introduces a new npm script, `npm run vtest`, to run Visual Regression
  Tests for our HTML reporter
2021-11-03 15:26:18 -07:00
Yury Semikhatsky bce9a8187d
fix(fetch): post data for delete request (#10019) 2021-11-03 13:26:15 -07:00
Dmitry Gozman 3120f81629
fix(reporters): move Running X tests using Y workers to reporters (#10014) 2021-11-03 11:17:23 -07:00
Dmitry Gozman c373986ca0
fix(evaluate): reject all context operations when frame detaches (#9987) 2021-11-03 10:44:50 -07:00
Dmitry Gozman 9cebe60831
feat(reporters): augment non-stdio reporters with dot/line (#10003) 2021-11-03 08:25:16 -07:00
Yury Semikhatsky 5abb5d74ea
chore: delete legacy response interception code (#9995) 2021-11-02 17:48:38 -07:00
Pavel Feldman cd47bf26e8
feat(trace-viewer): render snapshot urls (#9993) 2021-11-02 17:35:23 -07:00
Yury Semikhatsky 06cf7132be
test: unflake wheel test in ff (#9989) 2021-11-02 13:56:12 -07:00
Pavel Feldman 6a30c90590
feat(trace-viewer): render wall time for each action (#9982) 2021-11-02 12:16:12 -07:00
Max Schmitt ad36a194a5
test: add screenshot test with hanging main resource (#9972) 2021-11-02 16:55:32 +01:00
Nick Partridge a51ac39275
feat: add snapshotDir to set base snapshot directory (#9260) 2021-11-02 08:02:49 -07:00
Pavel Feldman 56ca3a18f5
feat(trace-viewer): show test name (#9957) 2021-11-01 21:23:35 -07:00
Pavel Feldman 3673776330
Revert "chore(test-runner): use test name as trace file name prefix (#9619)" (#9956)
This reverts commit 3c420a7cf1.
2021-11-01 20:27:41 -07:00
Pavel Feldman 85497c5509
feat(tracing): allow including source files in traces (#9946) 2021-11-01 19:57:29 -07:00
Yury Semikhatsky 13cc266b37
feat: add "commit" to possible waitUntil options (#9892) 2021-11-01 17:12:19 -07:00
github-actions[bot] 5d49c41a52
feat(firefox): roll to r1299 (#9932)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2021-11-02 00:23:16 +01:00
Pavel Feldman 9ac8829583
feat(html): bake report zip into the html report, allow opening from fs (#9939) 2021-11-01 16:14:52 -07:00
Pavel Feldman d79aae633c
feat(html): render the suite name before test title (#9909) 2021-11-01 10:53:42 -07:00
Dmitry Gozman 4f1027bdd0
feat(test runner): introduce TestInfo.parallelIndex (#9762)
This is a worker number between `0` and `workers - 1` that
does not change after worker process restart.
2021-11-01 10:37:34 -07:00
Max Schmitt 3c420a7cf1
chore(test-runner): use test name as trace file name prefix (#9619) 2021-11-01 18:15:19 +01:00
Max Schmitt ff8494c7e3
test: add test for downloads in nested downloads (#9924) 2021-11-01 09:45:22 -07:00
Pavel Feldman 8991bbde33
feat(html): live filtering, opt-out from auto-open (#9889) 2021-10-29 16:24:08 -07:00
Dmitry Gozman 227e3da62f
fix(test runner): do not reuse worker that did not teardown scopes (#9872)
Two bug fixes:
- Do not use the worker that is being shutdown for a new job.
- Report unhandled errors during "expected to fail" tests as
  fatal errors.
2021-10-29 13:36:12 -07:00
Dmitry Gozman cea61691fa
fix(connectOverCDP): ensure cleanup when remote ws closes (#9873) 2021-10-29 10:35:24 -07:00
Yury Semikhatsky 1886897e5c
fix(matchers): support builtin types (#9865) 2021-10-28 18:45:59 -07:00
Pavel Feldman 75ac579fac
fix(toHaveCount): another edge case fix (#9860) 2021-10-28 16:04:21 -07:00
Max Schmitt 21c4435060
test: do kill testProcesses accordingly (#9818)
* test: do kill testProcesses accordingly

* fix it
2021-10-28 23:08:04 +02:00
Pavel Feldman 2e4722d460
test: migrate to upstream fixtures (#9835) 2021-10-28 08:31:30 -07:00
github-actions[bot] 9af5aaabbb
feat(firefox): roll to r1298 (#9848) 2021-10-28 07:41:58 -07:00
Andrey Lushnikov 4b22e1d54a
chore: bump stable test runner to 1.16.2 (#9836) 2021-10-28 01:11:26 -07:00
Pavel Feldman 16afb5064f
test: remove browserOptions (#9829) 2021-10-27 19:00:06 -07:00
Pavel Feldman 5bf1c03b51
test: add several toBeEnabled methods (#9821) 2021-10-27 16:12:54 -07:00
Max Schmitt be64b5f6e2
chore: add missing dynamic sdkLanguage based CLI invocations (#9817) 2021-10-27 18:58:13 +02:00
Pavel Feldman 31faa7d9a7
chore: sort out base fixtures (#9809) 2021-10-27 08:28:53 -07:00
Yury Semikhatsky dd0dae623b
fix(fetch): ignoreHTTPSErrors after redirects (#9806) 2021-10-26 23:20:52 -07:00
Joel Einbinder c8addef03a
fix(monorepo): export package.json from every package (#9803) 2021-10-26 18:41:18 -04:00
Dmitry Gozman 22e4a0d580
fix(snapshots): --update-snapshots + retries should work (#9799) 2021-10-26 13:50:16 -07:00
Pavel Feldman 273122b761
chore: align internal test harness w/ @playwright/test (#9796) 2021-10-26 13:45:53 -07:00
Pavel Feldman 87c64b2c1c
chore: align client side instrumentations (#9771) 2021-10-26 11:13:35 -07:00
Pavel Feldman 5451808df1
chore: append sources to traces (#9736) 2021-10-25 18:56:57 -07:00
Dmitry Gozman 13ed1dee50
fix(test runner): do not mask uncaught error in beforeEach (#9764) 2021-10-25 14:17:27 -07:00
Pavel Feldman 2d4db7a6f0
fix(stack): hide test runner stack frames (#9735) 2021-10-25 11:49:59 -07:00
Dmitry Gozman 7527ad27d3
fix(snapshot): empty adopted stylesheet should not prevent node refs (#9731)
We never marked empty stylesheets as "stale", so we never computed
css text for them. This prevented node reuse, because empty string
is not equal to undefined.
2021-10-23 16:26:46 -07:00
Pavel Feldman f08c22b467
fix(trace-viewer): show source files in local version (#9732) 2021-10-23 11:23:39 -07:00
Yury Semikhatsky d183d364a0
fix(fetch): remove content-length from redirected post requests (#9728) 2021-10-22 16:52:49 -07:00
Dmitry Gozman 23aa0be15b
docs: update afterEach docs with an example (#9727)
Also add a test for TestInfo.status in afterEach.
2021-10-22 16:32:22 -07:00
Joel Einbinder f2888395a6
fix: support even older node 12 (#9718) 2021-10-22 15:59:52 -04:00
Dmitry Gozman 193c79a685
feat(test runner): rewrite dispatcher to avoid unneded stalls (#9629)
In some circumstances, dispatcher was waiting for all exisitng jobs
to finish before scheduling a new one. This leads to unneded stalls.
Instead, we can schedule jobs right away, if we have a worker
available.
2021-10-22 11:10:37 -07:00
Pavel Feldman 6a3e08d1ac
fix(reporter): fix locator stacks to hide internal detail (#9693) 2021-10-21 16:10:47 -07:00
Pavel Feldman 299dffbdb3
fix(expect): fix toHaveCount(0) (#9690) 2021-10-21 14:46:54 -07:00
Joel Einbinder deec8fd6c2
test(chromium): failing test for resolving responses after a navigation (#9651) 2021-10-21 08:29:04 -07:00
Max Schmitt ba2576c5b9
test: page.content() hangs when there is no iframe src (#9674) 2021-10-21 16:59:33 +02:00
Pavel Feldman 225145fc3e
fix(expect): do not fail on navigated frames while polling (#9659) 2021-10-20 13:01:05 -07:00
Max Schmitt 5426e24248
chore: roll stable-test-runner to ToT (#9655) 2021-10-20 10:35:37 -07:00
Andrey Lushnikov fe49b91680
tests: skip Docker ipv6 tests (#9646)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2021-10-20 02:03:46 -07:00
Pavel Feldman 2b9d7815b2
chore: enable html report by default (#9639) 2021-10-19 21:31:59 -07:00
Andrey Lushnikov 500e43931e
test: bring the snapshotter test back (#9635) 2021-10-19 17:38:01 -07:00
Andrey Lushnikov f29ff86344
test: further fixes to snapshotter tests (#9633) 2021-10-19 17:26:57 -07:00
Andrey Lushnikov dddf70cead
tests: fix variety of tests (#9630) 2021-10-19 17:03:15 -07:00
Pavel Feldman ba57be99a9
fix(infobar): account for infobar in headed mode (#9627) 2021-10-19 15:36:17 -07:00
Joel Einbinder 53c866c0f2
fix(monorepo): explicitly list package.json exports instead of using glob patterns (#9584) 2021-10-19 12:28:02 -04:00
Dmitry Gozman fc6a93d24e
test: expect.not.toBeChecked when no elements match (#9607) 2021-10-19 09:13:16 -07:00