Commit graph

1741 commits

Author SHA1 Message Date
Max Schmitt 99bbc51760
fix(test-runner): support ANSII terminals with list reporter (#7258) 2021-06-22 19:04:24 +02:00
Dmitry Gozman 73a43fce60
chore(test runner): self profile under PWTEST_PROFILE_DIR (#7252)
`PWTEST_CLI_ALLOW_TEST_COMMAND=1 PWTEST_PROFILE_DIR=./profile node lib/cli/cli.js test -c <dir/config>`
2021-06-21 14:49:43 -07:00
Max Schmitt 143689e34f
fix: Jest isError handling to work with TypeError (#7251) 2021-06-21 21:48:24 +02:00
Dmitry Gozman 6118d16edd
fix(test runner): properly keep track of requireFile to support helpers/wrappers (#7243)
This fixes an issue where we incorrectly labeled and assigned ids for tests
that declared tests in require'd files or used test wrappers.
See new tests for examples.
2021-06-21 11:25:15 -07:00
Max Schmitt 590184998f
chore: improve connectOverCDP error handling (#7239) 2021-06-21 19:57:00 +02:00
Pavel Feldman ea4eebeb2d
feat(test-runner): document tagging, implement grep-invert (#7227) 2021-06-18 17:56:59 -07:00
Yury Semikhatsky c0cd2d4579
feat: response interception (#7122) 2021-06-18 11:04:48 -07:00
Yury Semikhatsky 79b2e5ddeb
feat(webkit): bump to 1500 (#7224) 2021-06-18 10:20:09 -07:00
Dmitry Gozman 388c6f5196
feature(test runner): capture error and pending calls when timing out (#7193) 2021-06-17 15:09:38 -07:00
Max Schmitt 5bf69b667e
fix(test-runner): allow additional params when pw-test is not installed (#7200) 2021-06-17 13:40:49 -07:00
Ross Wollman 219e5138be
feat(response): expose connection details in API (#7150) 2021-06-17 13:04:55 -07:00
Dmitry Gozman 5e471a3ece
feat(test runner): add type to TestInfo.error (#7207)
It has a very special `TestError` type, but we say it's `any` for some reason.
2021-06-17 12:14:28 -07:00
Joel Einbinder 10a82f862c
fix(snapshotter): support constructed CSSStyleSheet
Fixes #7085
2021-06-17 09:41:29 -07:00
Pavel Feldman 82a50b0e1d
fix(test-runner): property handle artifacts in context of preserveOutput (#7181) 2021-06-16 16:05:30 -07:00
Andrey Lushnikov 0cfea9a623
feat(chromium): roll to r891964 (#7185) 2021-06-16 16:00:54 -07:00
Pavel Feldman 184f2c2e93
feat(test-runner): allow specifying video size (#7158) 2021-06-16 07:51:54 -07:00
Andrey Lushnikov a3e4185b2f
chore: nuke 'webkit-technology-preview' experiment (#7164)
It proved to be unmaintainable due to the following reasons:
- determining upstream branch for the technology preview was a manual
and tedious process
- once determined, actual Safari Technology Preview occasionally
diverted from the branch
2021-06-15 23:58:30 -07:00
Andrey Lushnikov ec68b31dc4
feat: support chromium-with-symbols channel (#7135)
References #7130
2021-06-15 17:57:31 -07:00
Joel Einbinder 38e27c9c6c
fix(test-runner): case insensitive test filtering 2021-06-15 17:27:52 -07:00
Yury Semikhatsky e5d0a535a8
fix: return valid browser from context.browser() when connect over CDP (#7155) 2021-06-15 16:06:38 -07:00
Dmitry Gozman b74ca36fb3
chore: create Selectors for each Playwright object (#7154)
We currently have singleton `serverSelectors` that is shared between
the real Playwright and internal Playwrights.
2021-06-15 14:56:29 -07:00
Joel Einbinder 4c2a3fb443
fix(test-runner): accept relative paths for outputDir (#7149)
fixes #7124
2021-06-15 13:39:07 -07:00
Pavel Feldman 9550106e1d
fix(test-runner): video: 'retain-on-failure' leaves empty folders behind (#7125) 2021-06-15 10:06:49 -07:00
Ross Wollman 195eab8787
feat(har): record remote IP:PORT and SSL details (#6631) 2021-06-15 09:48:08 +02:00
Dmitry Gozman 742cce3a1d
feat(test runner): configurable reportSlowTests (#7120)
Also splits tests by projects and reports them with nice relative paths.
2021-06-14 22:45:58 -07:00
Joel Einbinder aa72b2b9bb
fix(pwt): max-failures should work with retries (#7127)
fixes #7112
2021-06-14 22:16:16 -07:00
Dmitry Gozman 2041aab010
fix(chromium): background pages on persistent close error (#7118)
This is a speculative fix that moves "background pages cleanup"
to `_didCloseInternal` so that it is only run once, but on both
context closure and browser closure.

Symptom from a flaky test:

```log
browserContext.close: page@18087c372d32819222707ca5e8fd1030 is sending "close" event after being disposed
    at PageDispatcher._dispatchEvent (D:\a\playwright\playwright\src\dispatchers\dispatcher.ts:86:15)
    at Page.<anonymous> (D:\a\playwright\playwright\src\dispatchers\pageDispatcher.ts:59:12)
    at Page.emit (events.js:314:20)
    at Page._didClose (D:\a\playwright\playwright\src\server\page.ts:220:10)
    at CRPage.didClose (D:\a\playwright\playwright\src\server\chromium\crPage.ts:165:16)
    at CRBrowserContext._onClosePersistent (D:\a\playwright\playwright\src\server\chromium\crBrowser.ts:476:24)
    at CRBrowserContext.close (D:\a\playwright\playwright\src\server\browserContext.ts:288:20)
```
2021-06-14 21:55:55 -07:00
Pavel Feldman 970bb6a70d
feat(test-runner): allow setting pixel match threshold for project (#7123) 2021-06-14 21:52:10 -07:00
Dmitry Gozman 060f7ffa92
fix: specify the right apiName for artifact errors (#7119)
This produces errors like `download.saveAs: <error>` instead of `.saveAs: <error>`.
Drive-by: fix the flaky test.
2021-06-14 16:41:53 -07:00
Max Schmitt ad5280e5cb
fix(trace-viewer): display trace without viewport (#7101) 2021-06-14 17:11:30 +02:00
Peng-Yu Chen 5f6d4a7b73
feat(download): adding a new Download._cancel method (#6236) 2021-06-12 22:23:22 +02:00
Andrey Lushnikov 8a8b3932f8
Revert "fix(test-runner): support passing slowMo option (#6991)" (#7077)
This reverts commit 178489d091.

Reason for revert: this clashes with testrunner options.
2021-06-11 16:19:50 -07:00
Andrey Lushnikov d6831df26f
feat: roll Firefox to r1271 (#7067)
References #7015
2021-06-11 10:50:02 -07:00
Yury Semikhatsky c903b04c7a
feat(webkit): bump to 1499 (#7001) 2021-06-11 09:35:48 -07:00
Pavel Feldman 49a8f67c0f
fix(test-runner): resolve global hooks relative to the config dir (#7061) 2021-06-10 22:31:27 -07:00
Pavel Feldman cbce7cbdec
fix(tracing): do not stall on dialogs (#7059) 2021-06-10 22:24:04 -07:00
Pavel Feldman e4d93cd1f3
fix(video): do not fail when removing non-existent video (#7060) 2021-06-10 22:23:02 -07:00
Andrey Lushnikov f52290d4ea
Revert "feat: add defaultTimeout browser context configuration option (#6944)" (#7057)
This reverts commit 617dfdef9e.

It turns out this might conflict with our bright testrunner future.
2021-06-10 18:38:56 -07:00
Amit Abershitz 617dfdef9e
feat: add defaultTimeout browser context configuration option (#6944)
Fixes #6940

Co-authored-by: Amit Abershitz <aabershitz@proofpoint.com>
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2021-06-10 18:10:20 -07:00
Pavel Feldman 3b1bae8a40
feat(codegen): allow generating tests (#7049) 2021-06-10 16:52:59 -07:00
Max Schmitt 05382c997b
fix(test-runner): do only match JS/TS files when collecting (#7014) 2021-06-10 16:41:57 +02:00
Andrey Lushnikov 3c7d2aae6a
fix: add best-effort support for MacOS beta versions (#7003)
Similarly to how we provide best-effort support for non-LTS ubuntu
versions, this patch adds support for beta versions of MacOS releases.
2021-06-09 14:58:20 -07:00
Max Schmitt 178489d091
fix(test-runner): support passing slowMo option (#6991)
Fixes #6984
Reverts #6967
2021-06-09 14:02:05 -07:00
Pavel Feldman 19d69b792d
fix(tracing): do not merge srcset urls (#6995)
Fixes #6982
2021-06-09 12:05:21 -07:00
Joel Einbinder 6788b9cf8c
fix(@playwright/test): plumbing (#6992)
`npx playwright test` should find and forward to the @playwright/test package.
2021-06-09 11:18:32 -07:00
Pavel Feldman 521153844a
docs: fix the nunit example (#6978) 2021-06-08 19:14:18 -07:00
Dmitry Gozman ff3f951dc8
fix(firefox): race between markAsError and Page.ready (#6976)
When new page turns into download, we can have `_markAsError` followed
by the `Page.ready` signal and report the page twice (as an error and
as a real page). This is flaky and depends on whether `Page.ready` is
fast enough before the page closes or not.

Exposed by the "should report new window downloads" test.
2021-06-08 17:28:24 -07:00
Dmitry Gozman 6eeafc171b
fix(chromium): fix a race when intialization does not finish before page close (#6975)
This is exposed by the flaky "should report new window downloads" test.
In this test a new page is created, initialized and closed before initialization
finishes. If `lifecycleEventsEnabled` fails with "Target closed error",
we correctly ignore the initialization failure, but a single usage of the
failed promise with `.then` fails anyway.
2021-06-08 17:28:16 -07:00
Pavel Feldman 021f51cc1f
feat(trace): add a trace option into the test runner (#6961)
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2021-06-08 11:22:07 -07:00
Joel Einbinder 93f6b57ca5
feat(playwright-test): scan the world (#6935)
Running `npx playwright test` results in a cryptic error message that says to check `--help` for more information.

I started to rewrite it, but I think instead it would be much nicer if we just treated `npx playwright test` as an alias for `npx playwright test -c .`. There is a comment in the code that worries this will be slow because it will scan the world. However we need to support this case anyway for the tests-next-to-code pattern.

It looks like this should be fast because we ignore node_modules/.gitignore entries. If this is still too slow we should make it faster. If we can't make it faster we should throw a more specific error telling the user how to make it faster.
2021-06-08 11:02:16 -07:00