Commit graph

798 commits

Author SHA1 Message Date
Dmitry Gozman 7879cf30f0
fix(reuse): serial mode with tracing should not throw (#21726)
Regressed in #19733.
Fixes #21113.
2023-03-16 14:32:12 -07:00
Dmitry Gozman ea85242941
chore: improve client-side options plumbing (#21697) 2023-03-16 07:03:33 -07:00
Pavel Feldman c45d8749b0
chore: split trace events into phases (#21696) 2023-03-15 22:33:40 -07:00
Dmitry Gozman b149d132a6
chore(test runner): split loadAllTests into parts (#21674) 2023-03-15 13:40:58 -07:00
Pavel Feldman 27048adebe
chore: introduce testInfo.testId (#21670) 2023-03-14 15:58:55 -07:00
Dmitry Gozman 455e11c1d9
fix(shards): shard test groups instead of files (#21638)
Fixes #21497.
2023-03-14 10:41:37 -07:00
Max Schmitt a2c96494e0
chore: throw pretty error if rest parameters is used inside fixtures (#21659)
Fixes https://github.com/microsoft/playwright/issues/21566
2023-03-14 17:35:52 +01:00
Pavel Feldman f5fa18a279
chore: reimplement filters (#21647) 2023-03-13 22:19:31 -07:00
Dmitry Gozman 809725e114
fix(reuse): only reuse the default browser, not explicitly launched ones (#21631)
Fixes #20399.
2023-03-13 16:50:51 -07:00
Pavel Feldman 1d870ac407
chore(ui): start adding ui mode tests (2) (#21608) 2023-03-13 12:14:51 -07:00
Pavel Feldman a12e909a40
chore(ui): start adding ui mode tests (#21601) 2023-03-12 15:18:47 -07:00
Pavel Feldman 493171cb6b
chore: select files to watch explicitly (#21599) 2023-03-12 10:50:21 -07:00
Dmitry Gozman ea8aa63f1a
feat: support PWTEST_PROFILE_DIR in run-server (#21582)
This creates one profile per connection in the specified directory.
2023-03-10 19:07:40 -08:00
Pavel Feldman 0b9c037f36
chore(ui): do not add sources into the zip file (#21588) 2023-03-10 17:01:30 -08:00
Dmitry Gozman 6d27d2c029
fix(test runner): improved fixture teardown timeout error message (#21572)
The new message looks like this:

```
Test finished within timeout of 500ms, but tearing down "playwright configuration" ran out of time.
Please allow more time for the test, since teardown is attributed towards the test timeout budget.·
```

References #21259.
2023-03-10 13:45:47 -08:00
Pavel Feldman 1073e42686
chore: allow running failing (#21569) 2023-03-10 12:41:00 -08:00
Dmitry Gozman 58a23bc7a0
fix(test runner): allow worker-only dynamic imports (#21545)
Fixes #21409.
2023-03-10 08:58:26 -08:00
Pavel Feldman e7b21425f7
chore: support both test --ui and ui clis (#21537) 2023-03-09 13:03:01 -08:00
Pavel Feldman 47a71731f4
chore: add explicit command for gui runner (#21509) 2023-03-09 08:04:02 -08:00
Pavel Feldman 5ef2edb9f7
chore: support suites in the tree (#21508) 2023-03-08 19:50:32 -08:00
Pavel Feldman a2490a8fc8
chore(ui): maintain test order when updating (#21478) 2023-03-07 20:34:57 -08:00
Pavel Feldman f5894ed089
chore: open location in vscode (#21472) 2023-03-07 17:20:41 -08:00
Pavel Feldman e9f94f0346
chore: add ui mode terminal (#21470) 2023-03-07 14:24:50 -08:00
Pavel Feldman 65117702e7
chore: consistent xterm naming (#21446) 2023-03-06 20:39:52 -08:00
Pavel Feldman b6ff3bad98
chore: split code mirror and xterm modules (#21415) 2023-03-06 10:40:45 -08:00
Pavel Feldman bae9173208
chore: split ui mode toolbar into lines, show progress (#21403) 2023-03-05 13:46:21 -08:00
Pavel Feldman ec056a6312
chore: merge projects in the tree (#21401) 2023-03-04 16:28:30 -08:00
Pavel Feldman f0cd123a45
chore(ui): make ui-side drive things (#21398) 2023-03-04 15:05:41 -08:00
Dmitry Gozman eb3f8bfba8
fix(test runner): do not optimize filtering when sourcemap is present (#21359)
Fixes #21204.
2023-03-03 07:49:19 -08:00
Dmitry Gozman 6769a311ed
feat(test runner): error out when one test file imports another (#21357)
This situation is not supported, and we can now detect it by looking at
collected file dependencies.

Fixes #21270.
2023-03-02 15:09:50 -08:00
Pavel Feldman 57624bc01b
chore: ui mode ui improvements (#21325) 2023-03-02 13:45:15 -08:00
Dmitry Gozman 0b300f455c
fix(test runner): empty dependency should not skip other projects (#21354)
References #21270.
2023-03-02 13:32:23 -08:00
Max Schmitt 6a7347f134
chore: have pretty error if CT config has no defineConfig (#21318)
Fixes https://github.com/microsoft/playwright/issues/21312
2023-03-02 00:47:05 +01:00
Pavel Feldman e222874445
chore: ui mode first cut (#21291) 2023-03-01 15:27:23 -08:00
Yury Semikhatsky 60e5a93832
fix(store): support text and binary values (#21006) 2023-03-01 08:49:31 -08:00
Pavel Feldman de3a5e2a91
chore(trace): include expect steps in a trace (#21199) 2023-02-28 13:26:23 -08:00
Dmitry Gozman 27027658dc
chore: fix all Proxy() to account for symbol properties (#21272)
Fixes #20940.
2023-02-28 12:45:14 -08:00
Pavel Feldman 81bd637d94
chore: merge traces into a single zip file (#21242) 2023-02-27 22:31:47 -08:00
Yury Semikhatsky cfdddcf9ba
fix(runner): finish dispatching if remaining tests were skipped (#21238)
Fixes #21226
2023-02-27 17:20:30 -08:00
Leo Sklyut 6c3d35fa58
fix(junit): use the JUnit schema's specified ISO format for timestamp (#21062)
Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2023-02-27 11:04:22 +01:00
Pavel Feldman 4d91248c1e
chore: simplify the trace collection fixtures (#21203) 2023-02-25 21:21:09 -08:00
Pavel Feldman 09f77c41dd
chore: migrate to protocol's StackFrame type (#21198) 2023-02-24 18:36:15 -08:00
Pavel Feldman 3fa19e80ad
chore: wrap expect call in a zone (#21191) 2023-02-24 12:17:03 -08:00
Pavel Feldman 8ccc02a9d3
chore: use client wall time in traces (#21163) 2023-02-23 14:37:53 -08:00
Nowell Strite 000583e048
fix(loader): experimentalLoader with node@18 (#21106)
There is currently a bug when running `node@18.14.2` when running with
experimentalLoader

```
TypeError: The URL must be of scheme file
  at new NodeError (node:internal/errors:399:5)
  at Object.fileURLToPath (node:internal/url:1492:11)
  at resolve (./node_modules/@playwright/test/lib/experimentalLoader.js:39:48)
  at nextResolve (node:internal/modules/esm/loader:163:22)
  at ESMLoader.resolve (node:internal/modules/esm/loader:838:24)
  at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:7)
  at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:21)
```

This came from my test `vrt.spec.ts` which had a non package import
inside of it

```ts
import fs from "node:fs/promises"
```

The test run failed due to node imports not returning fileUrls when
resolved.

---------

Co-authored-by: Nowell Strite <nstrite@nvidia.com>
2023-02-22 14:28:03 -08:00
Max Schmitt ae369144b6
chore: consolidate http/https fetching (#21104)
Fixes https://github.com/microsoft/playwright/issues/20784
Supersedes https://github.com/microsoft/playwright/pull/21076
2023-02-22 17:09:56 +01:00
Pavel Feldman 3f8f2a0fdd
chore: do not use library stack capturer in test runner (#21075) 2023-02-21 19:24:17 -08:00
Pavel Feldman 06fc72b6ed
revert(#12706): also fix related bugs it introduced (#21070) 2023-02-21 14:15:11 -08:00
Andrey Lushnikov ce692830b3
fix: filter out comments inside fixture destructuring (#20989)
Fixes #20944
2023-02-21 12:51:40 -08:00
Yury Semikhatsky 09be9d6425
feat: key value store backed by filesystem (#20932)
Keys are used as a relative file path without any sanitization assuming
that the underlying fs will throw on error.
2023-02-16 16:48:28 -08:00
Andrey Lushnikov 1db04f2839
fix: simplify protocol for toBeInViewport (#20960) 2023-02-16 11:02:19 -08:00
Pavel Feldman 96050a260e
chore(junit): render project as an agent name / hostname (#20927) 2023-02-16 07:59:05 -08:00
Yury Semikhatsky 90c4e6f9b2
chore: revert sharded html report (#20923)
We've decided not to ship it in the current form.

#10437
2023-02-15 12:38:03 -08:00
Pavel Feldman 08be39a80e
fix(watch): fix the tests on win (#20844) 2023-02-14 14:55:49 -08:00
Pavel Feldman 7f0763d789
chore: hide watch mode, it is not ready (#20905) 2023-02-14 13:22:44 -08:00
Dmitry Gozman 6d03211439
fix(test runner): --list should ignore '.only' annotations (#20868) 2023-02-13 11:13:30 -08:00
Andrey Lushnikov fdcd7b549d
chore: mark comparator option back as experimental (#20816)
This reverts commit 303c5998f8.

Reason for revert: I tried enabling `ssim-cie94` by default on
ionic-framework test suite, and it proves to be overly strict for their
usecase.
2023-02-13 11:11:44 -08:00
Joel Einbinder 997dfa9274
fix(reporter): properly indent multiline attachments (#20847)
Text attachments were only indented on the first line.
2023-02-13 10:28:34 -08:00
Pavel Feldman e9ca483666
chore: dim watch separator (#20836) 2023-02-12 10:00:40 -08:00
Dmitry Gozman 8a1612ceec
fix(reporters): preserve the order between deferred errors and stdio (#20830) 2023-02-10 14:58:44 -08:00
Pavel Feldman 1ba768bf60
chore: make watch + ct happy (#20804) 2023-02-10 08:33:25 -08:00
Andrey Lushnikov 68e170ef89
feat: implement expect(locator).toBeInViewport() (#20668)
The method accepts a `ratio` option to assert the ratio
of the element in viewport. `ratio` defaults to `Number.MIN_VALUE`.

NOTE: this reverts commit d950f5b6ee and
adds `ratio` option support + does the rename.

Fixes #8740
2023-02-10 04:33:22 -08:00
Dmitry Gozman ab4cd83e44
chore: move testInfo and timeoutManager to worker/ (#20803) 2023-02-09 19:22:17 -08:00
Pavel Feldman 4469e57695
chore: always grow component bundle (#20799)
Fixes https://github.com/microsoft/playwright/issues/20581
2023-02-09 18:39:20 -08:00
Pavel Feldman e1f287f255
chore: more watch tests (#20797) 2023-02-09 16:03:54 -08:00
Dmitry Gozman 8002baf44f
chore: move option overrides logic to FixturePool (#20795) 2023-02-09 14:50:40 -08:00
Dmitry Gozman 4d37491e9b
chore: split FixtureRunner from fixtures (#20787)
It is only used in worker process.
2023-02-09 11:31:54 -08:00
Pavel Feldman 5112abc1d2
chore: fix a racy time-based test (#20783) 2023-02-09 08:49:51 -08:00
Dmitry Gozman 190c121e66
chore: bundle chokidar, leave fsevents as an optional dependency (#20762) 2023-02-09 08:40:27 -08:00
Pavel Feldman b247bfe153
test(watch): start adding tests (#20764) 2023-02-09 08:31:02 -08:00
Pavel Feldman 6b0a7003c4
chore(watch): run by default again (#20763) 2023-02-08 18:53:07 -08:00
Pavel Feldman 027d6b5239
chore: set filters and run tests separately (#20759) 2023-02-08 14:30:53 -08:00
Pavel Feldman 7a093329fa
chore: do not run all on watch (#20758) 2023-02-08 12:44:51 -08:00
Pavel Feldman 0678b6575f
chore(watch): allow toggling browser (#20738) 2023-02-08 08:36:02 -08:00
Yury Semikhatsky a93cf767a1
feat: html reporter sharded option (#20737)
Make sharded report feature an opt-in:

```ts
{
   reporter: [['html', { sharded: true }]]
};
```

#10437
2023-02-07 22:21:50 -08:00
Pavel Feldman 4259d4e1d6
chore: implement repeat last run (#20727) 2023-02-07 15:56:39 -08:00
Pavel Feldman 96f0674e41
chore: calcular pool digests on the loader process side (#20728) 2023-02-07 14:08:17 -08:00
Yury Semikhatsky 8f53bf7b41
feat: sharded html report (#20700)
This implementation is based on the [original
PR](https://github.com/microsoft/playwright/pull/19691) by @kevin940726.
It makes the reporter produce single file when there is no sharding and
multiple out-of-line report-x-of-y.zip reports which are automatically
merged together when put in one folder.

References https://github.com/microsoft/playwright/issues/10437

Co-authored-by: Kai Hao <kevin830726@gmail.com>
2023-02-07 10:21:26 -08:00
Pavel Feldman 98e348d16a
chore(watch): print current filters (#20696) 2023-02-07 09:48:46 -08:00
Andrey Lushnikov 303c5998f8
feat: release "comparator" option from experiment (#20720)
The option defines a comparator to be used to compare images.
Possible values are `"pixelmatch"` and `"ssim-cie94"`.

Note: This reverts commit 8167f8bf54.
2023-02-07 08:51:48 -08:00
Pavel Feldman 361ea949aa
chore: respect deps when watching files (#20695) 2023-02-06 17:09:16 -08:00
Pavel Feldman 430d08f4fb
chore: watch mode first cut (#20647) 2023-02-06 15:52:14 -08:00
Pavel Feldman b6df48758d
chore: collect test dependencies (#20645) 2023-02-06 14:52:40 -08:00
Andrey Lushnikov 0a752f3fd6
chore(docker): remove experimental docker integration (#20676)
This removes everything related to docker integration experiments that
we conducted over the last 6 months.

I'll send a follow-up with an alternative suggestion that was demo'ed on
a team meeting in the end of December.
2023-02-06 10:50:45 -08:00
Andrey Lushnikov f45f20a8c9
fix: handle not.toPass() with expect.soft inside (#20596)
Fixes #20518
2023-02-03 09:14:05 -08:00
Pavel Feldman ffb719385b
chore: fix oop loading, prepare to watch (#20618) 2023-02-03 09:11:02 -08:00
Pavel Feldman b0b9b08167
chore: fix plugin reporting, rearrange steps (#20616) 2023-02-03 08:44:01 -08:00
Pavel Feldman 8b5e55d432
chore: implement compilation cache (#20597) 2023-02-02 16:46:54 -08:00
Dmitry Gozman 3180bc9804
fix(types): make most fixtures non-undefined (#20573)
Fixes #20416.
2023-02-02 13:11:50 -08:00
Andrey Lushnikov 4f61105032
fix: swallow expect.soft errors inside successful toPass matcher (#20509)
Fixes #20437
2023-02-02 10:47:20 -08:00
Dmitry Gozman 190b0ab609
fix(options): do not default navigationTimeout to actionTimeout (#20578)
Fixes #20549.
2023-02-02 07:36:16 -08:00
Dmitry Gozman 5fb430a743
fix(test runner): do not print timeout message upon interrupt (#20574)
We should only print "Test was interrupted."

Regressed in #18321.
2023-02-01 19:39:43 -08:00
Pavel Feldman 421dd884a7
chore: implement --no-deps (#20569) 2023-02-01 16:32:13 -08:00
Pavel Feldman cb9ace6035
chore(test): move run options into config (#20568) 2023-02-01 15:25:26 -08:00
Pavel Feldman a1cdd939f8
chore: load tests file by file (#20557) 2023-02-01 12:33:42 -08:00
Pavel Feldman 635b47025e
chore: expose project dependencies api (#20546) 2023-02-01 08:39:07 -08:00
Pavel Feldman 08e4b50ff6
chore: project deps (#20514) 2023-01-31 15:59:13 -08:00
Dmitry Gozman b39079b51e
feat(trace viewer): popout snapshot in a new tab (#20475) 2023-01-30 19:07:52 -08:00
Yury Semikhatsky a03f3223c4
fix: trim long expect messages (#20517)
Fixes #20324
2023-01-30 17:24:12 -08:00
Pavel Feldman 0dd090aeab
chore: prepare to deps (#20513) 2023-01-30 14:34:48 -08:00