Commit graph

1132 commits

Author SHA1 Message Date
Andrey Lushnikov 88a9262dbb
fix: resolve WebServer cwd dir wrt configDir (#13626)
Drive-by: speed-up webserver tests by not waiting 750ms to start it.

Fixes #13115
2022-04-19 07:45:36 -07:00
Andrey Lushnikov a617604962
test: make sure toHaveScreenshot doesn't tolerate bad option values (#13627)
Fixes #13495
2022-04-19 07:43:18 -07:00
Andrey Lushnikov 8a24c95e08
chore: introduce experimental VRT rebaseline workflow (#13560) 2022-04-19 07:05:43 -07:00
Andrey Lushnikov 80bc532874
fix: support undo/redo editing commands (#13606)
Fixes #13601
2022-04-18 22:22:52 -07:00
Pavel Feldman 3c896f6bc5
fix(tests): installation tests (#13623) 2022-04-18 22:17:56 -07:00
Pavel Feldman 007e908cd9
chore: bundle pwt deps (#13622) 2022-04-18 21:47:18 -07:00
Pavel Feldman 7989427ea6
chore: bundle core deps (#13621) 2022-04-18 20:20:49 -07:00
github-actions[bot] 41db08eca5
feat(webkit): roll to r1631 (#13612) 2022-04-18 18:48:43 -07:00
Pavel Feldman 63ea81ec54
chore: make bundles typed (#13620) 2022-04-18 17:50:25 -07:00
Yury Semikhatsky ed0dcdabc9
docs: clarify toBeDisabled behavior (#13616) 2022-04-18 17:06:01 -07:00
Pavel Feldman 58d79e5e4d
chore: add core utils bundle (#13615) 2022-04-18 13:47:23 -07:00
Pavel Feldman 4f0103fef0
chore: babel, expect, zip bundles (#13588) 2022-04-18 11:31:58 -07:00
Pavel Feldman eafb92f6a3
chore: embed expect types (#13609) 2022-04-18 10:54:40 -07:00
Ross Wollman 71fbd2454e
fix(test-runner): avoid premature stop of worker in fullyParallel (#13584)
Prior to this change, we were pre-maturely stopping a worker (since it
was deemed redundant under a race condition), and then we immediately
created a new worker with the same hash to finish off the test run. The
worker creation is expensive, so this slowed down the overall test run
time.

See the following for logs of the old code illustrating the extra stops and starts: https://gist.github.com/rwoll/1c592ed9e8f9169274fa972674de6703
2022-04-17 16:22:57 -07:00
Pavel Feldman 5f843c347d
chore: bundle pixelmatch, it has not changed for 2 years (#13585) 2022-04-15 16:58:31 -07:00
Yury Semikhatsky 7ffce1da53
chore: remove PlaywrightClient and Docker factory (#13571) 2022-04-15 12:11:38 -07:00
Dmitry Gozman 02aa4fe617
chore: generate types for TestProject/TestConfig (#13574) 2022-04-15 10:27:38 -07:00
Pavel Feldman 09754100aa chore: bump ct versions 2022-04-15 09:22:53 -07:00
Dmitry Gozman 189a5fa0f8
chore: freez FullProject public type (#13570)
This is similar to FullConfig.
Properties `expect` and `fullyParallel` are not exposed.
2022-04-14 16:58:01 -07:00
Yury Semikhatsky aee6ba299a
chore: remove GridClient, run only page tests in service mode (#13566) 2022-04-14 15:30:04 -07:00
Wojciech Jureczka 7ba527c65f
chore: expose JSON Reporter types (#13241)
Resolves #13208
2022-04-14 13:55:29 -07:00
Dmitry Gozman c86c2e8762
feat: restore toHaveScreenshot as experimental feature (#13549)
- Restore docs.
- Make `TestConfig.expect` generated.
- Allow experimental properties with "e" marker: `- foo e<float>`.
2022-04-14 13:22:42 -07:00
Ross Wollman 11179982fc
chore: force localPaths to be resolved (#13544) 2022-04-14 11:19:36 -07:00
github-actions[bot] a29a986d67
feat(chromium): roll to r1002 (#13554)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-14 14:51:03 +02:00
Max Schmitt 584140f97a
fix: make fetch api work with connectOverCDP (#13521) 2022-04-14 12:53:49 +02:00
Dmitry Gozman 20dcc45afa
feat: support experimental doc entries (#13446)
feat: support experimental doc entries

- Params/options/members are marked as experimental in the docs.
- `experimental.d.ts` is generated that contains all types and
  includes experimental features.
- `experimental.d.ts` is references in our tests so that we
  can test experimental features.
- `fonts` option is restored as experimental.
2022-04-13 16:13:30 -07:00
Dmitry Gozman 166675b9c1
feat(test): more information about timing out fixtures (#13546)
- Always show a fixture that was running during timeout.
- Give custom titles to built-in fixtures.
- Specify setup/teardown fixture phase in the message.
- Split connect vs launch browser fixtures for better naming.

Example timeout message:

```log
Timeout of 2000ms exceeded while running fixture "built-in playwright configuration" teardown.
```
2022-04-13 15:13:31 -07:00
Yury Semikhatsky e2fff31848
chore: send run_id from agent to server (#13542) 2022-04-13 12:13:35 -07:00
Yury Semikhatsky 00199b5617
chore: add link to workflow (#13541) 2022-04-13 11:56:00 -07:00
Andrey Lushnikov 269716d7d7
revert(#12877): chore: print error if install-deps is used != ubuntu (#13536)
Reason: turns out Debian Buster requires just one source list to
install `ttf-ubuntu-font-family` font.

All other dependencies are satisfied.

Fixes #13530
2022-04-13 11:03:28 -07:00
Yury Semikhatsky 9b8aceaa61
feat: support os parameter (#13511) 2022-04-12 22:20:48 -07:00
Dmitry Gozman 7a5b070e95
fix(type): focus switch between contenteditables in shadow dom (#13510)
Firefox has a bug: calling `node.focus()` does make the node focused,
but some internal "current contenteditable node" is not changed.
Blurring the previous one and focusing the new one helps.
2022-04-12 16:44:27 -07:00
Yury Semikhatsky ae4d2e75aa
chore: adjust gha agent params (#13509) 2022-04-12 13:39:08 -07:00
Max Schmitt 95d4041b8b
chore: fix mark-docker-image CLI command (#13499) 2022-04-12 10:10:34 -07:00
Andrey Lushnikov 0f6638190e
fix: do not require --force flag when installing channel on CI (#13487) 2022-04-11 13:42:21 -07:00
Dmitry Gozman 1e1df6395f
chore: generate expect types (#13439) 2022-04-11 10:42:19 -07:00
github-actions[bot] d90d2a6c0e
feat(webkit): roll to r1630 (#13443) 2022-04-11 08:54:30 -07:00
Dmitry Gozman f6ccd4847e
fix(test runner): default workerIndex to -1 (#13440) 2022-04-08 15:23:23 -07:00
Yury Semikhatsky 9712b9ee08
fix: match against secure path (#13442) 2022-04-08 14:57:43 -07:00
Yury Semikhatsky 212c665152
fix: pass full grid url to agent (#13438) 2022-04-08 14:15:38 -07:00
Dmitry Gozman 916dd1e698
feat(remote): handle timeout manually in browser fixture (#13406)
To make launch/connect not affect the test timeout:
- `browser` fixture is declared with `timeout: 0`.
- Default timeouts are set to 30sec for launch and 3min for connect.
2022-04-08 13:39:12 -07:00
Ross Wollman 1af32e400f
feat(test-runner): introduce GlobalInfo (#13083) 2022-04-08 13:22:14 -07:00
Pavel Feldman f0156d057e
chore: flatten supplements (#13437) 2022-04-08 12:52:40 -07:00
kaivean d65263f151
feat(android): allow getting webviews by socket name (#13248) 2022-04-08 20:52:16 +02:00
github-actions[bot] d91349f22a
feat(webkit): roll to r1629 (#13407) 2022-04-08 11:50:53 -07:00
Ross Wollman db7bd8ebd2
chore: don't auto-install browsers if global install (#13299) 2022-04-08 10:46:24 -07:00
github-actions[bot] e72975d9c6
feat(chromium): roll to r1001 (#13424)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-08 18:39:42 +02:00
Dmitry Gozman dfb1584703
chore: remove MEMUSAGE workaround on Windows (#13408)
This was introduced in #7500 to fight `ERROR: The process "4436" not found.`
messages when killing a process that did already exit.

Since then, we no longer inherit stdout/stderr, so the error message
should not appear anymore.
2022-04-08 09:37:51 -07:00
Pavel Feldman 845e32cbb8
chore: use utils via index export (6) (#13417) 2022-04-07 21:48:41 -07:00
Pavel Feldman e79b90f454
chore: use utils via index export (5) (#13413) 2022-04-07 20:18:22 -07:00
Dmitry Gozman 155bb7fcae
fix: wait for cleanup on double SIGINT (#13411)
This is a speculative fix to leftover tmp directories.
When users issues SIGINT twice, we enter `gracefullyClose()`
twice, and shortcut the second time. It turns out, we do
not wait for directories removal.

Note: it is unknown how often we reach this codepath in practice.
2022-04-07 19:20:54 -07:00
Dmitry Gozman cef476b89f
docs: support optional methods (#13415)
So far, these are used in `Reporter`.
2022-04-07 18:51:05 -07:00
Andrey Lushnikov 91333daf09
chore: cut v1.21.0 (#13405) 2022-04-07 18:16:22 -07:00
Yury Semikhatsky f803a929e4
feat: pass server address as argument (#13412) 2022-04-07 17:22:52 -07:00
Pavel Feldman a3c02222bf
chore: use utils via index export (4) (#13404) 2022-04-07 14:36:13 -07:00
Pavel Feldman 40d5e3a3c9
chore: use utils via index export (3) (#13403) 2022-04-07 13:55:44 -07:00
Max Schmitt 5536e64538
feat(test-runner): support filtering by columns (#13401) 2022-04-07 22:45:45 +02:00
Pavel Feldman 6c1fa62000
chore: use utils via index export (2) (#13399) 2022-04-07 13:28:30 -07:00
Pavel Feldman dbafe3bb4e
chore: use utils via index export (1) (#13398) 2022-04-07 13:26:50 -07:00
Dmitry Gozman b7116906fc
feat(remote): support headless/x-playwright-headless (#13391)
This only applies when remote launches the browser.
2022-04-07 10:19:56 -07:00
Steve Zhang de518338ce
docs: fix class-coverage snippet for v8toIstanbul (#13371)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-04-07 18:34:45 +02:00
Pavel Feldman 90d5990e75
chore: fix installation scripts (#13376) 2022-04-06 23:09:10 -07:00
Pavel Feldman bde7bf4ea9
chore: move registry & dispatchers to under server (#13370) 2022-04-06 22:21:27 -07:00
Yury Semikhatsky 32d30ae71d
chore: fix assert imports (#13359) 2022-04-06 21:30:48 -07:00
Dmitry Gozman c68f57d2d0
docs: mark optional properties/arguments explicitly with "?" (#13369)
Previously, we were guessing whether something is optional from the
comment (looking for stuff like "defaults to" or "optional").

To remove this error-prone technique, every optional property or
argument (but not option) is explicitly marked with "?".
2022-04-06 19:02:10 -07:00
Yury Semikhatsky fccc14cdfa
feat: github agent, browser worker (#13336) 2022-04-06 18:03:27 -07:00
Pavel Feldman c80365dd43
chore: distill deps after import types update (#13367) 2022-04-06 15:41:08 -07:00
Pavel Feldman 5ae2017a5b
chore: always import type (#13365) 2022-04-06 14:57:14 -07:00
Pavel Feldman 1c3db811f5
chore: move to folder-based deps lists (#13361) 2022-04-06 14:40:19 -07:00
Ivan Kaliada 424de6c38f
fix: resolve ts compilerOptions.paths with prefixes and suffixes (#13105) 2022-04-06 14:14:03 -07:00
Dmitry Gozman 4123a55be5
chore: generate more types (#13358)
As a result, fix documentation issues:
- Removed `TestStep.data` that is not a thing.
- Added `TestConfig.name` documentation.
- Refined a lot of optional types in the documentation.
- Aligned `test.fail()` and `test.slow()` docs with `test.skip()`.
- Strict set of docs vs types inconsistensies in the generator
2022-04-06 13:36:20 -07:00
Akul Srivastava d9d826b7f6
docs: fixes typo (#13094)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-04-06 12:00:51 +02:00
Dmitry Gozman faaac318f4
chore: remove elementsFromPoint workaround (#13184)
The issue was fixed upstream in r982637.
2022-04-05 19:37:07 -07:00
Pavel Feldman f3bd910820
chore(test-runner): rewrite poll, soft, wrap (#13335) 2022-04-05 17:47:35 -07:00
Dmitry Gozman 4bb563b015
fix(expect): proper return types (#13334)
A few changes:
- `Matchers<R, T>` now carries both return and argument type.
- Based on the argument type, we apply playwright-specific Page/Locator matchers.
- Return type is usually void, unless wrapped with `expect.resolves`,
  `expect.rejects` or `expect.poll()`.
- To preserve compatibility with any extended types in the wild,
  argument type is optional.
2022-04-05 16:11:11 -07:00
Pavel Feldman 6ca58e18cb
fix(electron): better support for custom schemas (#13329) 2022-04-05 16:10:12 -07:00
github-actions[bot] 67989e01d1
feat(webkit): roll to r1625 (#13333) 2022-04-05 15:34:59 -07:00
Yury Semikhatsky cdef8171c1
chore: extract remote connection to a separate file (#13331) 2022-04-05 14:47:11 -07:00
github-actions[bot] 67c251f483
feat(chromium): roll to r1000 (#13327)
References #13198

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-05 11:05:01 -07:00
Max Schmitt 844ecbe2eb
chore: better Electron error message when no executable path is given (#13318) 2022-04-05 18:45:21 +02:00
Dmitry Gozman e31a5b690a
chore: unrelease toHaveScreenshot and screenshotsDir (#13304) 2022-04-05 08:34:51 -07:00
Pavel Feldman a2f94f6336
fix: trace viewer pwa should have no base (#13308) 2022-04-04 21:53:19 -07:00
Pavel Feldman 23d9dbc011
chore: rewrite non-web protocol for Electron tracing (#13305) 2022-04-04 20:56:04 -07:00
Dmitry Gozman a71c151438
chore: introduce FullProjectInternal (#13301)
This is similar to FullConfigInternal to put private stuff on.
2022-04-04 16:07:04 -07:00
Dmitry Gozman a09b8dfa36
chore: unrelease 'fonts' screenshot option (#13300) 2022-04-04 14:51:11 -07:00
Thomas Scholtes 3e65ef35cf
fix: wait for worker to be ready before initializing (#13270)
fix: wait for worker to be ready before initializing

The test dispatcher waits until a worker is ready before sending the
`init` message. This guarantees that the worker process is listening for the
message. Otherwise, the worker process might miss the `init` message and
the subsequent `run` message would crash the worker.
2022-04-04 13:56:57 -07:00
Dmitry Gozman 4e1fb1728f
docs: small clarifications to hasText and has-text (#13298) 2022-04-04 13:18:03 -07:00
Pavel Feldman 1bebc28aed
chore: reset page/context for reuse in component tests (#13264) 2022-04-04 12:39:43 -07:00
Pavel Feldman 8232497c88
feat(electron): expose app process(), detach on exit (#13280) 2022-04-04 11:50:46 -07:00
Ross Wollman 42798b5857
chore: warn against using globally scoped install (#13196) 2022-04-03 20:26:23 -07:00
Pavel Feldman b0103566c9
fix(addInitScript): tolerate trailing comments (#13275) 2022-04-03 18:47:12 -07:00
Pavel Feldman bcb12fcf7f
chore: mark public methods on server as such (#13271) 2022-04-02 19:02:27 -07:00
kaivean 5d2e8918d8
feat(android): add androidDevice.options.omitDriverInstall (#13249) 2022-04-02 16:00:38 -07:00
Dmitry Gozman f6936c0461
chore: move private fields to FullConfigInternal (#13261) 2022-04-01 18:32:34 -07:00
Pavel Feldman 66cf82766e
test(html-reporter): add image diff tests (#13262) 2022-04-01 18:11:15 -07:00
Pavel Feldman 55ee41c848
feat(html): render image diff slider (#13257) 2022-04-01 15:27:51 -07:00
Ross Wollman 12abae7f31
feat(test-runner): friendly expect errors for typos (#13229)
If you typo'd an `expect` property, you got a cryptic error message:

```
Uncaught TypeError: Cannot create proxy with a non-object as target or handler
```

Now we get this nice friendly message:

```
  1) a.spec.ts:6:9 › explodes ======================================================================

    Error: expect: Property 'toBeLessThen' not found.

    Did you mean 'toBeLessThan'?

    See https://playwright.dev/docs/test-assertions for available options and documentation.

      5 |         const { test } = pwt;
      6 |         test('explodes', () => {
    > 7 |           expect.soft(1).toBeLessThen();
        |           ^
      8 |         });
      9 |

```

Fixes #13218
2022-04-01 13:38:22 -07:00
Pavel Feldman c262a80762
chore: extract image diff component (#13221) 2022-04-01 12:53:56 -07:00
Ross Wollman 16efbdef98
chore: remove/rename FullConfig._attachments (#13233)
This was originally introduced in #12734.

It will be replaced with GlobalInfo (#13083), but not before the 1.21
release.
2022-04-01 12:36:05 -07:00
Dmitry Gozman a200fe3528
feat(screenshot): rename "size" option to "scale" (#13254)
Drive-by: fix `caret` handling in `toHaveScreenshot`.
2022-04-01 12:28:40 -07:00