Commit graph

7187 commits

Author SHA1 Message Date
Pavel Feldman 1215057ca1
chore: use ipc transport for out-of-process driver (#11826) 2022-02-02 21:26:45 -08:00
Pavel Feldman fdda759a9d
feat(parallel): allow setting enclosing scope parallel mode (#11822) 2022-02-02 20:44:11 -08:00
Andrey Lushnikov ba0c7e679b
feat(test-runner): support expect.soft (#11800)
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
-  `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.

Fixes #7819
2022-02-02 18:33:51 -08:00
Dmitry Gozman f587a43932
feat(locator): "has" option (#11411)
This introduces `locator('div', { has: locator })` syntax that matches elements containing other elements.
Can be used together with `hasText`.

Internally, has selector engine takes an inner selector escaped with double-quotes:
`div >> has="li >> span >> text=Foo" >> span`.
2022-02-02 16:55:50 -08:00
Yury Semikhatsky 55b9d14bbd
browser(webkit): revert r288284 (#11821) 2022-02-02 15:38:55 -08:00
Leonardo Henrique Tsuda de8f22a8ff
docs(selectors.md): fix fill and FillAsync examples (#11790) 2022-02-02 12:10:34 -08:00
dependabot[bot] f6dd9b0c11
chore(deps): bump nanoid from 3.1.29 to 3.2.0 (#11778)
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.29 to 3.2.0.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.29...3.2.0)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-02 16:15:18 +01:00
Max Schmitt 20450b96ff
devops: fix broken emojis on TG on Windows bot (#11810) 2022-02-02 16:14:21 +01:00
github-actions[bot] fba523a9d0
feat(chromium): roll to r965416 (#11802)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-02-02 11:39:10 +01:00
Max Schmitt c752b28516
chore: pin production dependencies (#11793) 2022-02-02 11:14:41 +01:00
Andrey Lushnikov b0daa7754f
feat: filter stack traces to exclude test runner frames (#11795)
Before:

```bash
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at Proxy.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/expect.ts:151:30)
        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
        at /Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:335:13
        at runNextTicks (node:internal/process/task_queues:61:5)
        at processImmediate (node:internal/timers:437:9)
        at TestInfoImpl._runFn (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:164:7)
        at WorkerRunner._runTestWithBeforeHooks (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:317:24)
        at TimeoutRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-core/src/utils/async.ts:48:14)
        at TestInfoImpl._runWithTimeout (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:151:7)
        at WorkerRunner._runTestOrAllHook (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:276:5)
        at WorkerRunner._runSuite (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:190:11)
        at WorkerRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:137:9)
        at process.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/worker.ts:87:5)
```

after:

```
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
```
2022-02-01 18:40:44 -08:00
Pavel Feldman ceb3027bdf
chore(test-runner): force colors in worker processes (#11799) 2022-02-01 17:29:15 -08:00
Dmitry Gozman a30bc171c5
chore: make playwright-test tests debuggable with extension (#11798) 2022-02-01 17:15:55 -08:00
Zack Guo 4bbd38560a
docs(CONTRIBUTING.md): update how to run a specific test (#11697) 2022-02-01 16:08:08 -08:00
Sanghyeon Lee 731931ae58
docs(test-timeouts-js.md): fix inconsistent style of 'override' (#11766) 2022-02-01 15:59:30 -08:00
Pavel Feldman 6e2fcc4700
chore: do not expose suite load error (#11797) 2022-02-01 15:34:16 -08:00
Pavel Feldman 011d743f90
chore: roll test runner to ToT (#11796) 2022-02-01 15:34:09 -08:00
Yury Semikhatsky e7bd62f202
docs(electron): minimal version (#11792) 2022-02-01 14:45:30 -08:00
Dmitry Gozman 4a52bc9465
docs: add option fixtures section to release notes (#11794) 2022-02-01 14:34:47 -08:00
Pavel Feldman f875ebe730
fix(test-runner): respect source maps when reporting test files (#11770) 2022-02-01 13:04:54 -08:00
Max Schmitt e7120d2ca2
docs(python): add API testing guide (#11775) 2022-02-01 21:44:43 +01:00
Max Schmitt 028afb167b
chore: bump production dependencies (#11787) 2022-02-01 21:27:34 +01:00
Pavel Feldman a3bc911a3d
fix(debug): don't apply infinite timeouts when using debugger (#11785) 2022-02-01 11:51:37 -08:00
Caio Agiani b6b60decdf
fix: typos (#11789) 2022-02-01 11:09:41 -08:00
Pavel Feldman 3a5e8184b5
fix(html-reporter): open tests from required file (#11784) 2022-02-01 11:01:52 -08:00
Yury Semikhatsky c2f6462a6b
fix(trace-viewer): info message for actions without snapshots (#11765) 2022-02-01 09:26:38 -08:00
Max Schmitt e0c4e66272
chore: bump dev dependencies (#11779) 2022-02-01 17:12:11 +01:00
Pavel Feldman c82f2641d7
fet(list-files): report per-project test dir and filters (#11764) 2022-02-01 08:08:56 -08:00
Pavel Feldman c5d852f1bb
fix(test-runner): escape backslashes in win cli (#11763) 2022-01-31 17:48:05 -08:00
Andrey Lushnikov 2336692e8a
feat: support clarification message for expect (#11735)
The clarification message is displayed in the HTML report as the name of the step:

![image](https://user-images.githubusercontent.com/746130/151852652-48194140-5ea4-439d-afee-12583a8caf71.png)

It is also shown in terminal output:

![image](https://user-images.githubusercontent.com/746130/151852666-5c956ef1-6e94-4bc2-8e55-b58688dfc7e0.png)

Fixes #7816
2022-01-31 17:14:59 -08:00
Pavel Feldman 2b55adaafa
feat(breaking): always report onBegin/onEnd, report file errors (#11758) 2022-01-31 17:09:04 -08:00
Andrey Lushnikov 1dc0ddffce
browser(chromium): roll Chromium to an upstream fix (#11762)
References #11754
2022-01-31 16:36:15 -08:00
Dmitry Gozman 0b04c7d504
fix(drag&drop): relax layout shift logic when dropping (#11760)
When element that is being dragged stays under the mouse,
it prevents the hit target check on drop from working,
because drop target is overlayed by the dragged element.

To workaround this, we perform a one-time hit target check
before moving for the drop, as we used to.
2022-01-31 16:21:35 -08:00
Yury Semikhatsky bec050c4c4
browser(webkit): add missing include on windows (#11759) 2022-01-31 13:39:40 -08:00
Yury Semikhatsky 129d9d3643
docs(tracing): clarify capture semantics (#11756) 2022-01-31 11:44:10 -08:00
Diego Pino 5a413166b6
browser(webkit): roll to 01/31/22 (#11744) 2022-01-31 11:09:52 -08:00
github-actions[bot] 22989ffb98
feat(firefox-beta): roll to r1316 (#11675)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-01-31 19:06:09 +01:00
William Bergeron-Drouin 5a14aad368
docs: fix instructions for custom matchers type declarations (#11483)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-01-31 18:58:45 +01:00
Andrey Lushnikov a7e73cc389
feat: compute md5 hash for CDN uploads. (#11750)
By default, azure auto-computes MD5 for all blobs < 64MBs.
However, many of our binaries exceed 64MB so md5 has to be computed
manually.

With this patch, MD5 hash will be set to all our CDN uploads and could
be retrieved as `content-md5` header, e.g.:

```bash
$ curl -I https://playwright.azureedge.net/builds/ffmpeg/1001/ffmpeg-win32.zip
```

Fixes #10173
2022-01-31 09:13:43 -08:00
Andrey Lushnikov 3a4e506479
chore: extract SigIntWatcher (#11749)
This is to reduce the size of the long `_run` method in the `runner.ts`.
It also might come handy around the codebase.
2022-01-31 08:51:22 -08:00
Andrey Lushnikov b58b004f0f
fix: run reporter.onEnd after webserver teardown (#11712)
Fixes #11647
2022-01-31 06:19:33 -08:00
github-actions[bot] f0fc566004
browser(chromium): roll to r965044 (#11746)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-01-31 11:56:55 +01:00
Xiaoxing Ye a35c249fdc
fix: quote path to prevent space issue (#11733)
Co-authored-by: Xiaoxing Ye <xiaoye@microsoft.com>
2022-01-29 09:56:58 -08:00
dependabot[bot] 5635e840f8
chore(deps): bump node-fetch in /utils/flakiness-dashboard (#11732)
Bumps [node-fetch](https://github.com/node-fetch/node-fetch) from 2.6.1 to 2.6.7.
- [Release notes](https://github.com/node-fetch/node-fetch/releases)
- [Commits](https://github.com/node-fetch/node-fetch/compare/v2.6.1...v2.6.7)

---
updated-dependencies:
- dependency-name: node-fetch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-29 08:15:26 -08:00
Andrey Lushnikov 031219a757
chore: work-around npm install on windows (#11726)
Since yesterday all our windows bots are dying while trying to
install npm@8.

Turns out this is due to recent release of npm@8.4. This patch
moves our CI to use npm@8.3

https://github.com/npm/cli/issues/4341
2022-01-29 07:56:50 -08:00
Dmitry Gozman eeebcd53ae
chore(test runner): extract TestInfoImpl (#11725) 2022-01-28 17:39:42 -08:00
Yury Semikhatsky 401cd9c0ee
fix(fetch): do not throw on empty response body with br encoding (#11708) 2022-01-28 12:58:58 -08:00
Leonardo Henrique Tsuda 7dc2dce709
fix response status assertion on python example (#11723) 2022-01-28 11:50:14 -08:00
Yury Semikhatsky 312ab57f02
test: ctrl+click during provisional load (#11710) 2022-01-28 11:47:14 -08:00
Marcus Felling ac36608a69
docs(README.md): align with docs (#11717)
* Include other languages in intro, update capabilities and install steps to align with docs
2022-01-28 13:43:06 -06:00