Commit graph

987 commits

Author SHA1 Message Date
Andrey Lushnikov e57b4b5073
cherry-pick(#12070): fix: propagate exit code in experimental mode (#12133)
SHA: 5db7ce5964

In experimental ESM mode a child process is forked in order to run the tests. Currently the exit code of this child process is not propagated to the exit code of the parent process, which means that the process exits with a status code of `0` even if some of the tests failed.

This makes it difficult to use Playwright in CI in experimental mode, as the CI pipeline as a whole will pass despite the test failures.

This change addresses this by propagating the exit code in the case where it is non-zero.

Co-authored-by: pierscowburn <me@pierscowburn.com>
2022-02-15 13:25:14 -08:00
Yury Semikhatsky d8bc6dbeea
cherry-pick(1.19): always return non-empty body regardless of request method (#12102) (#12121) 2022-02-15 11:24:15 -08:00
Dmitry Gozman ca116db2cb
cherry-pick(#12016): fix(reporters): correctly handle missing stdout.columns (#12033)
When columns are not available, do not trim the output.
2022-02-11 08:58:13 -08:00
Pavel Feldman 35f921e7aa cherry-pick(#12004): chore: revert "fix(test-runner): escape backslashes in win cli" 2022-02-10 08:55:56 -08:00
Andrey Lushnikov 8f1f97f508
cherry-pick(#11984): fix(test-runner): fix browser initialization in test modifiers (#11992)
SHA: 6904b3294e

Fixes #11985
2022-02-09 15:35:36 -08:00
Andrey Lushnikov b651920bd5
cherry-pick(#11974): test: fix tests for chromium-based browser channels (#11979)
SHA: 439c8e9c40
2022-02-09 11:40:50 -08:00
Yury Semikhatsky eaeb7de95a
cherry-pick(#11954): respect tracing config for APIRequestContext (#11976)
706c897031

Fixes #10585
2022-02-09 11:31:10 -08:00
Pavel Feldman 72a767fd4d cherry-pick(#11953): feat(debug): allow preprocessing JS scripts as well 2022-02-09 10:23:00 -08:00
Dmitry Gozman e92caf01b3
fix(webServer): do not set baseURL equal to webServer.url (#11951) 2022-02-08 15:57:36 -08:00
Pavel Feldman 8dff2e35c8
chore: don't throw on no testDir (#11950) 2022-02-08 15:27:05 -08:00
Pavel Feldman b2e15b8c60
chore: mute more test runner extension vars (#11881) 2022-02-08 09:22:45 -08:00
Dmitry Gozman 8a5c93436d
fix(reporter): do not report parallel tests as slow (#11921) 2022-02-07 20:10:13 -08:00
Yury Semikhatsky 1e00218ead
feat(tracing): suport loading multiple files in trace viewer (#11880) 2022-02-07 17:05:42 -08:00
Dmitry Gozman 4ef22d3387
test: enable "should record open in a new tab with url" on webkit (#11917) 2022-02-07 16:09:11 -08:00
Pavel Feldman 9116adc684
chore: allow opt-into the legacy global setup mode (#11888) 2022-02-07 10:41:56 -08:00
Dmitry Gozman f82e09be04
feat(codegen): generate locators and frame locators (#11873) 2022-02-04 19:27:45 -08:00
Dmitry Gozman 1503264aca
test: decrease threshold for flaky "should play audio" (#11878) 2022-02-04 16:14:48 -08:00
Dmitry Gozman af4944388f
test: skip "should support has:locator" on tracing bots (#11876) 2022-02-04 15:34:33 -08:00
Dmitry Gozman c45dacc834
feat(codegen): make selector generator strict (#11856)
This is required to migrate to locators.
2022-02-04 07:34:23 -08:00
Andrey Lushnikov 547a32885b
feat: hide blinking cursor when making screenshots (#11854)
References #9938
2022-02-03 21:44:23 -08:00
Dmitry Gozman c9e99d5d97
test: reenable "should intercept network activity from worker 2" (#11851) 2022-02-03 17:16:40 -08:00
Dmitry Gozman d9a8bb057d
fix(test-fail): allow unhandled expects in test.fail (#11850)
Previously, we would consider this a worker error, but
we make an exception for "expect()" calls.
2022-02-03 17:14:12 -08:00
Yury Semikhatsky d237ad76c1
test: mark cltr+click test as timing out on windows (#11840) 2022-02-03 09:09:38 -08:00
Pavel Feldman fdda759a9d
feat(parallel): allow setting enclosing scope parallel mode (#11822) 2022-02-02 20:44:11 -08:00
Andrey Lushnikov ba0c7e679b
feat(test-runner): support expect.soft (#11800)
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
-  `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.

Fixes #7819
2022-02-02 18:33:51 -08:00
Dmitry Gozman f587a43932
feat(locator): "has" option (#11411)
This introduces `locator('div', { has: locator })` syntax that matches elements containing other elements.
Can be used together with `hasText`.

Internally, has selector engine takes an inner selector escaped with double-quotes:
`div >> has="li >> span >> text=Foo" >> span`.
2022-02-02 16:55:50 -08:00
github-actions[bot] fba523a9d0
feat(chromium): roll to r965416 (#11802)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-02-02 11:39:10 +01:00
Max Schmitt c752b28516
chore: pin production dependencies (#11793) 2022-02-02 11:14:41 +01:00
Andrey Lushnikov b0daa7754f
feat: filter stack traces to exclude test runner frames (#11795)
Before:

```bash
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at Proxy.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/expect.ts:151:30)
        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
        at /Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:335:13
        at runNextTicks (node:internal/process/task_queues:61:5)
        at processImmediate (node:internal/timers:437:9)
        at TestInfoImpl._runFn (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:164:7)
        at WorkerRunner._runTestWithBeforeHooks (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:317:24)
        at TimeoutRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-core/src/utils/async.ts:48:14)
        at TestInfoImpl._runWithTimeout (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:151:7)
        at WorkerRunner._runTestOrAllHook (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:276:5)
        at WorkerRunner._runSuite (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:190:11)
        at WorkerRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:137:9)
        at process.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/worker.ts:87:5)
```

after:

```
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
```
2022-02-01 18:40:44 -08:00
Dmitry Gozman a30bc171c5
chore: make playwright-test tests debuggable with extension (#11798) 2022-02-01 17:15:55 -08:00
Pavel Feldman 011d743f90
chore: roll test runner to ToT (#11796) 2022-02-01 15:34:09 -08:00
Pavel Feldman f875ebe730
fix(test-runner): respect source maps when reporting test files (#11770) 2022-02-01 13:04:54 -08:00
Caio Agiani b6b60decdf
fix: typos (#11789) 2022-02-01 11:09:41 -08:00
Pavel Feldman 3a5e8184b5
fix(html-reporter): open tests from required file (#11784) 2022-02-01 11:01:52 -08:00
Max Schmitt e0c4e66272
chore: bump dev dependencies (#11779) 2022-02-01 17:12:11 +01:00
Pavel Feldman c5d852f1bb
fix(test-runner): escape backslashes in win cli (#11763) 2022-01-31 17:48:05 -08:00
Andrey Lushnikov 2336692e8a
feat: support clarification message for expect (#11735)
The clarification message is displayed in the HTML report as the name of the step:

![image](https://user-images.githubusercontent.com/746130/151852652-48194140-5ea4-439d-afee-12583a8caf71.png)

It is also shown in terminal output:

![image](https://user-images.githubusercontent.com/746130/151852666-5c956ef1-6e94-4bc2-8e55-b58688dfc7e0.png)

Fixes #7816
2022-01-31 17:14:59 -08:00
Pavel Feldman 2b55adaafa
feat(breaking): always report onBegin/onEnd, report file errors (#11758) 2022-01-31 17:09:04 -08:00
Dmitry Gozman 0b04c7d504
fix(drag&drop): relax layout shift logic when dropping (#11760)
When element that is being dragged stays under the mouse,
it prevents the hit target check on drop from working,
because drop target is overlayed by the dragged element.

To workaround this, we perform a one-time hit target check
before moving for the drop, as we used to.
2022-01-31 16:21:35 -08:00
Andrey Lushnikov b58b004f0f
fix: run reporter.onEnd after webserver teardown (#11712)
Fixes #11647
2022-01-31 06:19:33 -08:00
Yury Semikhatsky 401cd9c0ee
fix(fetch): do not throw on empty response body with br encoding (#11708) 2022-01-28 12:58:58 -08:00
Yury Semikhatsky 312ab57f02
test: ctrl+click during provisional load (#11710) 2022-01-28 11:47:14 -08:00
Dmitry Gozman 30f3cbd6da
fix(fixtures): make sure defaultLaunchOptions are always there (#11713) 2022-01-28 10:51:55 -08:00
Yury Semikhatsky 84248f6e48
fix(webkit): handle will/didCheckPolicyForNavigation (#11631) 2022-01-27 14:58:43 -08:00
Yury Semikhatsky d305a2ab3f
fix: throw in route.continue if it is called twice (#11701) 2022-01-27 14:58:02 -08:00
Pavel Feldman 29a84b2df8
chore: allow pre-processing scripts (#11702) 2022-01-27 14:32:23 -08:00
Yury Semikhatsky 81a7b2488f
test: enable showDirectoryPicker test (#11669) 2022-01-27 11:40:06 -08:00
Dmitry Gozman b1fbc4fdbe
fix(test runner): resolve tsconfig for each file (#11662)
This allows us to properly handle path mappings
that are not too ambiguous.
2022-01-26 18:28:42 -08:00
divdavem 512a245f13
feat(test-runner): wait for a url before starting tests (#10138)
The webServer configuration in @playwright/test now accepts a url as an
alternative to a port number to wait for a url to return a 2xx status code.
2022-01-26 16:32:58 -08:00
Yury Semikhatsky ba86dab83a
test: enable filechooser test in chromium (#11653) 2022-01-26 12:39:59 -08:00