Commit graph

223 commits

Author SHA1 Message Date
Andrey Lushnikov d22bde13c4
chore: mark v1.19.1 (#12132) 2022-02-15 13:25:32 -08:00
Andrey Lushnikov aafaa2b9ed
chore: mark v1.19.0 (#11981) 2022-02-11 09:42:54 -08:00
Dmitry Gozman ca116db2cb
cherry-pick(#12016): fix(reporters): correctly handle missing stdout.columns (#12033)
When columns are not available, do not trim the output.
2022-02-11 08:58:13 -08:00
Pavel Feldman 55be85284c cherry-pick(#12012): fix(teardown): await teardown in failed test runs 2022-02-10 12:47:06 -08:00
Pavel Feldman 35f921e7aa cherry-pick(#12004): chore: revert "fix(test-runner): escape backslashes in win cli" 2022-02-10 08:55:56 -08:00
Andrey Lushnikov 8f1f97f508
cherry-pick(#11984): fix(test-runner): fix browser initialization in test modifiers (#11992)
SHA: 6904b3294e

Fixes #11985
2022-02-09 15:35:36 -08:00
Yury Semikhatsky eaeb7de95a
cherry-pick(#11954): respect tracing config for APIRequestContext (#11976)
706c897031

Fixes #10585
2022-02-09 11:31:10 -08:00
Pavel Feldman 72a767fd4d cherry-pick(#11953): feat(debug): allow preprocessing JS scripts as well 2022-02-09 10:23:00 -08:00
Dmitry Gozman e92caf01b3
fix(webServer): do not set baseURL equal to webServer.url (#11951) 2022-02-08 15:57:36 -08:00
Pavel Feldman 8dff2e35c8
chore: don't throw on no testDir (#11950) 2022-02-08 15:27:05 -08:00
Pavel Feldman d5158e8d24
chore: revert source-map-support to 0.4.18 (#11945) 2022-02-08 14:50:20 -08:00
Pavel Feldman 4bcca2c87e
chore: add a parallel mode note (#11943) 2022-02-08 14:36:14 -08:00
Dmitry Gozman 8a5c93436d
fix(reporter): do not report parallel tests as slow (#11921) 2022-02-07 20:10:13 -08:00
Dmitry Gozman 7912c515a3
fix(fixtures): account for default options being undefined (#11916) 2022-02-07 17:11:36 -08:00
Pavel Feldman 9116adc684
chore: allow opt-into the legacy global setup mode (#11888) 2022-02-07 10:41:56 -08:00
Yury Semikhatsky 46dfa45b4e
docs: fix parallel test docs build (#11877) 2022-02-04 16:44:04 -08:00
Yury Semikhatsky afc4b505aa
docs: fix duplicate sections (#11874) 2022-02-04 13:50:46 -08:00
Dmitry Gozman d9a8bb057d
fix(test-fail): allow unhandled expects in test.fail (#11850)
Previously, we would consider this a worker error, but
we make an exception for "expect()" calls.
2022-02-03 17:14:12 -08:00
Pavel Feldman 72424dc904
chore: allow setting reporter via env (#11848) 2022-02-03 16:10:39 -08:00
Pavel Feldman fdda759a9d
feat(parallel): allow setting enclosing scope parallel mode (#11822) 2022-02-02 20:44:11 -08:00
Andrey Lushnikov ba0c7e679b
feat(test-runner): support expect.soft (#11800)
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
-  `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.

Fixes #7819
2022-02-02 18:33:51 -08:00
Max Schmitt c752b28516
chore: pin production dependencies (#11793) 2022-02-02 11:14:41 +01:00
Andrey Lushnikov b0daa7754f
feat: filter stack traces to exclude test runner frames (#11795)
Before:

```bash
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at Proxy.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/expect.ts:151:30)
        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
        at /Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:335:13
        at runNextTicks (node:internal/process/task_queues:61:5)
        at processImmediate (node:internal/timers:437:9)
        at TestInfoImpl._runFn (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:164:7)
        at WorkerRunner._runTestWithBeforeHooks (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:317:24)
        at TimeoutRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-core/src/utils/async.ts:48:14)
        at TestInfoImpl._runWithTimeout (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:151:7)
        at WorkerRunner._runTestOrAllHook (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:276:5)
        at WorkerRunner._runSuite (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:190:11)
        at WorkerRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:137:9)
        at process.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/worker.ts:87:5)
```

after:

```
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
```
2022-02-01 18:40:44 -08:00
Pavel Feldman ceb3027bdf
chore(test-runner): force colors in worker processes (#11799) 2022-02-01 17:29:15 -08:00
Pavel Feldman 6e2fcc4700
chore: do not expose suite load error (#11797) 2022-02-01 15:34:16 -08:00
Pavel Feldman f875ebe730
fix(test-runner): respect source maps when reporting test files (#11770) 2022-02-01 13:04:54 -08:00
Max Schmitt 028afb167b
chore: bump production dependencies (#11787) 2022-02-01 21:27:34 +01:00
Pavel Feldman a3bc911a3d
fix(debug): don't apply infinite timeouts when using debugger (#11785) 2022-02-01 11:51:37 -08:00
Caio Agiani b6b60decdf
fix: typos (#11789) 2022-02-01 11:09:41 -08:00
Pavel Feldman 3a5e8184b5
fix(html-reporter): open tests from required file (#11784) 2022-02-01 11:01:52 -08:00
Pavel Feldman c82f2641d7
fet(list-files): report per-project test dir and filters (#11764) 2022-02-01 08:08:56 -08:00
Pavel Feldman c5d852f1bb
fix(test-runner): escape backslashes in win cli (#11763) 2022-01-31 17:48:05 -08:00
Andrey Lushnikov 2336692e8a
feat: support clarification message for expect (#11735)
The clarification message is displayed in the HTML report as the name of the step:

![image](https://user-images.githubusercontent.com/746130/151852652-48194140-5ea4-439d-afee-12583a8caf71.png)

It is also shown in terminal output:

![image](https://user-images.githubusercontent.com/746130/151852666-5c956ef1-6e94-4bc2-8e55-b58688dfc7e0.png)

Fixes #7816
2022-01-31 17:14:59 -08:00
Pavel Feldman 2b55adaafa
feat(breaking): always report onBegin/onEnd, report file errors (#11758) 2022-01-31 17:09:04 -08:00
Andrey Lushnikov 3a4e506479
chore: extract SigIntWatcher (#11749)
This is to reduce the size of the long `_run` method in the `runner.ts`.
It also might come handy around the codebase.
2022-01-31 08:51:22 -08:00
Andrey Lushnikov b58b004f0f
fix: run reporter.onEnd after webserver teardown (#11712)
Fixes #11647
2022-01-31 06:19:33 -08:00
Dmitry Gozman eeebcd53ae
chore(test runner): extract TestInfoImpl (#11725) 2022-01-28 17:39:42 -08:00
Dmitry Gozman 30f3cbd6da
fix(fixtures): make sure defaultLaunchOptions are always there (#11713) 2022-01-28 10:51:55 -08:00
Pavel Feldman 29a84b2df8
chore: allow pre-processing scripts (#11702) 2022-01-27 14:32:23 -08:00
Feng Yu 0698254253
docs: fix broken link (#11687)
fix #11680
2022-01-27 13:00:09 -08:00
Dmitry Gozman b1fbc4fdbe
fix(test runner): resolve tsconfig for each file (#11662)
This allows us to properly handle path mappings
that are not too ambiguous.
2022-01-26 18:28:42 -08:00
divdavem 512a245f13
feat(test-runner): wait for a url before starting tests (#10138)
The webServer configuration in @playwright/test now accepts a url as an
alternative to a port number to wait for a url to return a 2xx status code.
2022-01-26 16:32:58 -08:00
Dmitry Gozman 19820de7a9
fix(test runner): correctly save videos when running remotely (#11633) 2022-01-26 07:43:07 -08:00
Dmitry Gozman 4f42484c64
chore(test runner): simplify runTestWithBeforeHooks (#11629) 2022-01-25 16:46:50 -08:00
Dmitry Gozman 800b813d4b
chore(test runner): prepare to per-fixture timeout (#11605)
This reworks DeadlineRunner to use exception to signal timeout. This way,
we'll be able to run fixtures against a shared deadline vs their own
deadline and still get an easy control-flow timeout handling.
2022-01-25 11:22:28 -08:00
Andrey Lushnikov 42876a0528
fix: support PWDEBUG=0 to disable debug (#11611)
Fixes #11606
2022-01-25 08:40:24 -08:00
Ross Wollman 64e7557fb9
fix: falsey behavior in route.continue, page.post, testInfo.attach (#11421)
In several of the Playwright APIs, falsey values were not handled correctly. This changeset adds tests (and some fixes):

- route.continue: If options.postData was the empty string, the continue failed to override the post data.
- page.post (application/json with options.data: false|''|0|null): Raw falsey values were getting dropped (i.e. you can't do the equivalent of curl --header application/json … -d 'false'). This has been fixed with most values across all browsers, but an additional fix is needed for 'null' which the channel serializer treats extra specially.
- testInfo.attach: This didn't get reported as an error when options.path was the empty string, but should have been.
#11413 (and its fix #11414) inspired this search as they are the same
class of bug.
2022-01-24 15:06:36 -08:00
Pavel Feldman 9542b007cf
fix(esm): don't emit module shorthand warning (#11596) 2022-01-24 11:22:56 -08:00
Pavel Feldman 0b99d83911
chore: include config dir in quick list report (#11563) 2022-01-22 17:02:46 -08:00
Pavel Feldman baf0e5b47a
chore: implement lightweight test listing (#11551) 2022-01-21 19:11:22 -08:00