Commit graph

1843 commits

Author SHA1 Message Date
Max Schmitt c84c5c8c9b
fix(test-runner): do not list tests to stdout when JSON reporter is used (#7730) 2021-07-20 01:10:43 +02:00
Dmitry Gozman 2ac0c59156
feat(test-runner): rename Test to TestCase (#7725) 2021-07-19 14:54:18 -07:00
Joel Einbinder 56ada374df
feat(dragAndDrop): page.dragAndDrop (#6910) 2021-07-19 12:42:55 -05:00
Joel Einbinder d72efbe181
feat(test-runner): smaller error message paths (#7384) 2021-07-19 12:20:24 -05:00
Joel Einbinder 0cf9cf0829
fix(test-runner): set expand: false for expect. (#7722) 2021-07-19 11:59:53 -05:00
Andrey Lushnikov b2c5bfb72c
chore: nicer message to install dependencies (#7715)
If we know how to install all dependencies, then we should recommend
a Playwright CLI one-liner to install dependencies.

References #7682
2021-07-19 03:54:42 -07:00
Max Schmitt 2ff25522cc
test: rework port forwarding server tests to launch server out of process (#7693) 2021-07-19 07:53:12 +02:00
Dmitry Gozman 66ea613c4d
feat(test-runner): small changes to Reporter api (#7709)
- `TestResult.startTime`
- `Suite.location` is optional now
- `Test.status()` renamed to `Test.outcome()` to differentiate against a
  `Test.expectedStatus` and `TestResult.status` of the different type.
2021-07-18 17:40:59 -07:00
Dmitry Gozman 602d815981
fix(test-runner): list mode should print tests (#7665)
It was not doing anything before.
2021-07-16 22:34:55 -07:00
Dmitry Gozman 167db03f05
feat(test-runner): export reporter api as @playwright/test/reporter (#7692) 2021-07-16 21:15:03 -07:00
Andrey Lushnikov 3c6b058e2b
chore: nicer message to ask for browser installation (#7691)
References #7682
2021-07-16 17:00:27 -07:00
Dmitry Gozman 18be5f5319
feat(test-runner): suite per project (#7688)
This makes our suite structure the following:
```
Root(title='') > Project(title=projectName) > File(title=relativeFilePath) > ...suites > test
```

Removed `fullTitle()` because it is not used directly by anyone.
Default reporters now report each test as
```
[project-name] › relative/file/path.spec.ts:42:42 › suite subsuite test title
```
2021-07-16 15:23:50 -07:00
Pavel Feldman bde764085c
feat(test-runner): introduce attachments (#7685) 2021-07-16 13:48:37 -07:00
Dmitry Gozman 31572fc372
chore(test-runner): misc changes to reporter api (#7664)
- `Location` with `file`, `line` and `column`.
- `fullTitle` does not include project name.
- `titlePath` method.
- All methods of `Reporter` are optional.
- Removed `Test.skipped` property that is superseeded by `Test.status()`.
- Replaced `Suite.findTest()` with `Suite.allTests()`.
- Removed `Test.suite` property.
2021-07-16 12:40:33 -07:00
Joel Einbinder 2f07846c9a
fix(webkit): proper value for event.buttons (#7680) 2021-07-16 13:06:00 -05:00
Andrey Lushnikov 865aff8c9d
feat: support msedge-dev installation (#7679)
Fixes #7677
2021-07-16 09:56:51 -07:00
Andrey Lushnikov 272759f296
feat(chromium): roll Chromium to another new version (#7676) 2021-07-16 09:03:42 -07:00
divdavem 87b0cc530c
feat(cli): add --ignore-https-errors option (#7671) 2021-07-16 07:33:20 -07:00
Dmitry Gozman 8b2dd2e3d1
chore(test-runner): remove the notion of Spec (#7661)
We now have Suites and Tests. When running multiple projects the whole
suite is cloned for each project. Same happens for repeatEach.

This simplifies the reporters API, but there is still room for improvement.

JSON reporter continues to produce old json output.
2021-07-15 22:02:10 -07:00
Max Schmitt 86b21d5299
feat: add High DPI and Low DPI Desktop device descriptors (#7531) 2021-07-15 22:16:27 +02:00
Max Schmitt cf0fb33540
fix: add log entry if selector resolves to multiple elements on click (#7623) 2021-07-15 22:06:08 +02:00
Andrey Lushnikov dd26529b3d
feat(firefox): roll to Firefox 90 (#7652)
Fixes #7583
2021-07-15 10:53:45 -07:00
Joel Einbinder 7e95ee07ae
fix(utils): fix the list dependnecies script (#7649) 2021-07-15 10:27:16 -07:00
Max Schmitt 1b92c674f9
feat: have up2date user-agents for WK and FF (#7644) 2021-07-15 17:41:23 +02:00
Yash Ladha 03b26ec783
chore(android): fix type in DeviceBackend (#7628) 2021-07-14 23:42:09 -07:00
Pavel Feldman 67f5fb6a4f
fix(webkit): add libevdev dep (#7630) 2021-07-14 20:20:03 -07:00
Pavel Feldman 3eb21e2322
feat(webkit): roll to r1515 (#7625) 2021-07-14 18:43:51 -07:00
Dmitry Gozman bb34d7a953
fix(test-runner): sanitize snapshot name before constructing a path (#7620)
This avoids problems with `toMatchSnapshot('../../dir/file.png')`
where we append this path to `snapshotDir` and end up in some random
place.

Also added a note to documentation.
2021-07-14 16:31:19 -07:00
Max Schmitt 6cc2fe178e
feat(test-runner): migrate to launch config/server (#7603) 2021-07-15 01:19:45 +02:00
Max Schmitt a26fe65db3
fix(test-runner): start webServer before globalSetup/teardown (#7604) 2021-07-14 19:01:46 +02:00
Dmitry Gozman d1170b30fc
Revert "api: remove timeout option from isVisible and isHidden methods (#7414)" (#7600)
This reverts commit 9f71c96740.
2021-07-14 10:46:00 +02:00
Dmitry Gozman 0742cb9076
chore: move working with browser channels to Registry Executables (#7581) 2021-07-13 19:03:49 -07:00
Dmitry Gozman 57c5e4d8cf
chore: replace Registry api with Executable list (#7544) 2021-07-13 15:57:40 -07:00
Max Schmitt 767e22c6b2
fix(network): process last inserted routes first in request interception (#7585) 2021-07-13 20:22:01 +02:00
Alex Plischke 27d89732ce
Use correct skip check when generating junit reports (#7571) 2021-07-13 05:05:34 -07:00
Joel Einbinder 9264a8d5b1
fix(chromium): disable use-zoom-for-dsf (#7407) 2021-07-12 22:56:00 -05:00
Joel Einbinder eb31b9e4a9
feat(test-runner): support esm modules in more places (#7542) 2021-07-12 11:59:58 -05:00
Dmitry Gozman 65606c093a
chore: simplify Registry api (#7451) 2021-07-09 16:10:23 -07:00
Max Schmitt 894ff15e65
Revert "chore(eval): merge internal evaluate functions (#7517)" (#7535)
This reverts commit 7a5ef0d157.
2021-07-09 16:19:42 +02:00
Joel Einbinder 7a5ef0d157
chore(eval): merge internal evaluate functions (#7517) 2021-07-08 23:55:01 -07:00
Dmitry Gozman 77deca1d6b
feat(test runner): export testInfo.data (#7525)
This is a key-value storage for any information that goes into the report.
Also export JSONReport types.
2021-07-08 17:16:36 -07:00
Max Schmitt e604f185ca
fix(test-runner): with automatic fixtures workerInfo was undefined after conditional skip (#7521) 2021-07-08 21:55:43 +02:00
Yury Semikhatsky b2742976a8
fix(runner): do not restart worker after skipping tests (#7511) 2021-07-08 09:36:28 -07:00
Max Schmitt ae489b1c43
fix(test-runner): do not override error with unhandled error (#7507) 2021-07-08 18:24:07 +02:00
Max Schmitt 1cc2a2dc59
feat(har): add bodySize, transportSize, headersSize (#7470)
Co-authored-by: tnolet <tim@checklyhq.com>
2021-07-08 18:22:37 +02:00
Dmitry Zakharov 5464ad849e
fix(test-runner): handle negated toMatchSnapshot result (#7345) 2021-07-07 18:51:38 -07:00
Max Schmitt 08da9d207e
chore: move processLauncher into src/utils (#7504) 2021-07-07 21:14:16 +02:00
Dmitry Gozman 2073193c36
fix(test runner): keep track of remaining tests on the runner side (#7486)
This fixes two issues:
- Sudden worker process exit is properly accounted for.
- We can stop() workers willy-nilly, e.g. after reaching maxFailures.

Details:
- DonePayload does not send `reamining` anymore, and worker does not track it.
- Instead, `Dispatcher._runJob` track remaining tests and acts accordingly.
- Upon worker exit, we emulate a fatal error for all remaining tests.

Drive-by:
- Do not report onTestBegin after reaching maxFailures to avoid confusion.
  Before, we did report onTestBegin, but not onTestEnd.
- List reporter aligned between "running" and "finished" state - it was
  one character misplaced.
- Added a SIGINT test.
2021-07-07 12:04:43 -07:00
Max Schmitt 98bcf26656
feat(test-runner): add webServer (#7368) 2021-07-07 20:19:42 +02:00
Max Schmitt ee0497c725
chore: do not log process not found warning on win when PID is not found (#7500) 2021-07-07 19:40:09 +02:00