We used to not report fatal error and hang forever because worker
did not run any tests but also did not report any errors.
Also properly show stack-less errors.
This makes `test.fail` tests considered as passing when they actually fail:
- Stop restarting the worker.
- Retry when it passes instead of a fail.
- Behaves similar to regular tests in a `describe.serial` suite.
Each hook gets its own test scope. This is not too useful for
object fixtures like `page` (although one can use a page in
`beforeAll` to save storage state), but much more useful for option
fixtures like `viewport`.
- `TestResult.startTime`
- `Suite.location` is optional now
- `Test.status()` renamed to `Test.outcome()` to differentiate against a
`Test.expectedStatus` and `TestResult.status` of the different type.
This makes our suite structure the following:
```
Root(title='') > Project(title=projectName) > File(title=relativeFilePath) > ...suites > test
```
Removed `fullTitle()` because it is not used directly by anyone.
Default reporters now report each test as
```
[project-name] › relative/file/path.spec.ts:42:42 › suite subsuite test title
```
- `Location` with `file`, `line` and `column`.
- `fullTitle` does not include project name.
- `titlePath` method.
- All methods of `Reporter` are optional.
- Removed `Test.skipped` property that is superseeded by `Test.status()`.
- Replaced `Suite.findTest()` with `Suite.allTests()`.
- Removed `Test.suite` property.
We now have Suites and Tests. When running multiple projects the whole
suite is cloned for each project. Same happens for repeatEach.
This simplifies the reporters API, but there is still room for improvement.
JSON reporter continues to produce old json output.
This avoids problems with `toMatchSnapshot('../../dir/file.png')`
where we append this path to `snapshotDir` and end up in some random
place.
Also added a note to documentation.
This fixes two issues:
- Sudden worker process exit is properly accounted for.
- We can stop() workers willy-nilly, e.g. after reaching maxFailures.
Details:
- DonePayload does not send `reamining` anymore, and worker does not track it.
- Instead, `Dispatcher._runJob` track remaining tests and acts accordingly.
- Upon worker exit, we emulate a fatal error for all remaining tests.
Drive-by:
- Do not report onTestBegin after reaching maxFailures to avoid confusion.
Before, we did report onTestBegin, but not onTestEnd.
- List reporter aligned between "running" and "finished" state - it was
one character misplaced.
- Added a SIGINT test.
This fixes an issue where we incorrectly labeled and assigned ids for tests
that declared tests in require'd files or used test wrappers.
See new tests for examples.
- Source now lives at `src/test`.
- Former folio tests live at `tests/playwright-test`.
- We use `src/test/internal.ts` that exposes base test without
Playwright fixtures for most tests (to avoid modifications for now).
- Test types live in `types/testFoo.d.ts`.
- Stable test runner is installed to `tests/config/test-runner` during `npm install`.
- All deps including test-only are now listed in `package.json`.
Non-test deps must also be listed in `build_package.js` to get included.
