We used to not report fatal error and hang forever because worker
did not run any tests but also did not report any errors.
Also properly show stack-less errors.
This makes `test.fail` tests considered as passing when they actually fail:
- Stop restarting the worker.
- Retry when it passes instead of a fail.
- Behaves similar to regular tests in a `describe.serial` suite.
fix(test runner): avoid internal error for step end without begin
Consider the following scenario:
- Test finishes and starts tearing down fixtures.
- Fixture teardown starts a step S and then times out.
- We declare the test finished (with timeout).
- Dispatcher shuts down the worker and spins a new one for a retry.
Additionally, it clears steps information for the test to be
ready for the new retry. Step S information is lost.
- Meanwhile, during worker teardown, the step S does
actually finish (usually with an error), and we send stepEnd for S.
- Dispatcher does not know what to do with step S end and
prints an internal error.
The fix is to ignore certain messages from the shutting down worker that failed.
When sharing a context between tests and using `'on-first-retry'` we
could end up with tracing still running in non-retried tests. That's
extra overhead without a reason.
Using a worker fixture forces a new worker. This might be unexpected
when part of the test file runs in one worker, and another runs
in another worker. Top-level use of worker fixtures is still fine.
- Uses some auto fixtures to set default options and instrumentation on BrowserType.
- Moves screenshot, trace and video to worker-scoped fixtures.
- Throws in page/context when used from beforeAll/afterAll.
- Plumbs around BrowserType to be accessible from Browser and BrowserContext.
Each hook gets its own test scope. This is not too useful for
object fixtures like `page` (although one can use a page in
`beforeAll` to save storage state), but much more useful for option
fixtures like `viewport`.
By default, CI's are set up to upload `//test-results` folder to
artifacts storage. Storing missing snapshots in the `//test-results`
folder too will simplify collecting new baselines from the CI.
We used to sort based on workerHash, and that changes depending on
the exact worker fixtures list. Now we replace workerHash with
an ordinal when constructing the TestGroup list to preserve the
natural order.
When running through NPM, we can get multiple SIGINT signals
for a single Ctrl+C - the more npm layers you use, the more SIGINTs you get.
This is a known NPM bug present since at least NPM v6.
Workaround is to give NPM one second for the fake signals, and
only then remove out listener.
