Filip/playwright
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
6426 commits 34 branches 141 tags 223 MiB
Commit graph

241 commits

Author SHA1 Message Date
Pavel Feldman 6afa85927e
chore(ui): redesign html report a bit (#9577) 2021-10-17 20:58:06 -07:00
Pavel Feldman 99f3389b3b
feat(html): automatically use console reporter with html (#9559) 2021-10-15 19:18:56 -07:00
Pavel Feldman 9135847950
feat(ui): merge stdio in html report (#9560) 2021-10-15 16:15:06 -07:00
Yury Semikhatsky 235cd10a43
fix(runner): clarify where test.use() can be called (#9486) 2021-10-15 12:03:26 -07:00
Pavel Feldman 75cfe5d1f5
chore: start adding html runner e2e tests (#9518) 2021-10-14 15:48:05 -07:00
Pavel Feldman 584014f6fa
feat(html): unhide html reporter (#9512) 2021-10-14 11:17:35 -07:00
Yury Semikhatsky e827bde1c2
fix(runner): skip global hooks in --list mode (#9480) 2021-10-13 14:18:15 -07:00
Dmitry Gozman 64a3099655
chore(expect): simplify expect implementation (#9459) 2021-10-13 08:56:57 -07:00
Pavel Feldman 876e08315b
fix(expect): continue polling in array-based matchers, fix edge cases (#9430) 2021-10-11 14:01:09 -07:00
Max Schmitt b1160ec239
chore(test-runner): make outputDir based on cwd instead of testDir (#9360) 2021-10-11 22:24:46 +02:00
Joel Einbinder d34b53a0d7
feat(test runner): allow comparing binary files to strings (#9315) 
Co-authored-by: Max Schmitt <max@schmitt.mx>
 2021-10-11 21:57:59 +02:00
Max Schmitt 76142a33ae
test: fix tests which include special signal handling (#9433) 2021-10-11 12:07:40 -07:00
Max Schmitt 9dd6bb8c6a Revert "feat(test-runner): allow parametrized tests that don't extend base (#9301)" 
This reverts commit f63af830de.
 2021-10-11 17:32:31 +02:00
Joel Einbinder c89d5a50dd
chore: migrate to monorepo (#9421) 2021-10-11 16:52:17 +02:00
Max Schmitt b69b8889fe test: fix broken relative outputDir test 
#9380
 2021-10-09 00:12:21 +02:00
Max Schmitt 804ee6e979
chore: roll stable-test-runner to ToT (#9397) 2021-10-08 18:37:06 +02:00
Max Schmitt b3a8ccd064
test(test-runner): add test for relative outputDir (#9381) 2021-10-08 09:20:41 +02:00
Pavel Feldman 083d882b2b
test: add missing request tests (#9343) 2021-10-06 10:15:13 -07:00
Pavel Feldman bc71d20d0f
feat(request): add global request fixture (#9332) 2021-10-06 10:09:27 -07:00
Pavel Feldman f63af830de
feat(test-runner): allow parametrized tests that don't extend base (#9301) 2021-10-04 17:16:33 -07:00
Sidharth Vinod f4cbd9d574
test: remove checking duration in github reporter to reduce flakiness 2021-10-04 15:53:26 +02:00
Sidharth Vinod be30f9f1c4
feat(test-runner): Add GitHub Actions reporter (#9191) 2021-10-04 10:32:56 +02:00
Pavel Feldman 913821f675
chore: roll Electron to 12.2.1 (#9271) 2021-10-01 19:40:47 -07:00
Nick Partridge b126a5685b
feat: add path option to toMatchSnapshot (#9156) 2021-10-01 09:15:44 -07:00
Max Schmitt e674d873a3
feat(test-runner): add reuse context mode to share a single context between tests (#9115) 2021-10-01 09:16:03 +02:00
Dmitry Gozman 5e3ad63b42
fix(test runner): do not write missing snapshot until the last retry (#9246) 
This prevents future retries from passing because of the actual
snapshot being written.

In theory, we can avoid running the retry since it should fail anyway.
However, this brings problems, for example in the `describe.serial` mode
where running a test also has some side effects and so it should not be
skipped. Since running a test without a snapshot is rare, it should be
fine to retry it.
 2021-09-30 16:44:52 -07:00
Sidharth Vinod fcb7d2b15a
feat(reporters): Add error position to JSON Report (#9151) 2021-09-30 14:18:36 -07:00
Pavel Feldman f78302e8dd
fix(toBeHidden): return true to missing elements (#9205) 2021-09-28 17:11:04 -07:00
Dmitry Gozman ed9b42a92d
feat(test runner): collect test error from worker teardown (#9190) 
When the test fails (usually with timeout), we wait until all hooks are run
and worker scope is teared down before reporting test end result.

This allows us to collect any error details populated by teardown
in addition to the "timed out" message.
 2021-09-28 10:56:50 -07:00
Dmitry Gozman fa536786f2
fix(test runner): proper serial mode with beforeAll/afterAll failures (#9183) 2021-09-27 15:58:26 -07:00
Dmitry Gozman 8dc8777ab4
feat(expect): toContainText(array) (#9160) 
This matches when each expected item from the array
is matched to one of the resolved elements, in order.
Note this performs both "sub-array" and "substring" matching.

Drive-by: documentation fixes.
Drive-by: added "selector resolved to 3 elements" log line
when expecting arrays.
 2021-09-27 11:14:35 -07:00
Max Schmitt cd22072685
chore: enable object-curly-spacing in ESLint (#9168) 2021-09-27 18:58:08 +02:00
Pavel Feldman 241411ad42
chore: render expect in trace viewer (#9141) 2021-09-27 09:19:59 -07:00
Max Schmitt 4573ce0cf8
chore(test-runner): launch -> webServer (#9167) 2021-09-27 11:32:57 +02:00
Dmitry Gozman d22dd4a4e7
feat(test runner): improve fixture typings for function fixtures (#9138) 
When fixture value `R` is a function, TypeScript sometimes confuses
function `R` and function `async ({}, use) => {}`. This leads to
`any` types in the latter because it could be either of the functions
as TS thinks.

The solution is to only accept the second syntax, assuming that noone
passes fixture value that is a function as is:

```js
// This will stop working.
test.extend<{ foo: (x: number) => number }>({
  foo: x => 2 * x,
});

// This will get inferred types and autocomplete.
test.extend<{ foo: (x: number) => number }>({
  foo: async ({}, use) => {
    await use(x => 2 * x);
  },
});
```
 2021-09-24 19:59:30 -07:00
Dmitry Gozman f4aaebfba0
fix(expect): produce "waiting for selector" log, corner cases (#9140) 2021-09-24 18:55:45 -07:00
Pavel Feldman db6c55af51
chore: add a test for the log scale polling (#9136) 2021-09-24 12:54:33 -07:00
Pavel Feldman 0908dc98c8
chore: migrate expect(locator).toMatchText to protocol (#9117) 2021-09-23 16:46:46 -07:00
Dmitry Gozman 568ec05a97
fix(junit): produce a single system-out entry (#9091) 
Instead of multiple `system-out` entries we produce a single one
with concatenated content. This is compatible with various junit xml
parsers in the wild.
 2021-09-22 15:28:57 -07:00
Dmitry Gozman 018467911b
test: introduce some common test fixtures (#9060) 2021-09-21 16:24:48 -07:00
Max Schmitt f9c5279c3a
fix(test-runner): toHaveURL respect baseURL (#9050) 2021-09-21 21:41:24 +02:00
Dmitry Gozman 7ec1035b98
test: improve child process utilities in tests (#9036) 2021-09-20 17:17:12 -07:00
Pavel Feldman 63ff405e6e
fix(inspector): stop on all snapshottable actions (#8990) 2021-09-17 15:24:15 -07:00
Dmitry Gozman 43213614a1
fix(test runner): after hooks step should not be nested (#8969) 2021-09-16 15:51:27 -07:00
Dmitry Gozman 5379b2dcba
fix(test runner): account for errors with inconsistent stack/message (#8950) 2021-09-15 21:28:36 -07:00
Darrell Breeden d82cb9a2ff
feat: addition of tags to JSON reporter output (#8920) 2021-09-15 12:30:22 -07:00
Pavel Feldman 14bc663742
chrome: don't nest pw:api steps (#8932) 2021-09-15 11:34:23 -07:00
Dmitry Gozman b8a46580dd
fix(expect): toHaveText, toContainText and toHaveTitle normalize whitespace (#8929) 2021-09-14 19:24:29 -07:00
Pavel Feldman 5253a7eb54
feat(html): restore trace, video, screenshot (#8925) 2021-09-14 16:26:31 -07:00
Pavel Feldman 16baaa317d
chore: remove stale html experiments (#8905) 2021-09-13 20:34:46 -07:00
Pavel Feldman 1925c85dfb
feat(report): render attachment as a part of failure (#8903) 2021-09-13 18:07:40 -07:00
Pavel Feldman b76e993951
chore: remove log from htmlreport (#8898) 2021-09-13 18:07:15 -07:00
Dmitry Gozman d9d2d809a2
fix(test): make use in config accept option values only (#8828) 
Also include default options in FullConfig/FullProject.
Also make examples compile and add a test.
 2021-09-13 17:50:31 -07:00
Dmitry Gozman ed34a67d4a
fix(junit reporter): put stdio under testcase (#8900) 
When output happened during test execution, it should be
under `<testcase><system-out>...</system-out></testcase>`.
 2021-09-13 17:50:08 -07:00
Dmitry Gozman 440651e05c
test(test runner): calling test.step() from fixture (#8899) 2021-09-13 17:49:58 -07:00
Dmitry Gozman 6c41ad3329
test: expect(locator).toHaveCount should wait (#8901) 2021-09-13 17:06:19 -07:00
Joel Einbinder bf35da3656
fix(test-runner): accept unix separators even on windows (#8881) 
.
 2021-09-13 12:09:38 -04:00
Dmitry Gozman bcb0c1745b
fix(waitForEventInfo): reply from the server side (#8825) 
Otherwise, client thinks that `waitForEventInfo` is a pending operation.
 2021-09-09 21:20:26 -07:00
Pavel Feldman 665143d629
chore(report): don't generate file per test (#8822) 2021-09-09 17:35:31 -07:00
Pavel Feldman 7bbb63d143
feat(test): start authoring raw reporter (#8790) 2021-09-09 14:17:18 -07:00
Yury Semikhatsky afe92a6fcf
fix(runner): avoid line wrapping in list reporter (#8705) 2021-09-07 17:42:17 -07:00
Daniel Dyssegaard Kallick b2c59a15dd
feat(test-runner): include test step in json report (#8476) 2021-09-07 13:35:30 -07:00
Dmitry Gozman 3739113e74
fix(test runner): allow dot-files and dot-directories with tests (#8751) 2021-09-07 10:32:47 -07:00
Max Schmitt e40b805782
feat(test-runner): support baseURL in toHaveURL (#8743) 2021-09-07 18:34:02 +02:00
Yury Semikhatsky e5e461c0de
fix: use logger from config if specified (#8697) 2021-09-03 14:21:25 -07:00
Dmitry Gozman e2b092c1a0
feat(html reporter): show log for pw:api and expect steps (#8692) 2021-09-03 13:08:17 -07:00
Dmitry Gozman 2b4a93972d
test: use ToT test runner for non-test-runner tests (#8671) 2021-09-03 11:22:25 -07:00
Dmitry Gozman 620712a5d9
feat(expect): support array of RegExp objects in toHaveText/toHaveClass (#8667) 2021-09-02 15:48:04 -07:00
Dmitry Gozman e691b649de
feat(test runner): describe.parallel (#8662) 2021-09-02 15:42:07 -07:00
Fokke Zandbergen 0ae38b5aec
fix(test-runner): WebServer: use socket for reuseExistingServer to detect servers that aren't using SO_REUSEADDR (#8537) 2021-09-02 18:39:41 +02:00
Yury Semikhatsky bafa426231
feat(runner): support multiple names in project filter (#8600) 2021-09-02 09:29:55 -07:00
Dmitry Gozman b1260602ac
docs: update global setup docs (#8637) 
Changing example to "authenticate once", and also updating the auth doc.
Adding a test with the same setup.
 2021-09-01 15:35:46 -07:00
Dmitry Gozman 4f4cf448c2
fix(test runner): generate unique outputDir for beforeAll/afterAll (#8633) 2021-09-01 13:41:35 -07:00
Dmitry Gozman 1a9215a6ec
fix(html reporter): do not throw when attachment is actually missing (#8630) 2021-09-01 12:20:28 -07:00
Dmitry Gozman b8f8ca7493
feat(tracing): tracing.{start,stop}Chunk instead of tracing._export (#8521) 2021-08-31 17:03:31 -07:00
Dmitry Gozman bee8ed117b
feat(html reporter): preview source code, steps and step errors (#8598) 2021-08-31 16:34:52 -07:00
Dmitry Gozman 900362ec0b
fix(test runner): report unhandled rejections during worker teardown (#8592) 2021-08-31 10:50:30 -07:00
Dmitry Gozman 37a897c9b4
fix(test runner): report beforeAll timeout instead of hanging (#8529) 
We used to not report fatal error and hang forever because worker
did not run any tests but also did not report any errors.

Also properly show stack-less errors.
 2021-08-28 07:19:45 -07:00
Michael Rienstra ed0c47a06f
fix(test-runner): minor error message typo (#8489) 2021-08-27 10:15:16 +02:00
Dmitry Gozman a479cb6f52
fix(test runner): add an overload for test.skip(title, fn) (#8454) 
We shipped this feature, but forgot to add the right overload to d.ts.
 2021-08-25 14:36:36 -07:00
Dmitry Gozman de85d8bb83
fix(test runner): do not special case test.fail (#8447) 
This makes `test.fail` tests considered as passing when they actually fail:
- Stop restarting the worker.
- Retry when it passes instead of a fail.
- Behaves similar to regular tests in a `describe.serial` suite.
 2021-08-25 12:19:50 -07:00
Dmitry Gozman 75fb77355a
test(test runner): check retries with beforeAll failure and multiple tests (#8413) 2021-08-24 12:22:16 -07:00
Joel Einbinder a8a3799e9d
fix(test-runner): use describe name in output dir (#8282) 2021-08-24 10:33:40 -04:00
Dmitry Gozman f9b87268a7
fix(test runner): afterAll error should not mask beforeAll error (#8358) 2021-08-23 09:21:40 -07:00
Pavel Feldman 0997c13151
fix(test-runner): do not attach non-existent diff (#8297) 2021-08-20 13:40:27 -07:00
Dmitry Gozman e5be2c9205
feat(test runner): show failure details for flaky test runs (#8332) 
Currently, we just say "foo.spec.ts > my test" is flaky, but do not
show how exactly the failed run went.
 2021-08-19 18:20:53 -07:00
Dmitry Gozman 70eca0401d
fix(test runner): avoid internal error for step end without begin (#8327) 
fix(test runner): avoid internal error for step end without begin

Consider the following scenario:
- Test finishes and starts tearing down fixtures.
- Fixture teardown starts a step S and then times out.
- We declare the test finished (with timeout).
- Dispatcher shuts down the worker and spins a new one for a retry.
  Additionally, it clears steps information for the test to be
  ready for the new retry. Step S information is lost.
- Meanwhile, during worker teardown, the step S does
  actually finish (usually with an error), and we send stepEnd for S.
- Dispatcher does not know what to do with step S end and
  prints an internal error.

The fix is to ignore certain messages from the shutting down worker that failed.
 2021-08-19 17:04:09 -07:00
Pavel Feldman 710cec80a0
feat(test-runner): render step titles (#8270) 2021-08-17 16:41:36 -07:00
Pavel Feldman 97ba4f22f3
feat(test-runner): report nested steps (#8266) 2021-08-17 13:57:26 -07:00
Dmitry Gozman 2aff06ec73
fix(test runner): make sure tracing is not running on non-retries (#8232) 
When sharing a context between tests and using `'on-first-retry'` we
could end up with tracing still running in non-retried tests. That's
extra overhead without a reason.
 2021-08-16 16:46:35 -07:00
Dmitry Gozman d32d50a906 fix(test runner): make sure we always teardown all fixtures (#8158) 
Even if one of the fixtures throws, we should teardown all of them
so that we can run afterAll hooks.
 2021-08-12 09:08:56 -07:00
Pavel Feldman d2d71c4cdb
fix(reporter): group fixture initialization under before hooks (#8072) 2021-08-12 07:58:00 -07:00
Dmitry Gozman 44cdda43fe
feat(test runner): show stdio for failures in terminal reporters (#8150) 2021-08-11 16:44:19 -07:00
Dmitry Gozman 052e0e197c
fix(types): allow specifying scope when overriding fixtures (#8139) 
Otherwise it show a confusing error.
 2021-08-11 10:44:15 -07:00
Dmitry Gozman a5e0965087
feat(test runner): implement test.describe.serial (#8132) 2021-08-10 21:26:45 -07:00
Joel Einbinder 64da74fba8
feat(test-runner): allow non-ascii characters in the output dir path (#8093) 2021-08-10 21:24:35 -07:00
Dmitry Gozman c9c1ea6546
fix(test runner): disallow use(workerFixture) in describes (#8119) 
Using a worker fixture forces a new worker. This might be unexpected
when part of the test file runs in one worker, and another runs
in another worker. Top-level use of worker fixtures is still fine.
 2021-08-10 16:32:32 -07:00
Dmitry Gozman 9d6c7cdf20
fix(test runner): ensure we run after hooks after failures (#8102) 2021-08-10 10:54:05 -07:00
Dmitry Gozman 2744cd6c9a
feat(test runner): remove createContext fixture (#8109) 
We can now use `browser.newContext()` or `browser.newPage()` instead.
 2021-08-10 09:26:36 -07:00
Dmitry Gozman 3bf3318350
fix(test runner): make sure options, trace and screenshot apply to all contexts (#8083) 
- Uses some auto fixtures to set default options and instrumentation on BrowserType.
- Moves screenshot, trace and video to worker-scoped fixtures.
- Throws in page/context when used from beforeAll/afterAll.
- Plumbs around BrowserType to be accessible from Browser and BrowserContext.
 2021-08-09 18:09:11 -07:00