Filip/playwright
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
7109 commits 34 branches 141 tags 223 MiB
Commit graph

203 commits

Author SHA1 Message Date
Dmitry Gozman 192071d5bc
feat(test runner): save traces for beforeAll/afterAll hooks (#10950) 2021-12-15 16:06:10 -08:00
Dmitry Gozman 19b08332ce
feat(test runner): disable test timeout on page.pause() call (#10944) 2021-12-15 11:12:52 -08:00
Dmitry Gozman 0d54afab9c
feat(test runner): show beforeAll/afterAll hooks similar to tests (#10923) 
Reporters now get notified about hooks start/end/steps.
 2021-12-15 10:39:49 -08:00
Pavel Feldman f579f9c806
chore: parse tsx tests (#10917) 2021-12-14 19:25:07 -08:00
Dmitry Gozman 34b84841b0
chore(test runner): create TestResult instances lazily in dispatcher (#10921) 
This prepares for beforeAll/afterAll hooks to be handled in the same way.
Since we do not know in advance whether a hook will run, we must create
TestResults lazily.
 2021-12-14 14:10:56 -08:00
Dmitry Gozman 5c4ebdce54
fix(line reporter): clarify about retries when going over total counter (#10901) 2021-12-13 19:06:13 -08:00
Joel Einbinder c27491cd4d
feat(test-runner): shorten long output paths (#10523) 2021-12-13 10:56:03 -08:00
Pavel Feldman b5933db279
feat(tsconfig): respect baseUrl and paths from tsconfig (#10525) 2021-12-08 22:43:00 -08:00
Pavel Feldman 4d683cef7f
fix(html): render text attachments as text (#10778) 2021-12-08 08:51:44 -08:00
Pavel Feldman a08a41f6c9
chore: render annotations in html report (#10774) 2021-12-07 18:35:06 -08:00
Pavel Feldman feb4c62da1
fix(html): html reporter fixes (#10770) 2021-12-07 16:47:47 -08:00
Pavel Feldman 518d67add5
feat(test.info): expose information on the currently running test (#10708) 2021-12-06 09:25:11 -08:00
Pavel Feldman 31e0a63fcd
feat(toBeChecked): allow passing checked: false (#10665) 2021-12-02 10:31:26 -08:00
Pavel Feldman b9731a904e
chore: add validations into check_deps (#10661) 2021-12-01 18:14:13 -08:00
Saransh Miglani f05252874a
chore: minor code rearrangement (#10650) 2021-12-01 09:18:16 -08:00
Yury Semikhatsky d66b7aab3b
feat(expext): toBeOK for APIResponse (#10596) 2021-11-30 18:12:19 -08:00
Dmitry Gozman 729da65eba
fix(test runner): allow multiple missing snapshots per test (#10621) 
Instead of failing right away, continue test execution but mark
the test as failed.
 2021-11-30 17:50:19 -08:00
Max Schmitt 682d1f41c7
fix(test-runner): wrong test-runner options got determined (#10607) 
Follow-up for #10507
 2021-11-30 19:59:35 +01:00
Max Schmitt 9337aa3cdf
fix(test-runner): webServer port detection on Node.js 17 (#10538) 2021-11-29 19:36:35 +01:00
Joel Einbinder 15053d9b3b
fix(test-runner): apply default options to all browserTypes (#10507) 2021-11-29 12:21:15 -05:00
Joel Einbinder 20c0facfb9
fix(test runner): dont mangle test names with multiple dashes (#10447) 2021-11-24 19:36:38 -05:00
Pavel Feldman 7eb3f76f49
feat(esm): allow running tests in type module projects (#10503) 2021-11-24 12:42:48 -08:00
Max Schmitt 85197e68c9
chore: support range requests in trace viewer http server (#10434) 2021-11-23 21:37:55 +01:00
Ross Wollman 854f321532
feat(api): add explicit async testInfo.attach (#10121) 
feat(api): add explicit async testInfo.attach

We add an explicit async API for attaching file paths (and Buffers) to
tests that can be awaited to help users ensure they are attaching files
that actually exist at both the time of the invocation and later when
reporters (like the HTML Reporter) run and package up test artifacts.

This is intended to help surface attachment issues as soon as possible
so you aren't silently left with a missing attachment
minutes/days/months later when you go to debug a suddenly breaking test
expecting an attachment to be there.

NB: The current implemntation incurs an extra file copy compared to
manipulating the raw attachments array. If users encounter performance
issues because of this, we can consider an option parameter that uses
rename under the hood instead of copy. However, that would need to be
used with care if the file were to be accessed later in the test.
 2021-11-23 09:30:53 -08:00
Dmitry Gozman 0302e759df
feat(test runner): allow top-level test.fixme similar to test.skip (#10250) 
```js
test.fixme('my test name', () => {});
```
 2021-11-19 11:40:40 -08:00
Dmitry Gozman d9f849fb14
feat(test runner): replace declare/define with "options" (#10293) 
1. Fixtures defined in test.extend() can now have `{ option: true }` configuration that makes them overridable in the config. Options support all other properties of fixtures - value/function, scope, auto.
```
const test = base.extend<MyOptions>({
  foo: ['default', { option: true }],
});
```

2. test.declare() and project.define are removed.

3. project.use applies overrides to default option values and nothing else. Any test.extend() and test.use() calls take priority over config options.

Required user changes: if someone used to define fixture options with test.extend(), overriding them in config will stop working. The solution is to add `{ option: true }`.

```
// Old code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: 123,
  myFixture: ({ myOption }, use) => use(2 * myOption),
});

// New code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: [123, { option: true }],
  myFixture: ({ myOption }, use) => use(2 * myOption),
});
```
 2021-11-18 15:45:52 -08:00
Dmitry Gozman 8f43f4c98f
feat(serial): better errors from beforeAll (#10419) 
When beforeAll hook times out or fails with an exception, we now
close the context and show a nice error.
 2021-11-18 14:36:55 -08:00
Dmitry Gozman bd93fc499f
fix(html reporter): show missing attachments as warnings (#10400) 2021-11-17 18:03:13 -08:00
Dmitry Gozman ce2c0c59a7
feat(expect): show expect timeout in the error message (#10388) 
Makes it easier to understand that expect does indeed have a separate timeout.

```
    Error: expect(received).toHaveCount(expected) // deep equality

    Expected: 0
    Received: 1

    Call log:
      - expect.toHaveCount with timeout 500ms
      - waiting for selector "span"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
```
 2021-11-17 17:28:30 -08:00
Max Schmitt 0781d0303b
docs(python): enable web-first assertions (#10390) 2021-11-18 00:46:30 +01:00
Dmitry Gozman 638ebd6dd6
fix(test runner): do not validate fixtures in tests/hooks that are never run (#10328) 2021-11-15 13:17:26 -08:00
Dmitry Gozman 4bb123d4b7
fix(test runner): route more errors to reporter (#10263) 
For example, top-level errors in files or global setup issues.
 2021-11-12 06:47:41 -08:00
Andrey Lushnikov bd1ce399e3
fix(html-report): fix command to open HTML report (#10231) 2021-11-12 00:12:23 -08:00
Pavel Feldman 8dcd77270f
fix(sigint): remove sigint handler early (#10266) 2021-11-11 17:14:36 -08:00
Dmitry Gozman 7eec66d0f9
chore(test runner): route runner errors through Reporter.onError (#10257) 2021-11-11 16:48:08 -08:00
Dmitry Gozman f38f611478
chore: add npm run lint-tests (#10252) 2021-11-11 13:27:50 -08:00
Dmitry Gozman 4c93417e8a
fix(github reporter): handle global onError (#10256) 2021-11-11 13:25:38 -08:00
Dmitry Gozman 9622704a8a
fix(test runner): update TestInfo.duration before running afterEach hooks (#10228) 2021-11-10 16:02:27 -08:00
Pavel Feldman 7b64161a37
feat(test-runner): allow specifying fine-grained trace options (#10147) 2021-11-08 15:39:58 -08:00
Joel Einbinder 0cad0de3e3
fix(test runner): better error message when importing typescript from esmodule (#10061) 2021-11-08 16:25:40 -05:00
Dmitry Gozman 0a104bc500
docs: clarify that slow tests are about files (#10134) 2021-11-08 12:54:18 -08:00
Dmitry Gozman 3c1aaa5338
fix(trace): do not call tracing.stopChunk() twice (#10054) 2021-11-04 21:08:42 -07:00
Dmitry Gozman 3120f81629
fix(reporters): move Running X tests using Y workers to reporters (#10014) 2021-11-03 11:17:23 -07:00
Dmitry Gozman 9cebe60831
feat(reporters): augment non-stdio reporters with dot/line (#10003) 2021-11-03 08:25:16 -07:00
Andrey Lushnikov 62a01a41c6
feat: various docker integration improvements (#9988) 
- shorter docker terminal message
- terminate testrunner if docker cannot be found
- use `-docker` suffix for snapshots / screenshots.
 2021-11-02 13:58:26 -07:00
Pavel Feldman 6a30c90590
feat(trace-viewer): render wall time for each action (#9982) 2021-11-02 12:16:12 -07:00
Nick Partridge a51ac39275
feat: add snapshotDir to set base snapshot directory (#9260) 2021-11-02 08:02:49 -07:00
Pavel Feldman 56ca3a18f5
feat(trace-viewer): show test name (#9957) 2021-11-01 21:23:35 -07:00
Pavel Feldman 3673776330
Revert "chore(test-runner): use test name as trace file name prefix (#9619)" (#9956) 
This reverts commit 3c420a7cf1.
 2021-11-01 20:27:41 -07:00
Pavel Feldman 85497c5509
feat(tracing): allow including source files in traces (#9946) 2021-11-01 19:57:29 -07:00
Pavel Feldman 2bfbf65b8d
fix(html): strip ansi escaping from stdio (#9944) 2021-11-01 16:39:54 -07:00
Pavel Feldman 9ac8829583
feat(html): bake report zip into the html report, allow opening from fs (#9939) 2021-11-01 16:14:52 -07:00
Pavel Feldman d79aae633c
feat(html): render the suite name before test title (#9909) 2021-11-01 10:53:42 -07:00
Dmitry Gozman 4f1027bdd0
feat(test runner): introduce TestInfo.parallelIndex (#9762) 
This is a worker number between `0` and `workers - 1` that
does not change after worker process restart.
 2021-11-01 10:37:34 -07:00
Max Schmitt 3c420a7cf1
chore(test-runner): use test name as trace file name prefix (#9619) 2021-11-01 18:15:19 +01:00
Dmitry Gozman 13844a5b0a
feat(html reporter): open test details page when running a single test (#9910) 2021-11-01 09:54:53 -07:00
Pavel Feldman 8991bbde33
feat(html): live filtering, opt-out from auto-open (#9889) 2021-10-29 16:24:08 -07:00
Dmitry Gozman 227e3da62f
fix(test runner): do not reuse worker that did not teardown scopes (#9872) 
Two bug fixes:
- Do not use the worker that is being shutdown for a new job.
- Report unhandled errors during "expected to fail" tests as
  fatal errors.
 2021-10-29 13:36:12 -07:00
Pavel Feldman 2e4722d460
test: migrate to upstream fixtures (#9835) 2021-10-28 08:31:30 -07:00
Dmitry Gozman 22e4a0d580
fix(snapshots): --update-snapshots + retries should work (#9799) 2021-10-26 13:50:16 -07:00
Pavel Feldman 273122b761
chore: align internal test harness w/ @playwright/test (#9796) 2021-10-26 13:45:53 -07:00
Pavel Feldman 87c64b2c1c
chore: align client side instrumentations (#9771) 2021-10-26 11:13:35 -07:00
Pavel Feldman 5451808df1
chore: append sources to traces (#9736) 2021-10-25 18:56:57 -07:00
Dmitry Gozman 13ed1dee50
fix(test runner): do not mask uncaught error in beforeEach (#9764) 2021-10-25 14:17:27 -07:00
Pavel Feldman f08c22b467
fix(trace-viewer): show source files in local version (#9732) 2021-10-23 11:23:39 -07:00
Joel Einbinder f2888395a6
fix: support even older node 12 (#9718) 2021-10-22 15:59:52 -04:00
Dmitry Gozman 193c79a685
feat(test runner): rewrite dispatcher to avoid unneded stalls (#9629) 
In some circumstances, dispatcher was waiting for all exisitng jobs
to finish before scheduling a new one. This leads to unneded stalls.
Instead, we can schedule jobs right away, if we have a worker
available.
 2021-10-22 11:10:37 -07:00
Pavel Feldman 225145fc3e
fix(expect): do not fail on navigated frames while polling (#9659) 2021-10-20 13:01:05 -07:00
Pavel Feldman b6a9a8a34a
Revert "chore: print global errors when running html report (#9657)" (#9661) 
This reverts commit 68c9fce507.
 2021-10-20 12:51:12 -07:00
Pavel Feldman 68c9fce507
chore: print global errors when running html report (#9657) 2021-10-20 11:57:21 -07:00
Dmitry Gozman fb421e0a65
feat(github reporter): update docs, strip ansi escapes (#9640) 2021-10-20 07:56:03 -07:00
Pavel Feldman 4423de9996
chore: speed up snippet generation 50 times (#9638) 2021-10-19 21:10:14 -07:00
Andrey Lushnikov 983cfde4d4
feat: introduce docker integration for Playwright Test (#9599) 2021-10-19 16:10:24 -07:00
Pavel Feldman 9d03a85c30
chore: make web-first matchers work outside of pw tests (#9624) 2021-10-19 14:46:56 -07:00
Joel Einbinder 53c866c0f2
fix(monorepo): explicitly list package.json exports instead of using glob patterns (#9584) 2021-10-19 12:28:02 -04:00
Dmitry Gozman 6d554a5e30
feat(reporter): expose more apis (#9603) 2021-10-19 08:38:04 -07:00
Pavel Feldman bccd4c8906
feat(api): expose step location UI (#9605) 2021-10-18 22:14:01 -07:00
Pavel Feldman c06a6e1f63
fix(html): always highlight error code (#9604) 2021-10-18 21:37:19 -07:00
Pavel Feldman e37660b068
feat(api): expose step location (#9602) 2021-10-18 21:06:18 -07:00
Dmitry Gozman 4977edcaf3
fix(tracing): fix inconsistent calls of tracing.{start,stop} in fixtures (#9594) 
Drive-by: fix error logging.
 2021-10-18 21:05:59 -07:00
Pavel Feldman 99a2bd604d
chore: minor UI updates (#9590) 2021-10-18 13:34:02 -07:00
Pavel Feldman 432fb453e4
chore(html): only copy trace viewer for reports with traces (#9579) 2021-10-18 08:03:04 -07:00
Pavel Feldman 6afa85927e
chore(ui): redesign html report a bit (#9577) 2021-10-17 20:58:06 -07:00
Pavel Feldman 99f3389b3b
feat(html): automatically use console reporter with html (#9559) 2021-10-15 19:18:56 -07:00
Pavel Feldman 9135847950
feat(ui): merge stdio in html report (#9560) 2021-10-15 16:15:06 -07:00
Pavel Feldman e31b96cc26
feat(tracing): make context.request appear in the trace (#9555) 2021-10-15 15:22:49 -07:00
Yury Semikhatsky 235cd10a43
fix(runner): clarify where test.use() can be called (#9486) 2021-10-15 12:03:26 -07:00
Pavel Feldman 458945821f
doc(html): document html reporter (#9528) 2021-10-15 08:15:30 -07:00
Pavel Feldman 5ea4ec2f7a
feat(html): open show html report when there are failures (#9526) 2021-10-14 21:09:41 -07:00
Dmitry Gozman c711fb35ad
fix(expect): do not rely on displayName (#9523) 
Support for displayName was removed in Node 16.
Switching to Function.name instead.

Relevant V8 change: https://chromium-review.googlesource.com/c/v8/v8/+/2692189
 2021-10-14 19:23:45 -07:00
Pavel Feldman 75cfe5d1f5
chore: start adding html runner e2e tests (#9518) 2021-10-14 15:48:05 -07:00
Pavel Feldman 584014f6fa
feat(html): unhide html reporter (#9512) 2021-10-14 11:17:35 -07:00
Joel Einbinder c8f875a193
chore: move test runner types into the playwright test package (#9472) 
Co-authored-by: Max Schmitt <max@schmitt.mx>
 2021-10-14 11:55:08 +02:00
Pavel Feldman bb77912aee
chore: don't include babel compiled files in webpack bundles (#9491) 2021-10-13 19:27:50 -07:00
Yury Semikhatsky e827bde1c2
fix(runner): skip global hooks in --list mode (#9480) 2021-10-13 14:18:15 -07:00
Joel Einbinder 2a0a44bc09
chore: move diff_match_patch into the test package (#9475) 2021-10-13 13:38:26 -07:00
Pavel Feldman cd99ad0da2
feat(trace): allow multiple traces in a single app, gc traces (#9478) 2021-10-13 13:31:54 -07:00
Pavel Feldman 8b1a887756
feat(html): link traces from html report (#9473) 2021-10-13 11:07:29 -07:00
Dmitry Gozman 64a3099655
chore(expect): simplify expect implementation (#9459) 2021-10-13 08:56:57 -07:00
Yury Semikhatsky 82ff85b106
fix(matchers): make isNot required in protocol (#9436) 2021-10-12 09:01:46 -07:00
Max Schmitt b1160ec239
chore(test-runner): make outputDir based on cwd instead of testDir (#9360) 2021-10-11 22:24:46 +02:00
Joel Einbinder d34b53a0d7
feat(test runner): allow comparing binary files to strings (#9315) 
Co-authored-by: Max Schmitt <max@schmitt.mx>
 2021-10-11 21:57:59 +02:00
Joel Einbinder c89d5a50dd
chore: migrate to monorepo (#9421) 2021-10-11 16:52:17 +02:00