Commit graph

87 commits

Author SHA1 Message Date
Pavel Feldman 81bd637d94
chore: merge traces into a single zip file (#21242) 2023-02-27 22:31:47 -08:00
Pavel Feldman ed41fd0643
chore: use listview to render stack trace (#21197) 2023-02-24 15:31:10 -08:00
Max Schmitt f3a46f7405
feat(html): render annotations as links if needed (#21165)
Fixes https://github.com/microsoft/playwright/issues/20584
2023-02-23 21:57:02 +01:00
Dmitry Gozman 24c8d45c70
chore: enable tsc for ttest sources (#21018) 2023-02-19 11:18:07 -08:00
Dmitry Gozman 595502ac46
chore: remove unnecessary args in ttest (#21012) 2023-02-18 13:08:17 -08:00
Dmitry Gozman 56dbd1fcb0
chore: remove usesCustomOutputDir from tests (#21011) 2023-02-18 11:41:41 -08:00
Yury Semikhatsky 90c4e6f9b2
chore: revert sharded html report (#20923)
We've decided not to ship it in the current form.

#10437
2023-02-15 12:38:03 -08:00
Dmitry Gozman 91da67fab1
test: remove magic headers in ttest (#20867)
Instead, explicitly import from '@playwright/test'.
2023-02-14 19:20:56 -08:00
Dmitry Gozman de69b766d9
test: code health in ttest (#20837)
- Remove duplicate tests.
- Remove unused test helpers.
- Print full watch output on failures.
- Unflake some tests.
2023-02-10 20:26:19 -08:00
Pavel Feldman e1f287f255
chore: more watch tests (#20797) 2023-02-09 16:03:54 -08:00
Yury Semikhatsky a93cf767a1
feat: html reporter sharded option (#20737)
Make sharded report feature an opt-in:

```ts
{
   reporter: [['html', { sharded: true }]]
};
```

#10437
2023-02-07 22:21:50 -08:00
Pavel Feldman 1b941bcf2e
chore: simplify ttests (#20733) 2023-02-07 15:11:44 -08:00
Yury Semikhatsky f10b29fd5e
feat: show warning when some report shards are missing (#20731)
When some of the report shards are missing still show the report but
display an error in the status line:

![image](https://user-images.githubusercontent.com/9798949/217371112-2317a307-204e-4c11-8d01-42e91fe6ca7a.png)

#10437
2023-02-07 14:22:10 -08:00
Yury Semikhatsky 8f53bf7b41
feat: sharded html report (#20700)
This implementation is based on the [original
PR](https://github.com/microsoft/playwright/pull/19691) by @kevin940726.
It makes the reporter produce single file when there is no sharding and
multiple out-of-line report-x-of-y.zip reports which are automatically
merged together when put in one folder.

References https://github.com/microsoft/playwright/issues/10437

Co-authored-by: Kai Hao <kevin830726@gmail.com>
2023-02-07 10:21:26 -08:00
Sam Chen 340cacf4fd
feat: show diff tab by default (#19820)
Close https://github.com/microsoft/playwright/issues/19803

I had to manually inject `window.playwrightReportBase64` data into
`playwright/packages/html-reporter/index.html` when developing,
wondering if there's any method I don't know.
2023-02-03 10:32:23 -08:00
Pavel Feldman 2c27bd3b07
chore: remove plugin registration sink (#20460) 2023-01-27 14:36:41 -08:00
Dmitry Gozman b700c08dc5
feat(screenshots): when actual and expected have different sizes, pad and produce the diff image (#20208)
Also show sizes in the html report to easier spot the size mismatch
issue.

<img width="1030" alt="diff"
src="https://user-images.githubusercontent.com/9881434/213327632-b8fcd69c-8d08-460c-9de1-b5f4f8c56359.png">

Fixes #15802.
2023-01-20 19:41:43 -08:00
Pavel Feldman 7ff27600b4
chore: experimental oop loader (#20269) 2023-01-20 18:24:15 -08:00
Pavel Feldman d1c161ce99
chore: merge the util exports (#20110) 2023-01-13 13:50:38 -08:00
Max Schmitt e2a2196e6a
chore(html-reporter): show project name always for the test file (#19905)
Fixes https://github.com/microsoft/playwright/issues/19874
2023-01-10 17:11:38 +01:00
Dmitry Gozman 28577afde4
feat(soft expect): mark steps with failed soft expects as failed (#19973)
Fixes #19673.
2023-01-09 16:17:06 -08:00
Ross Wollman 06c7f1fb6c
fix(html): test and fix reporter timing display (#19670)
#19576 introduced a regression where the CLI reporters displayed some
times with way too many decimals. e.g. 7.123456789ms.

Prior to #19576, there were two monotonicTime implementations; that PR
updated the reporters to use the common definition that had existed in
utils.ts. However, that introduced a regression in the base.ts reporters
which used the ms duration humanizing package which did not account for
the more precise decimals used by the shared monotonicTime function.

This fix removes the dependency on the third-party ms package and now
consistently uses Pavel's humanize function which the HTML reporter had
been using and proved to have better defaults for decimals.
Additionally, we add more test coverage to limit future regressions
since this was caught in passing.

Closes #19556.
Relates #19576.
2022-12-22 15:57:55 -08:00
Ross Wollman 0844394270
feat(html): display overall duration (#19576)
<img width="1390" alt="Screenshot 2022-12-19 at 4 15 33 PM"
src="https://user-images.githubusercontent.com/11915034/208552484-c0127615-d2c6-414f-ae3b-e7836553d890.png">

* Adds duration (time ellapsed from `onBegin` to `onEnd`); roughly
equivalent to `time npx playwright test …`.
* Removes cumulative per-file time

Resolves #19566.
2022-12-20 14:13:10 -08:00
Andrey Lushnikov 6d491f928d
feat(playwright-test): introduce snapshotPathTemplate configuration (#18568)
This configuration option allows to set a string with template
values for precise control over snapshot path location.

An example of `snapshotPathTemplate` usage:

```ts
// playwright.config.ts
// Notice the `testDir` configuration!
export default {
  testDir: './tests',
  snapshotPathTemplate: './__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
}
```

Currently supported "magic tokens" inside the `snapshotPathTemplate`
are:

- `{testDir}` - project's `testDir`
- `{snapshotDir}` - project's `snapshotDir`
- `{platform}` - `process.platform`
- `{projectName}` - Project's sanitized name
- `{testFileDir}` - Directories in relative path from `testDir` to test
  file path (e.g. `page/` in the example below)
- `{testFileName}` - Test file name (with extension) (e.g.
  `page-click.spec.ts` in the example below)
- `{testFilePath}` - Relative path from `testDir` to test file path
  (e.g. `page/page-click.spec.ts` in the example below)
- `{ext}` - snapshot extension (with dots)
- `{arg}` - joined snapshot name parts, without extension (e.g.
`foo/bar/baz` in the example below)
- `{snapshotSuffix}` - `testInfo.snapshotSuffix` value.

Consider the following file structure:

```
playwright.config.ts
tests/
└── page/
    └── page-click.spec.ts
```

The following `page-click.spec.ts`:

```ts
// page-click.spec.ts
import { test, expect } from '@playwright/test';

test('should work', async ({ page }) => {
  await expect(page).toHaveScreenshot(['foo', 'bar', 'baz.png']);
});
```

Fixes #7792
2022-11-09 15:29:07 -08:00
Pavel Feldman 2183d9e9a2
chore: use codemirror for editor (#18482) 2022-11-01 15:04:30 -07:00
Ross Wollman e73676d094
fix(reporters): output relative to config (#17518)
Fixes #17412.
Supercedes #17413.

- if configured via playwright.config.ts, relative paths should be
relative to the config.
- if configured via env var, should be relative to `cwd`
2022-09-26 11:01:43 -07:00
Andrey Lushnikov 4f11a4d5e7
feat(docker): use the __screenshots__ dir for snapshots by default (#17311)
This patch opts into the `__screnshots__` folder snapshot management
for docker.

With this patch, docker-originating snapshots will be stored in the
following folder:

```
{testDir}/__screenshots__/{projectName}/{testFilePath}/{snapshotName}
```

Where `{testFilePath}` is a test file path relative to `testDir`

Drive-by: introduce and document the `PLAYWRIGHT_DOCKER` environment
variable that enables docker integration.
2022-09-13 15:49:04 -07:00
Max Schmitt 7e2aec7454
chore: align more ESLint rules with VSCode formatting (#16647) 2022-08-18 20:12:33 +02:00
Ross Wollman a3d99f1b4a
chore: trace viewer fallback error (#16365)
Fixes #16349.
2022-08-09 15:53:11 -07:00
Pavel Feldman e8070ee1b2
chore: warn users on clashing test output and html reporter folders (#14964) 2022-06-18 15:47:26 -07:00
Ross Wollman b9114f9cbc
fix(html-reporter): file-browser friendly extensions (#14943)
Fixes #14904.

This is done to make looking at the raw contents of the report
friendlier when using a file browser. However, it should be noted, the
public API of the HTML Reporter makes no guarantees of its contents
structure/layout/naming-conventions.
2022-06-17 08:09:49 -07:00
Dmitry Gozman dbcf039717
chore: remove experimental types (#14560) 2022-06-01 15:22:43 -07:00
Andrey Lushnikov 5aa82dc5e4
chore: un-experiment expect(pageOrLocator).toHaveScreenshot() (#14033)
This patch un-experiments `expect(pageOrLocator).toHaveScreenshot()`
method.

Fixes #13526
2022-05-09 08:34:03 -07:00
Andrey Lushnikov 98945a81a8
feat: move toHaveScreenshot to use old snapshot paths by default (#14006)
Note: all toHaveScreenshot tests still use `__screenshots__` directory
for their expectations. One more test was added to make sure that
by default, `toHaveScreenshot` uses old snapshots.
2022-05-09 07:34:53 -07:00
Pavel Feldman 29fd1d86df
chore: hide plugins for now (#13908) 2022-05-03 14:25:56 -07:00
Ross Wollman 3b3cad7d69
feat: rewrite gitCommitInfo plugin, drop GlobalInfo & attachments (#13837) 2022-05-02 16:28:14 -07:00
Dmitry Gozman c86c2e8762
feat: restore toHaveScreenshot as experimental feature (#13549)
- Restore docs.
- Make `TestConfig.expect` generated.
- Allow experimental properties with "e" marker: `- foo e<float>`.
2022-04-14 13:22:42 -07:00
Ross Wollman 1af32e400f
feat(test-runner): introduce GlobalInfo (#13083) 2022-04-08 13:22:14 -07:00
Pavel Feldman e79b90f454
chore: use utils via index export (5) (#13413) 2022-04-07 20:18:22 -07:00
Pavel Feldman 40d5e3a3c9
chore: use utils via index export (3) (#13403) 2022-04-07 13:55:44 -07:00
Pavel Feldman 5ae2017a5b
chore: always import type (#13365) 2022-04-06 14:57:14 -07:00
Dmitry Gozman e31a5b690a
chore: unrelease toHaveScreenshot and screenshotsDir (#13304) 2022-04-05 08:34:51 -07:00
Pavel Feldman 66cf82766e
test(html-reporter): add image diff tests (#13262) 2022-04-01 18:11:15 -07:00
Pavel Feldman 55ee41c848
feat(html): render image diff slider (#13257) 2022-04-01 15:27:51 -07:00
Ross Wollman 16efbdef98
chore: remove/rename FullConfig._attachments (#13233)
This was originally introduced in #12734.

It will be replaced with GlobalInfo (#13083), but not before the 1.21
release.
2022-04-01 12:36:05 -07:00
Pavel Feldman 6a463195c4
chore: allow multiple image diffs (#13202) 2022-03-31 14:11:34 -07:00
Pavel Feldman 42f260c688
chore: prepare image diff for refactornig (#13197) 2022-03-30 17:42:08 -07:00
Pavel Feldman 81e7c0a77c
chore(html-report): render metainfo as a chip (#13166) 2022-03-29 18:13:08 -07:00
Pavel Feldman 5e17ed137b
fix(html-report): only invoke git once (#13165) 2022-03-29 15:49:58 -07:00
Andrey Lushnikov aa1daeba85
fix(html): put HTML report next to package.json by default (#13141)
Fixes #12970
2022-03-29 14:19:31 -07:00