This enables `allowDeclareFields` flag that allows code like:
```ts
class Foo {
declare prop: string;
}
```
Declaring is a workaround for inheritance vs property initialization
issues.
https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-7.html#the-usedefineforclassfields-flag-and-the-declare-property-modifier
For `allowDeclareFields` to work, we have to run
plugin-transform-typescript before plugin-proposal-class-properties and
few others, so this change migrates from preset-typescript that always
runs last to individual plugin-transform-typescript, and reorders some
plugins.
References #21119.
There is currently a bug when running `node@18.14.2` when running with
experimentalLoader
```
TypeError: The URL must be of scheme file
at new NodeError (node:internal/errors:399:5)
at Object.fileURLToPath (node:internal/url:1492:11)
at resolve (./node_modules/@playwright/test/lib/experimentalLoader.js:39:48)
at nextResolve (node:internal/modules/esm/loader:163:22)
at ESMLoader.resolve (node:internal/modules/esm/loader:838:24)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:7)
at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:21)
```
This came from my test `vrt.spec.ts` which had a non package import
inside of it
```ts
import fs from "node:fs/promises"
```
The test run failed due to node imports not returning fileUrls when
resolved.
---------
Co-authored-by: Nowell Strite <nstrite@nvidia.com>
This reverts commit 303c5998f8.
Reason for revert: I tried enabling `ssim-cie94` by default on
ionic-framework test suite, and it proves to be overly strict for their
usecase.
The method accepts a `ratio` option to assert the ratio
of the element in viewport. `ratio` defaults to `Number.MIN_VALUE`.
NOTE: this reverts commit d950f5b6ee and
adds `ratio` option support + does the rename.
Fixes#8740
This implementation is based on the [original
PR](https://github.com/microsoft/playwright/pull/19691) by @kevin940726.
It makes the reporter produce single file when there is no sharding and
multiple out-of-line report-x-of-y.zip reports which are automatically
merged together when put in one folder.
References https://github.com/microsoft/playwright/issues/10437
Co-authored-by: Kai Hao <kevin830726@gmail.com>
The option defines a comparator to be used to compare images.
Possible values are `"pixelmatch"` and `"ssim-cie94"`.
Note: This reverts commit 8167f8bf54.
This removes everything related to docker integration experiments that
we conducted over the last 6 months.
I'll send a follow-up with an alternative suggestion that was demo'ed on
a team meeting in the end of December.
Introduce config.globalScripts. Tests from the matching files will run
before all projects. We'll only allow beforeAll/afterAll instead of
tests in such files (next PR).
Global scripts are executed as part of 'Global Scripts' project which is
not present in FullConfig.projects but may be referenced by
corresponding global setup Suites.
Signed-off-by: Yury Semikhatsky <yurys@chromium.org>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
* Changed `setup` to `test.projectSetup`
* Only `test.projectSetup.only` is supported on the new method
* test.* methods except for before/after/Each/All hooks can be called
inside the project setup files
This is a new web-first assertion that should be used like this:
```ts
test('should work', async ({ page }) => {
const locator = page.locator('body');
// New web-first assertion.
await expect(locator).toIntersectViewport();
// The same functionality.
await expect.poll(() => locator.viewportRatio()).toBeGreaterThan(0);
});
```
Fixes#8740
Note: this keeps existing behavior of `undefined` in `test.use()`
reverting to the config value and not to the original default value.
References #19615.
This patch adds a reverse proxy in front of novnc and playwright
server inside the container.
As a result:
- container exposes a single HTTP port to the host
- all Playwright clients can connect using this exposed port, e.g.
`http://127.0.0.1:5400`
- navigating to the exposed port in the browser lands on a nice HTML
page
with a link to view container screen
- users can also manually navigate to `http://127.0.0.1:5400/screen` to
view screen
This patch implements a new image comparison function, codenamed
"ssim-cie94". The goal of the new comparison function is to cancel out
browser non-determenistic rendering.
To use the new comparison function:
```ts
await expect(page).toHaveScreenshot({
comparator: 'ssim-cie94',
});
```
As of Nov 30, 2022, we identified the following sources of
non-determenistic rendering for Chromium:
- Anti-aliasing for certain shapes might be different due to the
way skia rasterizes certain shapes.
- Color blending might be different on `x86` and `aarch64`
architectures.
The new function employs a few heuristics to fight these
differences.
Consider two non-equal image pixels `(r1, g1, b1)` and `(r2, g2, b2)`:
1. If the [CIE94] metric is less then 1.0, then we consider these pixels
**EQUAL**. (The value `1.0` is the [just-noticeable difference] for
[CIE94].). Otherwise, proceed to next step.
1. If all the 8 neighbors of the first pixel match its color, or
if the 8 neighbors of the second pixel match its color, then these
pixels are **DIFFERENT**. (In case of anti-aliasing, some of the
direct neighbors have to be blended up or down.) Otherwise, proceed
to next step.
1. If SSIM in some locality around the different pixels is more than
0.99, then consider this pixels to be **EQUAL**. Otherwise, mark them
as **DIFFERENT**. (Local SSIM for anti-aliased pixels turns out to be
very close to 1.0).
[CIE94]: https://en.wikipedia.org/wiki/Color_difference#CIE94
[just-noticeable difference]:
https://en.wikipedia.org/wiki/Just-noticeable_difference
Fixes#19169
Note, I'm not sure why it was done this way before — e.g. Why `testDir`
were considered for resolution, but `testFileDir` wasn't.
Looking for guidance on the approach here, because there are still some
template tokens outside `path.resolve`
Co-authored-by: Shubham Kanodia <skanodia@atlassian.com>
The behavior regarding filters (both in config, command line and .only)
is the following:
- if some of tests match and none of setup match then we'll run all
setup files and all matching tests
- otherwise the filters apply to setup files the same way as to regular
tests
Running `npx playwright test file:123` will have the following behavior
- if only test files match then only matching subset of tests will run
but all setup files will run as well
- if only setup files match the filter then only those setup tests will
run
- if both setup and test files match an error will be thrown
This patch stops using `processLauncher` to launch web servers. Process
Launcher
will spawn a new process group which is separate from test runner.
This might result in unexpected behavior, e.g. `kill -sigkill
-<testrunnerpid>` will orphan web server process.
Instead, this patch simply spawns web server and sends `SIGTERM` to it
when ready.
This patch:
- updates documentation to lead users from `TestConfig.snapshotDir` and
`testInfo.snapshotSuffix` to `TestConfig.snapshotPathTemplate` as a
better and more flexible alternative.
- drops `{snapshotSuffix}` from documentation
- stops using `snapshotSuffix = ''` in our own tests and switches us
to the `snapshotPathTemplate`.
- adds `{testName}` token.
This configuration option allows to set a string with template
values for precise control over snapshot path location.
An example of `snapshotPathTemplate` usage:
```ts
// playwright.config.ts
// Notice the `testDir` configuration!
export default {
testDir: './tests',
snapshotPathTemplate: './__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
}
```
Currently supported "magic tokens" inside the `snapshotPathTemplate`
are:
- `{testDir}` - project's `testDir`
- `{snapshotDir}` - project's `snapshotDir`
- `{platform}` - `process.platform`
- `{projectName}` - Project's sanitized name
- `{testFileDir}` - Directories in relative path from `testDir` to test
file path (e.g. `page/` in the example below)
- `{testFileName}` - Test file name (with extension) (e.g.
`page-click.spec.ts` in the example below)
- `{testFilePath}` - Relative path from `testDir` to test file path
(e.g. `page/page-click.spec.ts` in the example below)
- `{ext}` - snapshot extension (with dots)
- `{arg}` - joined snapshot name parts, without extension (e.g.
`foo/bar/baz` in the example below)
- `{snapshotSuffix}` - `testInfo.snapshotSuffix` value.
Consider the following file structure:
```
playwright.config.ts
tests/
└── page/
└── page-click.spec.ts
```
The following `page-click.spec.ts`:
```ts
// page-click.spec.ts
import { test, expect } from '@playwright/test';
test('should work', async ({ page }) => {
await expect(page).toHaveScreenshot(['foo', 'bar', 'baz.png']);
});
```
Fixes#7792
Per discussion in #12950, adding sanitized name to the output filename
prefix. This can make debugging easier, and the filename structure more
human friendly.
This patch implements a new mode of network tethering for Playwright
server & its clients.
With this patch:
- playwright server could be launched with the
`--browser-proxy-mode=tether` flag to engage in the new mode
- a new type of client, "Network Tethering Client" can connect to the
server to provide network traffic to the browsers
- all clients that connect to the server with the `x-playwright-proxy:
*` header will get traffic from the "Network Tethering Client"
This patch also adds an environment variable
`PW_OWNED_BY_TETHER_CLIENT`. With this env, playwright server will
auto-close when the network tethering client disconnects. It will also
auto-close if the network client does not connect to the server in the
first 10 seconds of the server existence. This way we can ensure that
`npx playwright docker start` blocks terminal & controls the lifetime of
the started container.
We have to reliably know whether test timed out or not, and soft expect
error could have marked it with `status=failed` but it would still time
out. Now we have a separate `_didTimeout` flag for this.
Fixes#18023.
- Rename internal selectors `has`, `control` and `attr` to
`internal:has`, `internal:control` and `internal:attr`.
- Fix `getByLabel()` to respect strictness, by introducing
`internal:label` selector.
- Move tests essential for ports to `selectors-by.spec`.
This makes `HttpServer` accept `preferredPort` option that will first
try to listen on that port, and if that port is already in use, listen
on some available port instead.
Fixes#17201.
Fixes#17412.
Supercedes #17413.
- if configured via playwright.config.ts, relative paths should be
relative to the config.
- if configured via env var, should be relative to `cwd`
Currently, we ues `#root` vs `#root > *` selector for component roots
depending on the number of root children. This heuristic detects
fragments that render multiple elements inside the root.
However, this does not work with empty fragments that do not render
anything.
The fix is to make the `#root >> control=component` selector that would
dynamically detect the root. This supports empty fragments and also
allows for dynamic updates of the fragments.
Allows to set workers as a percentage of logical CPUs, for example
"50%".
Examples :
```bash
npx playwright test --workers 3
npx playwright test --workers 50%
```
```js
const config: PlaywrightTestConfig = {
// ...
workers: '33%',
}
```
This patch:
- Removes all `process.exit(1)` from `docker.ts` and instead throws
errors.
- Drops the `npx playwright docker test` command. We agreed to
engage docker when `PLAYWRIGHT_DOCKER` environment variable
is set.
- Introduces hidden `npx playwright docker status` command that
dumps a JSON with docker status:
```sh
aslushnikov:~/prog/playwright$ npx playwright docker status
{
"dockerEngineRunning": true,
"imageName": "playwright:local-1.27.0-next-focal",
"imageIsPulled": true,
"containerWSEndpoing":
"ws://127.0.0.1:55077/eafeb84c-571b-4d12-ac51-f6a2b43e9155",
"containerVNCEndpoint":
"http://127.0.0.1:55076/?path=fb6d4add-9adf-4c3c-b335-893bdc235cd7&resize=scale&autoconnect=1"
}
```
This patch moves parts of docker configuration to a plugin so that
it can rely on repoter for stdout.
Drive-by: provide all plugins with reporter in the `setup` callback.
This patch opts into the `__screnshots__` folder snapshot management
for docker.
With this patch, docker-originating snapshots will be stored in the
following folder:
```
{testDir}/__screenshots__/{projectName}/{testFilePath}/{snapshotName}
```
Where `{testFilePath}` is a test file path relative to `testDir`
Drive-by: introduce and document the `PLAYWRIGHT_DOCKER` environment
variable that enables docker integration.
Drive-by: make sure docker container does not expose ports on `0.0.0.0`
and instead registers to localhost. This way websocket and vnc ports
are not exposed to the public internet.
This patch introduces the following commands:
- `npx playwright docker build` that builds a VRT docker image
locally that is based off the `mcr.microsoft.com/playwright:jammy`
- `npx playwright docker start` that launches a docker container with
browsers.
- `npx playwright docker stop` that stops given docker container.
- `npx playwright docker test` that runs all the tests inside a
launched docker container.
Fixes#16456.
The extra span/name was coming from
077b8a9289/packages/html-reporter/src/links.tsx (L72)
(in conjunction with the line above it).
These two lines assume `path` and `body` are mutally exclusive.
Regression likely introduced by the PR that added the now removed line #10778.
This patch changes `expect(locator).toHaveAttribute()` so that the
`value` argument can be omitted. When done so, the method will
assert attribute existance.
Fixes#16517
Also, exlude certain errors from triggering this message:
- `no tests found`
- `duplicate test titles are not allowed`
- `--forbid-only found a focused test`
- `Timed out waiting 3600s for the entire test run`
In the following example, `locale` inside the `describe`
would be reverted to the default value:
```js
test.use({ locale: 'en-GB' });
test.decsribe(() => {
test.use({ locale: undefined });
});
```
A small change in the API requires `shortCircuit: true` when returning the
transformed source.
Another API change removes third argument from the chained resolve/load,
but passing three instead of two still works.
Line reporter now shows stats in addition to the test name:
```
[chromium] › page/page-click-react.spec.ts:108:1 › should not retarget when element changes on hover
[21/93] Passed: 17 Flaky: 0 Failed: 0 Skipped: 4 (7s)
```
- Two lines per test: title and location.
- Align project labels.
- Add trace badge that opens trace viewer.
- Add video and image diff badges that show scrolled test result view.
fix(test runner): more firendly test duration
Reported test duration now does not include time spent in
`beforeAll`, `afterAll` and fixtures that have a separate timeout.
This is to avoid different reported test execution time,
depending on the test execution order.
Fixes#8206.
Since #8206 is a long-awaited (~ 1 year old), popular (~ 45 reactions, frequently requested in community channels, etc.), this PR aims to unblock folks.
Notably, we do not innovate on the `webServer` API, despite knowing we're not in love with it. We'll save the innovation for either Plugins or a new `LaunchConfigs` option. (We haven't yet arrived at a Plugin API we like, and instead of launching a new option guessing what the "better" launchConfig API would be, let's wait and see how folks use this new Array-variant of `webServer` which—despite its name—can be used for non-Web Server launches!
When running without tty, line reporter outputs a line for each
percent of the tests, thus limiting the output to ~100 lines.
In addition, reporters now support PLAYWRIGHT_LIVE_TERMINAL
env variable to force tty mode.
Useful to set some options without coming up with an artificial name.
```js
test.describe(() => {
test.use({ colorScheme: 'dark' });
test('my test', () => {});
})
```
Currently, it is possible to run a function, e.g. a second `beforeEach` hook,
that will receive `null` for the fixture that has already failed before.
This PR skips running any user function that uses a fixture that has already
failed, just like if the fixture would be initialized again and failing for
the second time.
Currently, if `text.fixme()` or `test.skip()` is used within a test, we
add a `fixme` or `skip` annotation. However, if the wrapper style is
used:
```
test.fixme('should work', () => {…})
```
the annotations were missing. This change adds annotations for the
above.
These annotations are important for reporting purposes and knowing
exactly what flavor of "skipped" was used.
Fixes#15239.
Some of the current timeout error messages are confusing, because they do not suggest that the issue is most likely a slow test. This PR updates timeout messages as follows:
- Test timeout of 30000ms exceeded.
- Test timeout of 30000ms exceeded while setting up "browser".
- Test timeout of 30000ms exceeded while tearing down "context".
- Test timeout of 30000ms exceeded while setting up "playwright configuration".
- Test timeout of 30000ms exceeded while running "beforeEach" hook.
- Test timeout of 30000ms exceeded while running "afterEach" hook.
- "beforeAll" hook timeout of 30000ms exceeded.
- "afterAll" hook timeout of 30000ms exceeded.
- Worker teardown timeout of 30000ms exceeded.
- "skip" modifier timeout of 30000ms exceeded.
- Fixture "myCustomFixture" timeout of 5000ms exceeded.
