Otherwise, we might split the `beforeAll`-grouped test group into
`workers` parts instead of `shard.total` parts as the user would expect.
Fixes#33077.
After the context has been disposed we can't route any callbacks to it
because
it is not in the map, so the assertion doesn't make sense as it always
ends up in
the top level session.
Fixes the following error:
```
pw:browser <closing ws> Closing websocket due to failed onmessage callback. eventData={"type":"success","id":32,"result":{}} e=Assertion error Error: Assertion error
pw:browser at assert (/home/yurys/playwright/packages/playwright-core/src/utils/debug.ts:21:11)
pw:browser at BidiSession.dispatchMessage (/home/yurys/playwright/packages/playwright-core/src/server/bidi/bidiConnection.ts:229:13)
pw:browser at BidiConnection.call [as _dispatchMessage] (/home/yurys/playwright/packages/playwright-core/src/server/bidi/bidiConnection.ts:93:25)
```
Closes https://github.com/microsoft/playwright/issues/32960
If the socket is reused, the connect and DNS timings are set to -1,
because that timing doesn't apply to the current request. The time
between request start and the socket being free is counted as `blocked`.
The diff is now shown inline in the errors list.
There are 2 possible failures of toHaveScreenshot
* Previous and actual snapshot mismatch. In this case html report will
show diff between Actual/Previous and have Expected as a separate
screenshot.
* Actual/Previous are equal but they differ from the expected. In this
case html report only contains Actual/Expected images and the diff.
Reference: https://github.com/microsoft/playwright/issues/32341
<img width="1039" alt="image"
src="https://github.com/user-attachments/assets/b458f986-cc25-4721-862c-0cc2c1b01a42">
This reapplies what we reverted in
https://github.com/microsoft/playwright/pull/32989.
Max and me debugged this, and found that the test failures come from
SOCKS proxy now preferring IPv6 over IPv4. We've updated the tests and
made sure that this doesn't mask any breaking change.
I'm enabling CQ1 to make sure we don't oversee any other CI failures.
Closes https://github.com/microsoft/playwright/issues/32951
`node:http` reuses TCP Sockets under the hood. We weren't cleaning up
our listeners, leading to the `MaxListenersExceededWarning`.
This PR adds cleanup logic. It also raises the warning threshhold, so
that it doesn't trigger until there's 100 concurrent requests over the
same socket.
As discussed yesterday over
https://github.com/microsoft/playwright/issues/32807. Adds some words to
differentiate `setSystemTime` from `setFixedTime`.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Fixes a bug discovered in
https://github.com/microsoft/playwright/pull/32647. When using http
proxy, the `connect` event isn't emitted so we don't populate
`tcpConnectionAt`. The updated version of `https-proxy-agent` emits a
`proxyConnect` as a replacement, so this PR updates and listens to that
event.
For socks proxies, the `on("socket")` event is emitted once the SOCKS
connection is established, which is the equivalent of having a TCP
connection available.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Max Schmitt <max@schmitt.mx>
Closes https://github.com/microsoft/playwright/issues/32862.
`prefers-color-scheme: no-preference` was removed from browsers. This PR
marks it as deprecated in our docs and removes all mentions.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
As discussed yesterday, this PR replaces the "reporter showcase" with a
list of interesting implementations for folks who are writing their own
custom reporters.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
This includes all actions that perform locator handler check.
Note this makes it impossible to interact with the page while a main
frame navigation is ongoing. This was already the case for Chromium, but
now WebKit and Firefox align with it.
Setting `PLAYWRIGHT_SKIP_NAVIGATION_CHECK` environment variable disables
this behavior.
WebKit notifies about a pending same-document navigation through
`Page.frameScheduledNavigation`, and committing it should clear the
`pendingDocument()`.
Extracted from #32899.
When the list of highlighted elements changes over time, we should
update the elements marked as `__playwright_target__` in the snapshot.
A good example is an `expect(locator).toHaveText([...])` where the list
of elements changes from 4 items to 3 after clicking a "Delete" button.
In https://github.com/microsoft/playwright/issues/32861, an interested
.NET user wanted to try Playwright and found the VS Code Getting Started
guide. It didn't work for them because the VS Code Extension is for
usage with Node.js, and they don't have NPM installed. We can reduce
confusion by mentioning that VS Code Getting started is for Node.js.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Closes https://github.com/microsoft/playwright/issues/32853
Vite turns the shorthand fragment notation `<></>` into `import {
Fragment } from "react"; <Fragment></Fragment>`. On the Node.js side of
things, this `react` import resolves to our mock version of React, which
currently mocks `Fragment` as `{}`. Currently, we pass that straight to
`React.createElement`, which throws an error.
The fix is to make our `Fragment` mock detectable with a tag, and when
we render it replace it with the real `__pwReact.Fragment`.
This is fixing a case where the test failed with strict mode violation,
but all the matched elements are not highlighted in the trace.
For example, all the buttons will be highlighted when the following line
fails due to strict mode violation:
```ts
await page.locator('button').click();
```
To achieve this, we mark elements during `querySelector` phase instead
of inside `onBeforeInputAction`. This allows us to only mark from inside
the `InjectedScript` and remove the other way of marking from inside the
`Snapshotter`.
In https://github.com/microsoft/playwright/issues/32872, a user notes
that we document `.focus()` to not timeout by default, but in practice
when used without the test runner, it defaults to a 30s timeout.
I've discussed this with Dima, and he noted that our JS documentation
focuses on usage with the Playwright test runner, not with the library.
The test runner disables timeouts for operations in favour of timeouts
for test cases. In the library, we default to a 30s timeouts. This PR
adds this to the "key differences" table.
- Renamed to `page.requestGC`.
- Added a useful snippet to the docs.
References #32278.
---------
Signed-off-by: Dmitry Gozman <dgozman@gmail.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
Various settings that make Firefox behave when testing over Bidi. The
settings are copied from
ea36b7b1f0/packages/browsers/src/browser-data/firefox.ts (L190-L402).
Unlike Playwright bundled version, we write the settings into `user.js`
in the profile folder before launching the browser.
Fixes the following:
```
npm audit report
vite 5.0.0 - 5.2.13
Severity: moderate
Vite DOM Clobbering gadget found in vite bundled scripts that leads to XSS - https://github.com/advisories/GHSA-64vr-g452-qvp3
```
Closes https://github.com/microsoft/playwright/issues/32653.
Adds some test coverage to ensure we respect minimal mode for API
Requests in HAR tracing.
| omit setting | result |
| - | - |
| `omitCookies` | added test for it |
| `omitTiming` | already covered |
| `omitSecurityDetails` | not recorded yet |
| `omitServerIP` | we don't record it yet, so no action here. gonna open
a separate issue |
| `omitPages` | not relevant to API requests |
| `omitSizes` | added test for it |
| `omitScripts` | not relevant to API requests |
Fixes https://github.com/microsoft/playwright/issues/30160
### Description:
This pull request introduces the ability to specify custom locations for
test steps in Playwright. By enabling the provision of arbitrary
locations to the test.step method, it resolves the limitation where
helper methods obfuscate the original call site, providing more accurate
and meaningful location data in test reports.
### Motivation:
To enhance the utility and clarity of test reports in Playwright.
Specifically, it addresses the need to trace test steps back to their
precise location in the code, which is especially important when steps
are abstracted in helper functions. This feature is crucial for
maintaining accurate documentation and facilitating debugging processes.
### Changes:
Added functionality to pass a custom location object to test.step.
### Expected Outcome:
This PR is expected to significantly improve the precision and
usefulness of diagnostic data in test reports by allowing specific
locations within helper functions to be accurately documented. It
facilitates better tracking of test executions and simplifies the
debugging process, making it easier for developers to understand and
address issues within complex tests.
### References:
Closes https://github.com/microsoft/playwright/issues/30160 -
"[Feature]: allow to pass arbitrary location to test.step"
**Code Check**
I conducted tests on this new feature by integrating it into some
existing test codes, and it worked well. I will attach the code used for
testing and a screenshot showing the successful outcome.
<details>
<summary>toggle dropdown</summary>
<div markdown="1">
```
import type { Location } from '../../../packages/playwright/types/testReporter'
...
test('should respect the back button', async ({ page }) => {
await page.locator('.todo-list li .toggle').nth(1).check();
await checkNumberOfCompletedTodosInLocalStorage(page, 1);
...
await test.step('Showing active items', async () => {
await page.getByRole('link', { name: 'Active' }).click();
}, {location});
```
<img width="1109" alt="image"
src="https://github.com/user-attachments/assets/359feafa-0949-4c71-9426-46debef21bdd">
</div>
</details>
Related to https://github.com/microsoft/playwright/issues/19621
Adds some instrumentation to collect timings for `APIRequestContext`
requests and adds them to the HAR trace. Doesn't yet expose them via an
API, but makes our `Duration` field in the trace viewer show a nice
duration:
<img width="1392" alt="Screenshot 2024-09-14 at 11 46 04"
src="https://github.com/user-attachments/assets/8020382d-9494-4634-9cfd-22b6f4a1d770">
I'm gonna add it to our API in a separate PR.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Closes https://github.com/microsoft/playwright/issues/32578.
Adds a buffer mode that can be toggled by pressing <kbd>b</kbd>. When
engaged, changed test files are collected and shown on screen. The test
run is then kicked off by pressing <kbd>Enter</kbd>.
This changes the signal to start a test run from <kbd>Cmd+s</kbd> to a
<kbd>Enter</kbd> press in the test terminal. It should help users with
auto-save and make it easier to run on long-running tests. It feels very
similar to running `npx playwright test`, but without having to write a
filter.
https://github.com/user-attachments/assets/71e16139-9427-4e90-b523-8d218f09ed9d
Closes https://github.com/microsoft/playwright/issues/32632. A side
effect of Remix's hydration implementation is that it throws away the
entire DOM. This is broadly discussed in
https://github.com/remix-run/remix/issues/4822 - there might be a fix in
coming React versions, but who knows.
Besides breaking browser extensions, this also deletes our toolbar!
This PR fixes it by periodically checking in on `x-pw-glass`, and
remounting it if it was unmounted. Hacky but effective!
Adds a copy-to-clipboard button for each annotation so that text can be
copied easily.
This re-uses the existing `CopyToClipboard` component and adds a `small`
variant that can be used inline. The icon size and colour have been
chosen to avoid being overwhelming when used inline.
Related to #30141
I opted not to introduce the hover behaviour from #30749 as it's less
discoverable, but can understand why that might be favourable. Certainly
open to suggestions 😄
<img width="379" alt="Screenshot 2024-07-22 at 3 23 53 PM"
src="https://github.com/user-attachments/assets/3b9998cf-2e8d-40c9-9c8a-64eab3a9ed2e">
Changes `expect.extend` behaviour so that it doesn't mutate the global
instance and behaves closer to what users expect. This is formally a
breaking change, and I had to remove a test that asserts the breaking
behaviour.
TODO:
- [x] decide wether this is a separate method or a flag for
`expect.extend`
- [x] figure out if we need to change docs
Based on the expectations the tests that are expected to timeout or fail
will be skipped to save resources. The expectations can be manually
updated when corresponding feature is fixed.
What was happening?
- When we use CT, we go over the test files, look at the imports using
`tsxTransform.ts` and store them inside a map, these we feed into the
import registry which we build using Vite and have access inside the
browser
- In case of an inline component in the same file as where the test file
is, this is not happening.
- jsx-runtime via babel kicks in, transforms every JSX component in
something like that:
```
{
__pw_type: 'jsx',
type: [Function: MyInlineComponent],
props: { value: 'Max' },
key: undefined
}
```
this then gets passed into `wrapObject` which maps any function from the
Node.js side into expose function calls so they work inside the browser.
The assumption for `wrapObject` was to do it mostly for callbacks. So it
does for `type` - which is actually our component. We then pass this to
the React render function, which calls back the exposed function but we
never return anything, so it mounts `undefined`.
---
While there have been experiments from certain vendors to get the
'client only' code inside a server side file, we should throw for now to
not confuse users. We might revisit this in the future since Babel / TSX
doesn't support it outside of the box.
Fixes https://github.com/microsoft/playwright/issues/32167
We now hopefully align with `moduleResolution: bundler` tsconfig option,
allowing directory imports in every scenario, and allowing proper module
imports when not going through the type mapping.
This regressed in #32078. Fixes#32480, fixes#31811.
In this test, the trace recording goes super fast. Sometimes, this means
that the recording is finished before the screen recorder got a chance
to take a screenshot. If that happens, the tests fail because we never
show a screenshot.
This PR fixes the flakiness by delaying the trace recording so that
there's always a screenshot taken.
Those were just workarounds for browser-specific bugs, they should be
fixed upstream.
* individual mouse down/up/down/up events don't trigger dblclick event
in Firefox
* setContent throws when document.open/write is called in the utility
context in Firefox
Closes https://github.com/microsoft/playwright/issues/23964.
Trace snapshots are a best-effort snapshots of the browser DOM, but we
can't guarantee them to be exactly what the browser showed. One example
of this is `canvas` elements, where you just can't see their contents.
That makes snapshots useful, but not perfect.
For those cases where the snapshot doesn't show everything, this PR
introduces a new setting to show a screenshot instead. You won't be able
to scroll or inspect the DOM or select a locator anymore. But if the
snapshot was missing something, or displaying something wrong, you can
now check the screenshot instead.
Instead of plumbing it through a custom unspecified config field, make
it a part of plugin interface.
Additionally, use task runner for starting/stopping dev server.
This pull request introduces initial support for the WebDriver BiDi
protocol in Playwright. The primary goal of this PR is not to fully
implement BiDi but to experiment with the current state of the
specification and its implementation. We aim to identify the biggest
gaps and challenges that need to be addressed before considering BiDi as
the main protocol for Playwright.
In the `visit` method, we currently cache the rendered HTML for every
walked node. This re-use works well for traces that consist mostly of
references to earlier snapshots.
But for traces that don't share much, this is a large memory overhead
and leads to the memory crash documented in
https://github.com/microsoft/playwright/issues/32336. For the algocracks
amongst you, the current memory usage for an html tree $h$ is
$\mathcal{O}(|h| * \text{height}(h))$.
This PR removes that cache from the nodes and replaces it with a
snapshot-level cache, fixing the memory crash.
Traces *without* reference should not see a performance impact from
this.
Traces *with* references will have slower initial rendering, but
re-rendering maintains speed because of the snapshot-level cache.
Closes https://github.com/microsoft/playwright/issues/32336
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Max Schmitt <max@schmitt.mx>
After API review we decided to revert it:
* VSCode extension and UI mode users already get the (runtime) error if
the tag is not prefixed
* The typescript error message is not very nice
* The type change would break those clients that generate tests with
tags passed as string
This reverts commit 90e7b9ebac.
Closes https://github.com/microsoft/playwright/issues/32076.
This PR rewrites `watchMode.ts` to use `TestServer` under the hood. It's
essentially a complete rewrite, so don't pay too much attention on the
old implementation. Note that there's no changes to tests, so all
behaviour we have specced out there still works.
To make this work without a superfluous WebSocket connection, I had to
refactor `TestServerConnection` a little. Originally, I pulled this into
a [separate PR](https://github.com/microsoft/playwright/pull/32132), but
then realised how small the refactoring is. So it's in this PR now. Let
me know if you'd like to land it separately.
The `'should support custom matchers'` test asserts that the
functionality works, but it was a type error. This PR updates the types
so that it's allowed.
Closes https://github.com/microsoft/playwright/issues/32408
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Our CI operates on shallow clones. In vcs.ts, we perform a check for
shallow clones in `process.cwd()` instead of the test directory. This
makes the test in
3c208aeeff/tests/playwright-test/only-changed.spec.ts (L201)
failing in CI, but only for PRs. The fix is to perform the check on. the
test directory.
Closes https://github.com/microsoft/playwright/issues/32331
We're already passing the `outputDir` param to the UI, but the UI isn't
passing it back to the TestServer. This PR fixes that. I've added it to
`listTests`, which is requires to that
`TestServerDispatcher#_ignoredProjectOutputs` is populated with the
correct output dir. And i've added it to `runGlobalSetup`, which is what
the bug report was about.
Closes https://github.com/microsoft/playwright/issues/32256
We were expecting all errors to be of type `ExpectError`, but apparently
`expect` propagates rejections in the polling functions right through.
So we also need to handle that case.
I wonder if we have more cases of this. Would it make sense to enable
`useUnknownInCatchVariables` in TypeScript?
Closes https://github.com/microsoft/playwright/issues/32180
I was briefly wondering if we should output a log line a la "no tests
found", but my understanding is that that's the reporters job - so I
didn't change anything in that regard.
Closes https://github.com/microsoft/playwright/issues/32159. I
originally set out to enable Strict Mode for our React UI, but found a
way better thing: Enabling the lint rules we had already installed!
`eslint-plugin-react` is already in of our `package.json`, and this PR
enables it and fixes some of the reported issues. Most of them are
around the `key` prop which is mostly about performance, but there's
also fixes for misspelled `data-testid` props.
This PR moves around some of our CI docs. It moves the GitHub actions
docs from `ci-intro.md` to `ci.md`, reduces `ci-intro.md` to be an
introduction, adds a mention of Sharding to the best practices, and adds
a section on `--only-changed` called "Fail-Fast". Each of those changes
is a separate commit, to make this a little easier to review. If we find
any of those to commits to be contentious, i'll pull them out into
individual PRs.
While rolling this to playwright.dev, we'll also make the following
changes to its sidebar:
- move the `ci.md` document from the "Integrations" section to the
"Playwright Test" section
- make "Best Practices" the last item of the "Getting Started" section
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Yury Semikhatsky <yurys@chromium.org>
`x-unknown` is used as a placeholder for "no content-type" in the har.
We should not send it to the browser, because it is meaningfully
different from not sending `Content-Type` header. For example, Chromium
refuses to interpret stylesheets served with `x-unknown` content type.
Fixes https://github.com/microsoft/playwright-java/issues/1651.
partial fix for:
https://github.com/microsoft/playwright/issues/31927#issuecomment-2267065378
The options object wasn't treated as partial, unlike in other
frameworks, which led to the `component.update({ props: {} })` type
being selected instead the `component.update(<Component prop={} />)`
during jsx usage.
When two fixtures have different time slots, timeout in the first one
should not prevent the second one from tearing down.
Similarly, timeout in afterEach hook should not prevent fixture
teardowns.
This allows any time slot that has a legitimate timeout of zero to be
updated later on. See test for an example.
Previously, setting timeout to zero at any moment was considered a
"debug mode" and any subsequent timeouts were ignored.
Follow-up to https://github.com/microsoft/playwright/pull/32120
I made some changes suggested by @yury-s in the previous PR that make a
lot of sense:
- added an example to the documentation
- improved tests
- check params on the client and server end
- reverted to non-English characters being used as params
Closes https://github.com/microsoft/playwright/issues/32070. We were
applying `additionalFileMatcher` not just to `filteredProjectSuites`,
but also to `projectSuites`. `projectSuites` is where we take dependency
projects from, though - so `--only-changed` led to empty dependency
projects, resulting in the reported bug.
The fix is to only apply `additionalFileMatcher` on
`filteredProjectSuites`.
Supercedes https://github.com/microsoft/playwright/pull/31915, closes
https://github.com/microsoft/playwright/issues/31811.
When TypeScript resolves a specifier via path mapping, it does not
interpret `package.json`. If path mapping resolves to a directory, it
only looks at the `index.js` file in that directory if it's in CommonJS
mode.
We need to mirror this in our `esmLoader.ts`.
---------
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Also considered an alternative to not perform the locator handler check
during one-shot, but that would be somewhat against the promise of the
locator handler that is supposed to run **before** every expect check.
Fixes#32089.
Closes https://github.com/microsoft/playwright/issues/32050
When keyboarding through the action view, the UI continues showing the
hovered action. This makes keyboard nav hard to use.
The fix is to reset the higlighted action on keyboard navigation. This
is what we do when the mouse pointer leaves an action, and what I think
is reasonable.
Add Linux Mint 22 (Ubuntu based distro) version detection for dependency
installation
Linux Mint 22.x -> Ubuntu 24.04
(also see original Linux Mint support PR #28085)
Introduce `--tsconfig` to specify a single config to be used for all
imported files, instead of looking up tsconfig for each file separately.
Fixes#12829.
Update the documentation to match actual behavior.
The actual behavior today:
* Default mode is `full` when `recordHar` is passed to
`browser.newContext`
* Default mode is `minimal` when calling `context.routeFromHAR` and
`page.routeFromHAR`
Reference https://github.com/microsoft/playwright/issues/31983
Ideally we generate the timestamp when the Event gets created. This
patch adds a best-effort logic, since we can't override the constructor
of natively created events, e.g. `MouseEvent`.
Fixes https://github.com/microsoft/playwright/issues/31924
* Hide 'Status Code:' field for interrupted requests that don't have it.
* Clear up previously selected body when showing aborted requests.
* Highlight interrupted requests in red.
We populate `localStorage` using an init script. Currently, this script
isn't just run for the UI though, but also for all iframes. So we're
resetting `localStorage` every time the UI loads an iframe.
This hasn't been a problem in the past, because the only consumer of
`localStorage`, `Settings`, only read from `localStorage` once and kept
most state in `useState` afterwards. With
https://github.com/microsoft/playwright/pull/31911, this is no longer
true, so the bug starts biting us!
The fix is to ensure the init script isn't run on iframes.
This allows a dynamic import of a TS file to be processed by Babel.
For some reason, Playwright used to revert the CJS transforms. However,
ESM loader and transforms are always active, so CJS should be too.
When two attachments have the same content sha1, we used the first one's
name for the downloaded file, no matter which one the user clicked to
download. Now we pass the name explicitly.
References #31912.
Pulled out from https://github.com/microsoft/playwright/pull/31900
I stumbled over `React.Children`, because it's the first time I saw that
used. https://react.dev/reference/react/Children lists `React.Children`
it as "Legacy" and mentions it's uncommon. Also, the fact that SplitView
only displays its first two children, and all others are silently
discarded, can be a surprise to some.
By separating things out into `sidebar` and `main`, not only do we give
the two elements names (otherwise one needs to remember that sidebar is
always the first child), but we also prevent any "third children" from
being dropped.
Addresses https://github.com/microsoft/playwright/issues/31863. This PR
is chonky, but the individual commits should be easy to review. If
they're not, i'm happy to break them out into individual PRs.
There's two main things this does:
1. Remove some unused imports
2. Add a `clsx`-inspired helper function for classname templating
I wasn't able to replace `ReactDOM.render` with `ReactDOM.createRoot`.
This is the new recommended way starting with React 18, and the existing
one is going to be deprecated at some point. But it somehow breaks our
tests, i'll have to investigate that separately.
- Update attachments tab margins.
- Make sure to pass `&download` in attachment urls. This makes them
downloadable, regressed in #28727.
- Do not additionally list image diffs as screenshots.
Fixes#31912.
Closes https://github.com/microsoft/playwright/issues/22211
Currently, when the server notifies the UI about changed files, the UI
determines what files to re-run based on an old test list. By listing
tests before that, we make sure that the test list is up-to-date, and
that added tests are included in the next run.
I've also removed the `listChanged` event as discussed in the team sync.
The event isn't used anywhere and fires in exactly the same cases where
`testFilesChanged` fired, so i've folded them into one another. This allowed simplifying `Watcher`.
Motivation: When using client-certificates on a website on port `443`,
we would normalise the user input with `new URL` but still generate a
"bad" representation of the "origin" internally, since the just do
concatenated "host:port".
(The origin doesn't contain the port in case of :443)
We use `clientCertificatesToTLSOptions` in two places:
a) for APIRequestContext, there we pass one from the URL constructor
over and
b) from the socks proxy, there we **now** also pass a "good one" over.
Test plan: We don't want to run the tests on port :443, so only manually
validated the fix.
Relates https://github.com/microsoft/playwright/issues/31906
- Update copy to clipboard button.
- Reveal test source in the Source tab instead of external editor.
- New button to reveal in the external editor in the Source tab.
- Move the Pick Locator button next to snapshot tabs.
This TODO got added during our GHA refactoring a while ago, where I
thought it might make sense to run our itests in `bash`. We didn't do it
before the refactoring either. It seems good to keep them running in
`pwsh` instead, so lets just remove the TODO.
Investigation notes: Running in `bash` on Windows via `Git Bash` seems
like doing a lot of magic, especially with path handling. Things
[like](a02ed38e60/tests/installation/registry.ts (L143))
this break then, since `tar` even when using `Git Bash` [doesn't accept
a Windows
path](https://sourceforge.net/p/mingw/mailman/mingw-users/thread/54CE104A.7060108@hccnet.nl/).
Signed-off-by: Max Schmitt <max@schmitt.mx>
Upon calling `browser.close()` or dropping remote connection, make sure
to reject api calls before resolving `browser.close()` and firing a
`disconnected` event.
This change aligns the order guarantee with non-remote case.
Resolves https://github.com/microsoft/playwright/issues/31847 by adding
playwright config's `baseURL` value to the `context-options` trace
event, and showing that in the Trace Viewer.
Because the added property is optional, I didn't increment the trace
format version.
I've also considered pulling the `baseURL` from the existing
`browser.newContext` step to get around modifying the trace format, but
that felt pretty hacky.
https://github.com/user-attachments/assets/ecaef747-727d-4937-9ca3-1605ca9907b9
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Introduces an `--only-changed [base ref]` option.
`playwright test --only-changed` filters the test run to only run test
suites that have uncommitted changes.
`playwright test --only-changed=foo` runs only tests that were changed
since commit `foo`.
In pull request CI, this can be used to run changed tests first and fail
fast: `--only-changed=$GITHUB_BASE_REF`.
During local development, it can be used to quickly filter down to the
touched set of tests suites.
In some rare usecases, this can also help to cut down on CI usage for
pull requests. Tread with caution though.
File dependencies are taken into account to ensure that if you touched a
utility file, all relevant tests are still executed.
Closes https://github.com/microsoft/playwright/issues/15075
Broken out of https://github.com/microsoft/playwright/pull/31727 as per
@dgozman's
[request](https://github.com/microsoft/playwright/pull/31727#discussion_r1685793229).
The PR goal is to remove the `suite` argument from the Component
testing's Vite Plugin. `suite` is used to enrich Vite's dependency graph
with information about dependencies between test suites and helper
files. It essentially merges the Vite graph with the
`compilationCache.ts > fileDependencies` graph, and then writes the
result back into `compilationCache.ts > externalDependencies`.
By refactoring this to make the connection on the reading end in
`collectAffectedTestFiles`, we can drop the `suite` parameter.
We didn't yet have a test that depended on the dependency graph being
connected correctly between `fileDependencies` and
`externalDepedencies`, so I've [extended an existing
test](53a539938b)
to capture that.
This reverts commit 0aa2f06f68.
Discussed the new permission in the API review and decided not to
proceed with the feature as we are not ready to commit to supporting it
yet:
* the API is Chromium specific
* the API is still experimental
* there is no clarity to what extend the screen manipulation APIs will
work in old headless which is our main test environment
We'll keep an eye on the demand for the feature and may get back to
implementing it in the future.
Reference: https://github.com/microsoft/playwright/issues/27198
Fixes misspellings identified by the [check-spelling
action](https://github.com/marketplace/actions/check-spelling).
The misspellings have been reported at
https://github.com/jsoref/playwright/actions/runs/10015023629#summary-27685777352
The action will report that the changes in this PR would make it happy:
https://github.com/jsoref/playwright/actions/runs/10015023971#summary-27685778305
---
I understand that the commit messages will need to be reworded to match
house style. For the time being, these are merely noting the changes
they contain so that when I rebase or need to drop things, I can. --
I've already rebased once as someone fixed one of the items that my
draft work was going to fix.
---
## Testing
* The tests _mostly_ passed when I managed to trigger them, but there
were a handful of things that I didn't quite understand
* There are a large number of warnings relating to a bad interaction
between any workflow that uses this local action
b535139b32/.github/actions/run-test/action.yml (L74-L80)
and the action it calls -- I've opened Azure/login#474 asking them to
refactor their action so that it doesn't cause so much noise while
running this repository's tests
* I'm vaguely curious as to why this repository has a `branch`
constraint for its `pull_request` events in its workflows -- that
constraint gave me a number of additional headaches while trying to
prepare this branch for this PR.
---------
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Previously, only the "actual" attachment was created, pointing to the
file in `test-results`. Now, the "expected" attachment pointing to the
file in `__screenshots__` is also created. This will help any reporters
that would like to know the "expected" path, for example to do a manual
accept/decline of the baseline.
Fixes#30693.
The following actions keep `noWaitAfter` option: `click`, `selectOption`
and `press`.
All other actions that used to have `noWaitAfter` now behave like it was
set to true, not waiting for follow-up navigations. In the docs, this
option is marked as completely ignored.
A small logic change was made to compensate for this behavior: when
waiting for the `hitTargetInterceptor`, we now race it against
navigations to avoid stalling when navigation stalls. Previously,
waiting for the interceptor was disabled when `noWaitAfter` was passed,
and since it's impossible to pass this option now, we mitigate by never
stalling instead.
Fixes#31469.
... unless an array of file-system-friendly parts is provided.
Motivation: attachment name is used as a file system path when
downloading attachments, so we keep them fs-friendly.
References #30693.
When used in a terminal, the `list` reporter prints out information
about test steps to help debugging. In non-TTY environments like GitHub
Actions, currently it doesn't.
This PR changes that, so that in non-TTY environments you'll see the
"step end" messages appearing, but not the "step begin" messages. This
is a good middleground, because it helps the user understand test
progress, without being too verbose.
Closes https://github.com/microsoft/playwright/issues/31674
Retaining traces in the following scenarios:
- browser crash;
- manual `browser.close()`;
- implicit `browser.close()` from the `browser` fixture upon test end.
This does not affect the library, where `browser.close()` will not
retain the trace and will close the browser as fast as possible.
References #31541, #31535, #31537.
Redirects are always autoresumed, so the will always receive extra info
with raw headers. We only want to make raw headers available immediately
when there is a route.
Reference https://github.com/microsoft/playwright/issues/31351
* added more explanation to the sharding section for it to be more
understandable what it is and what it's purpose.
* because from the explanation in the docs. - it sound like that a
"shard" is a machine that is provided by playwright in addition to the
current setup , like a managed machine, but in reality it is simply a
separate job in the CI, dividing tests to separate jobs is sharding,
which sounds a lot easier when you think about it this way because it
could be confusing at the beginning IMO.
* Revert harTracer change from
aeba083da0
to make sure that har.Entry._monotonicTime always represents request
start time. The issue from the corresponding report was due to HEAD and
GET request sent for the same URL, that use case is still addressed as
we match by url + method
* Adjust resources monotonic time as well when several contexts are
shown in the trace viewer.
Fixes https://github.com/microsoft/playwright/issues/31133
Partially fixes https://github.com/microsoft/playwright/issues/31337 by
supporting ignoring node_modules on windows.
When I debug the function it gets a unix style path filename on windows,
so the function never ignores node_modules. The ignore path globs are
expected to use the unix path seperator and I've tested this fix works
on windows and I assume that since mac uses unix style, it also works
there (this is a pretty standard glob construct (chokidar points at any
match https://github.com/micromatch/anymatch and anymatch has this exact
example in their readme.md)
Signed-off-by: Luke Page <137174537+lukpsaxo@users.noreply.github.com>
Fixes https://github.com/microsoft/playwright/issues/31355
All changes were done with the Android Studio upgrade assistant. It
updates it to the latest Gradle to make it compatible with recent Java
while keeping the `targetSdkVersion` unchanged.
**Investigation**
~~We use `nonStallingEvaluateInExistingContext` as of today, which does
`eval()` inside (from our utilityScript) which breaks for some sites. It
causes a hang, since the returned `Promise` of `eval()` hangs. We don't
know as of today why this happens. Without wrapping it ini `eval()` it
does not hang.~~
~~Workaround: Do a plain Runtime.evaluate instead.~~
workaround: Only wait on main frame.
Relates https://github.com/microsoft/playwright/issues/28995 (keeping it
open until they confirm that it helps)
Historically, this plugin was important to translate dynamic imports
into require calls so that we can intercept them and transpile.
This is not needed anymore with ESM loader enabled by default, so we can
avoid this transformation and support dynamic imports of ESM-only
packages/files.
Fixes#17075, fixes#23255, fixes#31140, references #23662.
This allows snapshots of file:/// pages with external stylesheets,
images, etc to be rendered correctly in the trace viewer. (Otherwise, it
tries to request the file:/// URIs directly and the requests get blocked
by the browser.)
Fixes#31112.
* Column widths are now stored on in the NetworkPanel context, this way
they are not reset after selecting an empty range (and changing position
of the NetworkGridView in the component tree).
* Column widths values are now preserved if column set changes (e.g.
selecting entries from a single context and then from multiple
contexts).
When merging blob reports test ids are patched to make sure there is no
collision when merging reports that might have overlapping test ids.
However, even if you were merging reports that had no overlapping ids,
all test ids will be modified, which is an undesirable side effect.
This PR only modify test ids when the same test id has already been used
in a previous blob report.
----
This change is also part of
https://github.com/microsoft/playwright/pull/30962
The spirit of this change is reverting #23153. Since that time, we have
moved tracing and `artifactsDir` lifetime into the test runner, so the
reason for revert is mitigated.
Fixes#30287, fixes#30718, fixes#30959.
"To learn more about generating tests check out or detailed guide on
Codegen."
Should it be 'our'? And why does it link back to itself? Should be
removed.
Signed-off-by: Xan <57809064+devxan@users.noreply.github.com>
When a signal arrives late enough, after the last action was committed
in 5 seconds, we should still account for it. This includes downloads,
popups and dialogs, but not navigations.
Exposed by a flaky test "should record open in a new tab with url".
The documentation for command line usage of `--grep` did not provide the
correct string used for comparisons. This has now been fixed to include
the project name.
Fixes#30895
This includes two major changes:
- reuse `SerializedFS` for live test runner tracing;
- merge scheduled `appendFile` operations into a single `fs` call.
In some cases, this improves performance of UI mode by 61% and
performance of `trace: on` mode by 38%. Note that performance
improvement on the average test will not be as noticeable.
References #30875, #30635.
Chromium's `DevTools listening on` message sometimes arrives before
Playwright is finished connecting to Node. Without this patch, it would
miss the message and fail to connect.
- Documents `PLAYWRIGHT_FORCE_TTY` and `FORCE_COLOR` across terminal
reporters.
- New `PLAYWRIGHT_LIST_PRINT_STEPS`. Removes undocumented test-only
`PW_TEST_DEBUG_REPORTERS_PRINT_STEPS`.
- Replaces `PLAYWRIGHT_HTML_REPORT` with `PLAYWRIGHT_HTML_OUTPUT_DIR`
and `PW_TEST_HTML_REPORT_OPEN` with `PLAYWRIGHT_HTML_OPEN` for
consistency, supports older versions for backwards compatibility.
- New `PLAYWRIGHT_HTML_HOST`, `PLAYWRIGHT_HTML_PORT` and
`PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL`.
- New `PLAYWRIGHT_JUNIT_STRIP_ANSI` and
`PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME`.
- Removes `PW_HTML_REPORT` that was set for unknown reason.
This is a follow up #29564
I did a deep dive on a redirect issue I observed in my infrastructure
and originally attributed to some configuration mistakes on my part.
I have code hosted on `example.com/code` and use subdomain proxying.
This leads to the uimode being exposed on
`example.com/code/proxy/{{port}}`.
Clicking on the open uimode link shown by vscode redirected with a 302
to `example.com/proxy/{{port}}`
The absolute redirect url overruled the relative path handling reverse
proxies rely on.
This PR turns the absolute into a relative url to avoid this issue.
Previously, terminal reporters consulted `process.stdout.isTTY`. Now it
is possible to control the tty behavior:
- `PLAYWRIGHT_FORCE_TTY=0` or `PLAYWRIGHT_FORCE_TTY=false` to disable
TTY;
- `PLAYWRIGHT_FORCE_TTY=1` or `PLAYWRIGHT_FORCE_TTY=true` to enable TTY,
defaults to 100 columns when real columns are unavailable;
- `PLAYWRIGHT_FORCE_TTY=<number>` to force enable TTY and set the
columns.
Fixes#29422.
Previously, terminating worker always had a 30 seconds force exit.
Now, regular worker termination assumes that process will eventually
finish tearing down all the fixtures and exits. However, the
self-destruction routine keeps the 30 seconds timeout to avoid zombies.
Fixes#30504.
Implements feature requested in
https://github.com/microsoft/playwright/issues/30457
The test runner treats flaky tests as failures when the flag is enabled,
but still reports flaky tests as flaky in the reporting interface. It
feels like something worth discussing as this behaviour makes sense to
me, but looked a bit odd to @BJSS-russell-pollock when I ran this past
him.
Closes#30457.
Matching bu `apiName@wallTime` fails when two actions start at the same
time, e.g. two parallel api requests. Moreover, it results in trace
actions that have parent set to themselves, which in turn causes
infinite loop in the trace viewer. To avoid this problems we write
stepId explicitly to the library trace and use those step ids to find
corresponding test runner steps.
The stepId is passed via zone in case of expect, because the protocol
step is quite deep in the call chain after or explicitly in case of API
call steps created by the test runner instrumentation.
Pre Node.js `18.20.0`:
- `assert` is supported
Post Node.js `18.20.0`
- `assert` and `with` is supported.
Before https://github.com/microsoft/playwright/pull/30482 we kept
`asserts` in the JS code, Node.js was interpreting them. The `with`
keyword was not supported, this was what the PR was fixing.
After https://github.com/microsoft/playwright/pull/30482 Babel is
converting `assert` (deprecated) into `with` (successor) since we use
the `deprecatedAssertSyntax` option.
This means, that the minimum Node.js version we support in order to use
import attributes is now `18.20.0` where they added the `with` support.
This follows our principle of supporting only the latest minor release
for Node.js versions.
See here for the 18.20 changelog:
> #### Added support for import attributes
>
> Support has been added for import attributes, to replace the old
import
> assertions syntax. This will aid migration by making the new syntax
available
> across all currently supported Node.js release lines.
>
> This adds the `with` keyword which should be used in place of the
previous
> `assert` keyword, which will be removed in a future semver-major
Node.js
> release.
>
> For example,
>
> ```console
> import "foo" assert { ... }
> ```
>
> should be replaced with
>
> ```console
> import "foo" with { ... }
> ```
Fixes https://github.com/microsoft/playwright/pull/30482 - the tests
were a noop before, since they were tree-shaked by Babel.
Similarly to page.close, we pass test-runner specific reason to
facilitate better error messages.
```
1) a.test.ts:10:11 › test
Error: apiRequestContext.fetch: Fixture { request } from beforeAll cannot be reused in a test.
- Recommended fix: use a separate { request } in the test.
- Alternatively, manually create APIRequestContext in beforeAll and dispose it in afterAll.
See https://playwright.dev/docs/api-testing#sending-api-requests-from-ui-tests for more details.
9 |
10 | test('test', async () => {
> 11 | await context.fetch('http://example.com');
| ^
12 | });
13 |
```
Closes#29260.
* Use only monotonicTime for sorting, do not use wallTime for that
* Since test runner and the library can be running on different
machines, those machines may have clocks which are not synchronized. To
avoid problems in such cases we compute delta between test runner and
and library contexts based on a start time of action that exists in both
contexts.
When comparing `outputDir` and html-reporter `outputFolder`, we now make
sure that both paths end with a forward-slash.
Fixes#28677
---------
Co-authored-by: Georg Unterholzner <georg.unterholzner@dynatrace.com>
There are plenty of edge cases in this area:
- interrupted test run;
- did not run because of serial mode failure;
- failed before `test.skip()` call (e.g. in `beforeEach`) in one of the
retries;
- and more...
Related issues: #28322, #28321, #27455, #17652.
Prior changes: #27762, #26385, #28360, probably more.
There is still some duplication between `outcome()` and similar logic in
`base.ts`, which might be deduped in a follow-up.
Fixes#28322.
- Automatically waiting for the overlay locator to be hidden, with
`allowStayingVisible` opt-out.
- `times: 1` option.
- `removeLocatorHandler(locator, handler)` method.
- Passing `locator` as first argument to `handler`.
Fixes#30471. Fixes#30424. Fixes#29779.
Generate 'global timeout' and 'max failures' errors in the runner. Avoid
reading `config.globalTimeout` and `config.maxFailures` in the base
reporters.
Reference https://github.com/microsoft/playwright/issues/29768
There are new "non-manual" WPT accname tests that we now mostly pass,
which required a few tweeks in calculating role and name.
Also implemented accessible description computation, which is just a
small addition on top of accessible name, and passed respective wpt
tests.
References #18332.
Instead of filtering tests assuming there are no two projects with same
name we always rebuild test tree from scratch and restore previos test
results in the list mode.
Fixes https://github.com/microsoft/playwright/issues/30396
According to the spec, such controls should use the native value as long
as they have "aria-label". The relevant spec section is 2D.
However, there is an open issue that claims this should always apply,
and all browsers and wpt test actually do that:
https://github.com/w3c/accname/issues/64.
Fixes#28848.
The accessible name computation spec has changed to explicitly mention
this case:
Step 2A. Hidden Not Referenced. If the current node is hidden and is:
- Not part of an aria-labelledby or aria-describedby traversal, where
the node directly referenced by that relation was hidden.
- Nor part of a native host language text alternative element (e.g.
label in HTML) or attribute traversal, where the root of that traversal
was hidden.
See https://w3c.github.io/accname/#computation-steps. Chromium, Firefox
and Safari all agree with the spec.
Fixes#29796.
This makes it easier to create helper functions like:
```ts
function createReporter(options: MyOptions) {
return ['my-reporter', options] as const
}
```
At the moment, such functions can't be passed to `reporters` because a
readonly array is not assignable to the expected mutable array.
Playwirght certainly doesn't require those arrays to be mutable so it
would make sense to relax this.
Repro: `cd tests/components/ct-svelte && rm -rf node_modules
package-lock.json && npm i && npx playwright test --project=chromium`
Follow-up based on
https://github.com/microsoft/playwright/pull/28624#issuecomment-1858608101.
Svelte has no router by default, only SvelteKit - so lets remove the
package which is not maintained anymore and not recommended.
* FullConfig split into FullConfig and ConfigInWorker
* FullProject split into FullProject and ProjectInWorker
* Tests can only access {Config, Project}InWorker, while reporters can
only access Full{Config,Project}. This will allow in the future to
change the reporter API independently from the tests.
* Added documentation for the 4 classes.
When websocket disconnects during `browserType.connect()` call, the
error looks like this now:
```
browserType.connect: Custom error message received over WebSocket
```
Previously, it was a generic error:
```
browserType.connect: Target page, context or browser has been closed
```
Fixes#30040
# Context
In the `playwright.config` file, we can custom the default `timeout`
property for `expect.toPass` by the property:
```ts title='playwright.config.ts'
export default defineConfig({
// ...
expect: {
toPass: {
timeout: 60_000,
},
},
});
```
However, we can't customize the `intervals` option. So the default value
is always `[100, 250, 500, 1000]`.
# Goal
Add the possibility to customize the intervals option from the
`playwright.config` file.
```ts title='playwright.config.ts'
export default defineConfig({
// ...
expect: {
toPass: {
timeout: 60_000,
intervals: [1000, 2000, 3000]
},
},
});
```
* Add `Suite.entries` that returns tests and suites in their declaration
order
* Exposed `Suite.type` and `TestCase.type` for discriminating between
different entry types.
* Blob report format is updated to store entries instead of separate
lists for suites and tests.
* Bumped blob format version to 2, added modernizer.
Fixes https://github.com/microsoft/playwright/issues/29984
Workers use page's session for `Fetch` domain and worker's session for
`Network` domain. Therefore, `CRNetworkManager` should keep track of the
right session for each domain separately.
This is covered by currently flaky tests:
- `should report and intercept network from nested worker`,
- `should intercept network activity from worker`,
- `should intercept network activity from worker 2`.
This brings up the question that we would show the tag name twice if its
a tag in a title. This would be aligned to how HTML report is doing it.
Fixes https://github.com/microsoft/playwright/issues/29927
---------
Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
With the recent change that gave after hooks a separate timeout, fixture
teardown does not imply that "test finished successfully, but fixture
teardown was slow".
This was historically done to make `console.log()` have colors. However,
this makes any other code that checks `process.stdout.isTTY` incorrectly
assume real TTY support.
Node18 and Node20 now respect `FORCE_COLOR=1` in console, so our default
behavior of forcing colors in the worker process just works out of the
box. See https://github.com/nodejs/node/pull/48034.
- Migrate from `:::warning` to `:::danger[Warning]`.
- Migrate from `:::caution` to `:::warning`.
- `:::warning` and `:::caution` were the color same before.
- `:::caution` is deprecated and gets removed in v4 of Docusaurus
- Also migrate to the [newer title
syntax](3877c1254f/packages/docusaurus-utils/src/markdownUtils.ts (L86-L91))
---------
Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
**Investigation:**
- According to
[here](https://github.com/microsoft/playwright/actions/runs/8351676198),
the job got cancelled, because someone force-pushed and another commit
took priority.
- We have `if: always()` for uploading the blobs, even if they are
cancelled. We expect, that the task before (`npx playwright test`)
finished writing the blob, but it didn't - cancelled in between. We
still upload - a broken one.
- We download the broken zip and it breaks from there on.
Proposed change: Instead of uploading always, upload if it did not get
cancelled.
Quoting from the GitHub Actions docs about `always()`:
> Warning: Avoid using always for any task that could suffer from a
critical failure, for example: getting sources, otherwise the workflow
may hang until it times out. If you want to run a job or step regardless
of its success or failure, use the recommended alternative: if: `${{
!cancelled() }}`
This is phase 1/2 where it changes our code to use this new condition.
The actual roll-out happens once it works for us.
Relates https://github.com/microsoft/playwright/issues/29451
Previously, there was at most one "hard error", as opposite to multiple
"soft errors". This was done to preserve the historic behavior at the
time of introducing multiple `TestInfo.errors`.
With this change, every user callback that is executed `withRunnable()`
can throw an error and/or timeout, and both of these will end up in
`TestInfo.errors`.
Additionally, there is at most one "unhandled exception" error, to avoid
flooding the report with mass failures.
Drive-by: remove boolean arguments from `_failWithError()`.
Fixes#29876.
When collecting dependencies both from CJS loader and from ESM loader,
the latter would overwrite the dependencies set instead of appending.
Also make sure cts/cjs/mts/mjs are all supported equally.
References #29747.
Test-specific output dir and snapshot names are trimmed to 60 chars
instead of just 100. The snapshot names are still trimmed at 100 chars
for backwards compatibility.
Reference https://github.com/microsoft/playwright/issues/29719
This helps in a case where navigating to an origin fails for some
reason, for example because a registered service worker loads some
content into the supposedly blank page.
Fixes#29402.
When displaying remote context trace and local test runner trace
together, we should not compare monotonic time between the two.
This change reuses existing logic for merging actions timing to also
update context boundaries and events by the same time delta.
Fixes#29767.
This covers blocked requests, e.g. mixed-content, that receive
`loadingFailed` with empty `errorText`.
Also, forcefully resolve `allHeaders()` in this case, since we know
there will be no actual network headers.
Fixes#29833.
This fixes a case where we incorrectly used the final config's base path
when resolving relative path mappings in the absence of the baseUrl.
Fixes#29816.
This PR is a fix proposal for a bug when trying to record a omnibox
navigation after a recorded action (e.g., `fill`).
The following test, included in this PR, reproduces the problem:
```ts
test('should record omnibox navigations after recordAction', async ({ page, openRecorder, server }) => {
const recorder = await openRecorder();
await recorder.setContentAndWait(`<textarea></textarea>`);
await Promise.all([
recorder.waitForOutput('JavaScript', 'fill'),
page.locator('textarea').fill('Hello world'),
]);
// for performed actions, 5 seconds is the time needed to ensure they are committed
await page.waitForTimeout(5000);
await page.goto(server.PREFIX + `/empty.html`);
await recorder.waitForOutput('JavaScript', `await page.goto('${server.PREFIX}/empty.html');`);
});
```
After performed actions (e.g., `click`), it successfully records the
navigation as long as there's at least a 5 sec. gap between both
actions. That happens because after that 5 sec. interval the performed
action is automatically commited and therefore the navigation is not
stored as a signal of that action.
The proposed fix for recorded actions also forces that action to be
automatically commited after 5 sec (for testing, I'm using 500ms to
speed up the test execution).
It was already handling worker sessions, but not OOPIFs. As a result,
some functionality was properly implemented only for OOPIFs and not for
workers.
This change removes OOPIFs fanout for network-related calls from CRPage
and moves that to the CRNetworkManager, now also covering workers.
This was missed in https://github.com/microsoft/playwright/pull/29131
Fixes the following error in java on windows:
```
Error: Cannot find module 'C:\Users\RUNNER~1\AppData\Local\Temp\playwright-java-910508779801292985\package\lib\cli\cli.js'
at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
at Module._load (node:internal/modules/cjs/loader:985:27)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
at node:internal/main/run_main_module:28:49 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
```
This will be useful to run `beforeAll`/`afterAll` hooks with a separate
`TestInfo` instance, as well as run use helpers like
`_runAndFailOnError()` during scope teardown.
This fixes some edge cases where fixtures and steps inside them were
attached to the wrong parent (see the new test).
This will also allow to replace some `runAsStep` calls with a flat list
of tasks to do that do not rely on lexical scope.
API changes:
- `test(title, details, body)` where details contain `tag` and
`annotation`.
- similar `details` property added to `test.skip`, `test.fail`,
`test.fixme`, `test.only`, `test.describe` and other `test.describe.*`
variations.
- `TestProject.tagFilter`/`TestConfig.tagFilter` that supports logical
tag expressions with `(`, `)`, `and`, `or` and `not`.
- `--tag` CLI option to filter by tags.
- New annotations are available in `TestInfo.annotations` and
`TestCase.annotations`.
- New tags are available in `TestCase.tags`.
Reporter changes:
- `json` reporter includes new tags in addition to old `@smoke`-style
tags. **Breaking**: tags are now listed with the leading `@` symbol.
- `html` reporter filters by old and new tags with the same `@smoke`
token.
Fixes#29229, fixes#23180.
The metadata.error change was brought back in
https://github.com/microsoft/playwright/pull/29271and it broke java port
as we could have error and result set simulteniously. This PR moves the
logic to the trace recorder instead and keeps the protocol contract
clear that either error or result is present, but not both.
- Modifiers that only depend on the worker fixtures are implemented as
`beforeAll` hooks.
- Modifiers that depend on test fixtures are implemented as `beforeEach`
hooks.
- Pushed `_runAndFailOnError` down the stack, wrapping individual hooks
instead of the whole "before hooks" section.
- Reused the same code to run `beforeAll` and `afterAll` hooks and
modifiers.
**Behavior change**: `test.skip()` inside a `beforeAll` now skips the
hook and all tests in the suite.
When `updateSnapshots === 'missing'` we generate new expectations on the
first attempt and don't retry the test afterwards instead of trying it
retries-1 times and only writing new expectation on the last attempt.
This logic infects all serial mode suites that contain the test with
missing expectations, so they also will not be retried.
Reference https://github.com/microsoft/playwright/issues/29073
Motivation: Sometimes the files are not available anymore on the Service
Worker side. The Trace Viewer frontend then falls back to `file?path=`
and uses its response, no matter what the response status is.
This patch checks for the response status code before using its
response.
e.g. https://github.com/microsoft/playwright-dotnet/issues/2775 after
some time / and some other reports.
---------
Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
We stopped catching all exceptions in
https://github.com/microsoft/playwright/pull/28539 in hope that we'll
get loadingFailed even before Fetch.continue/fulfill command's error.
Turns out this is racy and may fail if the test cancels the request
while we are continuing it. The following test could in theory reproduce
it if stars align and the timing is good:
```js
it('page.continue on canceled request', async ({ page }) => {
let resolveRoute;
const routePromise = new Promise<Route>(f => resolveRoute = f);
await page.route('http://test.com/x', resolveRoute);
const evalPromise = page.evaluate(async () => {
const abortController = new AbortController();
(window as any).abortController = abortController;
return fetch('http://test.com/x', { signal: abortController.signal }).catch(e => 'cancelled');
});
const route = await routePromise;
void page.evaluate(() => (window as any).abortController.abort());
await new Promise(f => setTimeout(f, 10));
await route.continue();
const req = await evalPromise;
expect(req).toBe('cancelled');
});
```
Fixes https://github.com/microsoft/playwright/issues/29123
Motivation: On Windows we call around 50 times `PrintDeps.exe` which
takes on a very fast machine 500+ms. On Linux we do it around 120 times
(`ldd`) which takes around 150ms.
This change validates the dependencies once on browser install (`npx
playwright install`). In case its failing, it will emit a warning, in
case of a success, it will create a marker file that the binary has been
validated. For future `launch()` calls, we'll read this file and if
exists, we'll not validate again. Otherwise we'll validate again.
Note: If the marker file is older than 30 days, the browser will be
validated again.
This will now yield:
```
root@a85fb37f0c96:/work# npx playwright install
Failed to install browsers
Error: ERROR: Playwright does not support chromium on ubuntu18.04-x64
```
On Ubuntu 18.04.
Previously, new `Recorder` instance was given an existing
`InjectedScript`. However, we built a separate source for
`InjectedScript` vs `Recorder`, and both bundles contain their own copy
of all helper modules, e.g. `roleUtils`.
This resulted in two copies of helper modules, which is troublesome for
any module-level globals like a top-level cache. Depending on whether
`Recorder` or `InjectedScript` called into the helper, they would access
the different value of a module global, which lead to bugs.
To prevent this, we force any external dependencies to be imported
through the `InjectedScript.utils`.
* Before the change if if uploaded artifact path contained just one file
it would be [archived
directly](694cdabd8b/src/search.ts (L142-L151)),
but if there is a sister folder it would be an archive of the entire
dir. According to the action docs "If a wildcard pattern is used, the
path hierarchy will be preserved after the first wildcard pattern", so
hopefully it means that the directory is always compressed even if there
is a single file in it.
* Use `overwrite: true` to have only one artifact with pull request
number.
This supports mixed quotes locators in JavaScript where we are not sure
what quote is the correct one, so we normalize to unescaped single quote
when comparing with the original.
Drive-by: we were allowing single quotes in Python, Java and .NET, but
these are actually not allowed.
Regressed in #27718.
Fixes#28630.
Otherwise, we forever block SIGTERM and SIGHUP by registering a handler
that does not do anything (due to no browsers to close) and prevents
default handler that exits from running.
Fixes#28091.
- Replace action names with links to `Locator.click`.
- Remove "Attached" check in favor of "locator resolves to a single
element".
- Replace getter methods in assertion section with auto-retriying
assertions.
- Add missing actions.
---
<img width="837" alt="Screenshot 2024-01-11 at 1 02 34 PM"
src="https://github.com/microsoft/playwright/assets/9881434/1946678d-6a7e-45cf-a299-2f18cb3fb5a5">
Motivation: Before this change if a language binding invoked the CLI
with `--help` or without any arguments (which will also show the help
text) it was suggesting that test/merge-reports/show-report is something
we support. After this change this does not get shown anymore.
`actions/upload-artifact@v4` comes with the following [breaking
change](https://github.com/actions/upload-artifact?tab=readme-ov-file#breaking-changes):
"Due to how Artifacts are created in this new version, it is no longer
possible to upload to the same named Artifact multiple times. You must
either split the uploads into multiple Artifacts with different names,
or only upload once. Otherwise you will encounter an error."
Due to that we cannot copy multiple blob report folders into the same
artifact name and rely on the action to merge them. Instead, as
suggested by their migration guide, we upload each blob report into a
uniquely named artifact with prefix `blob-report-` and then download all
of them into same directory.
This version change also affects how we store pull_request_number.txt
into an artifact. Previously we relied on the fact that uploading
artifact with the same name would silently override existing one, but
now it's an error. To overcome that, we upload PR number file into
uniquely named artifacts `pull-request-*` and later extract them into
same location with `unzip -n` which will never override existing file,
so we end up with single `pull_request_number.txt`.
Reference #28800
This changes error message from `Error: ` to `Error: Protocol error
(Fetch.continueRequest): Internal server error, session closed.` when
running `npm run ctest -- --repeat-each 100 -x --headed --timeout 3000
--workers 1 library/browsercontext-route.spec.ts:172` prior to
9d91b7caf5.
- Keep main frames in a separate bucket, so that page methods that
redirect to the main frame continue to work.
- Increase default dispatchers limit to `10_000`.
- Increase dispatchers limit for `JSHandle`/`ElementHandle` to
`100_000`.
Fixes#28320, #28503.
Add the `'credentials': include` option on the trace fetch so the
browser can include cookies when fetching from a different origin,
assuming the origin returns the correct Access-Control-Allow-Origin and
Access-Control-Allow-Credentials headers.
Fixes#28501
This is a follow-up to 119afdf788 Since
continue/fulfill/abort now may throw on the server after page has been
closed, we need to catch the errors manually. On the client it's fixed
by the original change.
This fixes errors in the existing tests:
```
1) [chromium] › library/browsercontext-route.spec.ts:172:3 › should support Set-Cookie header ────
Error:
at ../packages/playwright-core/src/server/chromium/crConnection.ts:147
145 | const id = this._connection._rawSend(this._sessionId, method, params);
146 | return new Promise((resolve, reject) => {
> 147 | this._callbacks.set(id, { resolve, reject, error: new ProtocolError('error', method) });
| ^
148 | });
149 | }
150 |
at /Users/yurys/playwright/packages/playwright-core/src/server/chromium/crConnection.ts:147:57
at new Promise (<anonymous>)
at CRSession.send (/Users/yurys/playwright/packages/playwright-core/src/server/chromium/crConnection.ts:146:12)
at RouteImpl.continue (/Users/yurys/playwright/packages/playwright-core/src/server/chromium/crNetworkManager.ts:566:25)
at FrameManager.requestStarted (/Users/yurys/playwright/packages/playwright-core/src/server/frames.ts:299:23)
at CRNetworkManager._onRequest (/Users/yurys/playwright/packages/playwright-core/src/server/chromium/crNetworkManager.ts:314:57)
at CRNetworkManager._onRequestPaused (/Users/yurys/playwright/packages/playwright-core/src/server/chromium/crNetworkManager.ts:202:12)
at CRSession.emit (node:events:517:28)
at /Users/yurys/playwright/packages/playwright-core/src/server/chromium/crConnection.ts:172:14
at runNextTicks (node:internal/process/task_queues:60:5)
at processImmediate (node:internal/timers:447:9)
```
Reference https://github.com/microsoft/playwright/issues/28490
In some circumstances, like "No tests found" error, reporters produce a
lot of unnecessary empty lines:
```
$ npx playwright test
Error: No tests found
$ <next command>
```
Also, `line` reporter removes the `npx playwright test` command entirely
when `onError` happens before `onBegin`.
- in docs;
- in the error message.
Terminal output:
```
$ npx playwright test foobar
Error: No tests found.
Make sure that arguments are regular expressions matching test files.
You may need to escape symbols like "$" or "*" and quote the arguments.
```
References #28551.
If request gets cancelled by the page before we fulfill it, we receive
`loadingFailed` event. In that case we'll ignore interception error if
any, otherwise the error will be propagated to the caller.
Fixes https://github.com/microsoft/playwright/issues/28490
* Check if handler is still in the route list before calling it
* Check if the handler is still in the list before removing it after
`times` expiration
This PR fixes
https://github.com/microsoft/playwright/pull/27790#pullrequestreview-1738958803.
Previously this function returns only the first error when some of the
promises fail. But the type annotation suggests that the original
intention was to collect all the errors. This commit fixes the error
values, and unexpected `TypeError: object is not iterable`.
Otherwise, merging two configs without `projects` property will create a
config with an empty project list, which is semantically different and
always leads to "No tests found".
Goal - Capture minimal diagnostic information for each message being
sent between the playwright client and server.
---------
Co-authored-by: Siddharth Singha Roy <ssingharoy@microsoft.com>
We used to have a fake `error` property, so that trace viewer shows
failed expectes as such. Today, we have a step for each expect that
contains a proper error. Sending the fake error to the client confuses
language ports.
I have corrected some typos and grammar issues that I found when reading
your documentation.
---------
Signed-off-by: Elijah Mock <28277163+ekcom@users.noreply.github.com>
The documentation is misleading because `page.evaluate` never returns a
handle.
Some other parts of the documentation may need to be updated as well
(especially other languages than javascript, I think I saw this issue
also on other pages).
---------
Signed-off-by: divdavem <david-emmanuel.divernois@amadeus.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
Abstract issue: I have parameterised tests via the titlePath (describe)
and they are not shown on the flakiness dashboard. Before #24486 only
title() was shown, after #24486 titlePath() will be displayed, but we
still did have the testId based on the test.title() and not
test.titlePath(). This ends up that they will still be treated as a
single spec.
After this change they are not treated as a single spec anymore and
treated as different ones:
<img width="891" alt="image"
src="https://github.com/microsoft/playwright/assets/17984549/f24284cd-5d94-4f7e-a45d-8c8e5eb537ef">
Note: This is tested.
Follow-up on https://github.com/microsoft/playwright/pull/24486.
We saw in the past a lot of users reporting issues like this. Maybe a
more readable error helps.
Before they just saw:
```
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'register')
at index.38834ec3.js:1:3775
at index.38834ec3.js:1:4006
```
Fixes https://github.com/microsoft/playwright/issues/27655
- remove `onlyStartedTests` in favor of explicit branch with comments;
- produce one "test not found" error per test instead of a single large
error;
- extract `_failTestWithErrors` from `_massSkipTestsFromRemaining`.
Before this fix, unhandled error during test.fail():
- marks this test as "interrupted";
- fails next test in the file with "fatal error".
After this fix:
- marks this test as "failed as expected";
- restarts worker for the next test.
**Description**
When a language port was using Inspector with the "Locator Picker"
feature, it only recognised JavaScript as a language by default. As a
workaround the user was able to click record, interact with the page and
then the language would be correctly used -> csharp e.g. would work in
the "Locator Picker".
**Why?**
Our language bindings are setting `PW_LANG_NAME=<sdkLanguage>` env var
-> good. Our recorder harness also uses this along its internal state
here:
b9b289b641/packages/playwright-core/src/server/recorder.ts (L369)
and it gets used here (no parameter means: we use the first language
aka. primary language):
b9b289b641/packages/playwright-core/src/server/recorder.ts (L95)
The only issue is that the Inspector frontend in the beginning does not
know which language it should use and pass over to the server side, it
then falls back to JavaScript.
**Proposed fix**
Instead of passing it over from the frontend to the server side, we just
always use it from the server side, aka. "currentLanguage". When the
user switches languages in the frontend, "currentLanguage" already gets
updated properly via the "fileChanged" event.
https://github.com/microsoft/playwright-dotnet/issues/2718
---------
Signed-off-by: Max Schmitt <max@schmitt.mx>
On Linux platforms, specifically check that process.arch is x64, rather
than treating it as 'not arm64'.
Treat Raspbian's /etc/os-release file as Debian.
Document the supported platforms somewhat.
Fixes#27453
There is a mistake in the release version of the `toHaveAttribute`
method. It was released in the 1.39 version and not in the 1.40 like was
mentioned in the documentation.
description:Create a bug report to help us improve
title: '[Bug]:'
body:
- type:markdown
attributes:
value:|
# Please follow these steps first:
- type:markdown
attributes:
value:|
## Troubleshoot
If Playwright is not behaving the way you expect, we'd ask you to look at the [documentation](https://playwright.dev/docs/intro) and search the issue tracker for evidence supporting your expectation.
Please make reasonable efforts to troubleshoot and rule out issues with your code, the configuration, or any 3rd party libraries you might be using.
Playwright offers [several debugging tools](https://playwright.dev/docs/debug) that you can use to troubleshoot your issues.
- type:markdown
attributes:
value:|
## Ask for help through appropriate channels
If you feel unsure about the cause of the problem, consider asking for help on for example [StackOverflow](https://stackoverflow.com/questions/ask) or our [Discord channel](https://aka.ms/playwright/discord) before posting a bug report. The issue tracker is not a help forum.
- type:markdown
attributes:
value:|
## Make a minimal reproduction
To file the report, you will need a GitHub repository with a minimal (but complete) example and simple/clear steps on how to reproduce the bug.
The simpler you can make it, the more likely we are to successfully verify and fix the bug. You can create a new project with `npm init playwright@latest new-project` and then add the test code there.
Please make sure you only include the code and the dependencies absolutely necessary for your repro. Due to the security considerations, we can only run the code we trust. Major web frameworks are Ok to use, but smaller convenience libraries are not.
- type:markdown
attributes:
value:|
> [!IMPORTANT]
> Bug reports without a minimal reproduction will be rejected.
---
- type:input
id:version
attributes:
label:Version
description:|
The version of Playwright you are using.
Is it the [latest](https://github.com/microsoft/playwright/releases)? Test and see if the bug has already been fixed.
placeholder:ex. 1.41.1
validations:
required:true
- type:textarea
id:reproduction
attributes:
label:Steps to reproduce
description:Please link to a repository with a minimal reproduction and describe accurately how we can reproduce/verify the bug.
placeholder:|
Example steps (replace with your own):
1. Clone my repo at https://github.com/<myuser>/example
2. npm install
3. npm run test
4. You should see the error come up
validations:
required:true
- type:textarea
id:expected
attributes:
label:Expected behavior
description:A description of what you expect to happen.
placeholder:I expect to see X or Y
validations:
required:true
- type:textarea
id:what-happened
attributes:
label:Actual behavior
description:|
A clear and concise description of the unexpected behavior.
Please include any relevant output here, especially any error messages.
placeholder:A bug happened!
validations:
required:true
- type:textarea
id:context
attributes:
label:Additional context
description:Anything else that might be relevant
validations:
required:false
- type:textarea
id:envinfo
attributes:
label:Environment
description:|
Please paste the output of running `npx envinfo --preset playwright`.
This will be automatically formatted as a code block, so no need for backticks.
if [[ "$version" == "14" || "$version" == "15" ]]; then
sqlite3 $HOME/Library/Application\ Support/com.apple.TCC/TCC.db "INSERT OR IGNORE INTO access VALUES ('kTCCServiceMicrophone','/usr/local/opt/runner/provisioner/provisioner',1,2,4,1,NULL,NULL,0,'UNUSED',NULL,0,1687786159,NULL,NULL,'UNUSED',1687786159);"
sqlite3 $HOME/Library/Application\ Support/com.apple.TCC/TCC.db "INSERT OR REPLACE INTO access VALUES('kTCCServiceMicrophone','/usr/local/opt/runner/provisioner/provisioner',1,2,4,1,NULL,NULL,0,'UNUSED',NULL,0,1687786159);"
else
echo "Skipping unsupported macOS version $version"
Playwright **requires an issue** for every contribution, except for minor documentation updates. We strongly recommend to pick an issue labeled `open-to-a-pull-request` for your first contribution to the project.
We strongly recommend that you open an issue before beginning any code modifications. This is particularly important if the changes involve complex logic or if the existing code isn't immediately clear. By doing so, we can discuss and agree upon the best approach to address a bug or implement a feature, ensuring that our efforts are aligned.
If you are passioned about a bug/feature, but cannot find an issue describing it, **file an issue first**. This will facilitate the discussion and you might get some early feedback from project maintainers before spending your time on creating a pull request.
### Getting Code
Make sure you're running Node.js 16+ and NPM 8+, to verify and upgrade NPM do:
## Make a change
Make sure you're running Node.js 20 or later.
```bash
node --version
npm --version
npm i -g npm@latest
```
1. Clone this repository
Clone the repository. If you plan to send a pull request, it might be better to [fork the repository](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) first.
```bash
git clone https://github.com/microsoft/playwright
cd playwright
```
2. Install dependencies
Install dependencies and run the build in watch mode.
```bash
npm ci
npm run watch
npx playwright install
```
3. Build Playwright
**Experimental dev mode with Hot Module Replacement for recorder/trace-viewer/UI Mode**
```bash
npm run build
```
PW_HMR=1 npm run watch
PW_HMR=1 npx playwright show-trace
PW_HMR=1 npm run ctest -- --ui
PW_HMR=1 npx playwright codegen
PW_HMR=1 npx playwright show-report
```
4. Run all Playwright tests locally. For more information about tests, read [Running & Writing Tests](#running--writing-tests).
Playwright is a multi-package repository that uses npm workspaces. For browser APIs, look at [`packages/playwright-core`](https://github.com/microsoft/playwright/blob/main/packages/playwright-core). For test runner, see [`packages/playwright`](https://github.com/microsoft/playwright/blob/main/packages/playwright).
```bash
npm test
```
Note that some files are generated by the build, so the watch process might override your changes if done in the wrong file. For example, TypeScript types for the API are generated from the [`docs/src`](https://github.com/microsoft/playwright/blob/main/docs/src).
### Code reviews
Coding style is fully defined in [.eslintrc](https://github.com/microsoft/playwright/blob/main/.eslintrc.js). Before creating a pull request, or at any moment during development, run linter to check all kinds of things:
```bash
npm run lint
```
All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.
Comments should have an explicit purpose and should improve readability rather than hinder it. If the code would not be understood without comments, consider re-writing the code to make it self-explanatory.
### Code Style
### Write documentation
- Coding style is fully defined in [.eslintrc](https://github.com/microsoft/playwright/blob/main/.eslintrc.js)
- Comments should be generally avoided. If the code would not be understood without comments, consider re-writing the code to make it self-explanatory.
Every part of the public API should be documented in [`docs/src`](https://github.com/microsoft/playwright/blob/main/docs/src), in the same change that adds/changes the API. We use markdown files with custom structure to specify the API. Take a look around for an example.
To run code linter, use:
Various other files are generated from the API specification. If you are running `npm run watch`, these will be re-generated automatically.
```bash
npm run eslint
```
Larger changes will require updates to the documentation guides as well. This will be made clear during the code review.
### API guidelines
## Add a test
When authoring new API methods, consider the following:
Playwright requires a test for almost any new or modified functionality. An exception would be a pure refactoring, but chances are you are doing more than that.
- Expose as little information as needed. When in doubt, don’t expose new information.
- Methods are used in favor of getters/setters.
- The only exception is namespaces, e.g. `page.keyboard` and `page.coverage`
- All string literals must be lowercase. This includes event names and option values.
- Avoid adding "sugar" API (API that is trivially implementable in user-space) unless they're **very** common.
There are multiple [test suites](https://github.com/microsoft/playwright/blob/main/tests) in Playwright that will be executed on the CI. The two most important that you need to run locally are:
### Commit Messages
- Library tests cover APIs not related to the test runner.
```bash
# fast path runs all tests in Chromium
npm run ctest
Commit messages should follow the Semantic Commit Messages format:
# slow path runs all tests in three browsers
npm run test
```
- Test runner tests.
```bash
npm run ttest
```
Since Playwright tests are using Playwright under the hood, everything from our documentation applies, for example [this guide on running and debugging tests](https://playwright.dev/docs/running-tests#running-tests).
Note that tests should be *hermetic*, and not depend on external services. Tests should work on all three platforms: macOS, Linux and Windows.
## Write a commit message
Commit messages should follow the [Semantic Commit Messages](https://www.conventionalcommits.org/en/v1.0.0/) format:
```
label(namespace): title
@ -93,144 +92,57 @@ footer
```
1. *label* is one of the following:
- `fix` - playwright bug fixes.
- `feat` - playwright features.
- `docs` - changes to docs, e.g. `docs(api.md): ..` to change documentation.
- `test` - changes to playwright tests infrastructure.
- `devops` - build-related work, e.g. CI related patches and general changes to the browser build infrastructure
- `fix` - bug fixes
- `feat` - new features
- `docs` - documentation-only changes
- `test` - test-only changes
- `devops` - changes to the CI or build
- `chore` - everything that doesn't fall under previous categories
2. *namespace* is put in parenthesis after label and is optional. Must be lowercase.
3. *title* is a brief summary of changes.
4. *description* is **optional**, new-line separated from title and is in present tense.
5. *footer* is **optional**, new-line separated from *description* and contains "fixes" / "references" attribution to github issues.
1. *namespace* is put in parenthesis after label and is optional. Must be lowercase.
1. *title* is a brief summary of changes.
1. *description* is **optional**, new-line separated from title and is in present tense.
1. *footer* is **optional**, new-line separated from *description* and contains "fixes" / "references" attribution to github issues.
Example:
```
fix(firefox): make sure session cookies work
feat(trace viewer): network panel filtering
This patch fixes session cookies in the firefox browser.
This patch adds a filtering toolbar to the network panel.
<linktoascreenshot>
Fixes #123, fixes #234
Fixes #123, references #234.
```
### Writing Documentation
## Send a pull request
All API classes, methods, and events should have a description in [`docs/src`](https://github.com/microsoft/playwright/blob/main/docs/src). There's a [documentation linter](https://github.com/microsoft/playwright/tree/main/utils/doclint) which makes sure documentation is aligned with the codebase.
All submissions, including submissions by project members, require review. We use GitHub pull requests for this purpose. Consult [GitHub Help](https://help.github.com/articles/about-pull-requests/) for more information on using pull requests.
To run the documentation linter, use:
After a successful code review, one of the maintainers will merge your pull request. Congratulations!
```bash
npm run doc
```
## More details
To build the documentation site locally and test how your changes will look in practice:
**No new dependencies**
1. Clone the [microsoft/playwright.dev](https://github.com/microsoft/playwright.dev) repo
1. Follow [the playwright.dev README instructions to "roll docs"](https://github.com/microsoft/playwright.dev/#roll-docs) against your local `playwright` repo with your changes in progress
1. Follow [the playwright.dev README instructions to "run dev server"](https://github.com/microsoft/playwright.dev/#run-dev-server) to view your changes
There is a very high bar for new dependencies, including updating to a new version of an existing dependency. We recommend to explicitly discuss this in an issue and get a green light from a maintainer, before creating a pull request that updates dependencies.
### Adding New Dependencies
For all dependencies (both installation and development):
- **Do not add** a dependency if the desired functionality is easily implementable.
- If adding a dependency, it should be well-maintained and trustworthy.
A barrier for introducing new installation dependencies is especially high:
- **Do not add** installation dependency unless it's critical to project success.
### Running & Writing Tests
- Every feature should be accompanied by a test.
- Every public api event/method should be accompanied by a test.
- Tests should be *hermetic*. Tests should not depend on external services.
- Tests should work on all three platforms: Mac, Linux and Win. This is especially important for screenshot tests.
Playwright tests are located in [`tests`](https://github.com/microsoft/playwright/blob/main/tests) and use `@playwright/test` test runner.
These are integration tests, making sure public API methods and events work as expected.
- To run all tests:
```bash
npx playwright install
npm run test
```
Be sure to run `npm run build` or let `npm run watch` run before you re-run the
tests after making your changes to check them.
- To run all tests in Chromium
```bash
npm run ctest # also `ftest` for firefox and `wtest` for WebKit
```
- To run the Playwright test runner tests
```bash
npm run ttest
npm run ttest -- --grep "specific test"
```
- To run a specific test, substitute `it` with `it.only`, or use the `--grep 'My test'` CLI parameter:
- To run tests with custom browser executable, specify `CRPATH`, `WKPATH` or `FFPATH` env variable that points to browser executable:
**Custom browser build**
To run tests with custom browser executable, specify `CRPATH`, `WKPATH` or `FFPATH` env variable that points to browser executable:
```bash
CRPATH=<path-to-executable> npm run ctest
```
- To run tests in slow-mode:
You will also find `DEBUG=pw:browser` useful for debugging custom builds.
```bash
SLOW_MO=500 npm run wtest -- --headed
```
**Building documentation site**
- When should a test be marked with `skip` or `fail`?
The [playwright.dev](https://playwright.dev/) documentation site lives in a separate repository, and documentation from [`docs/src`](https://github.com/microsoft/playwright/blob/main/docs/src) is frequently rolled there.
- **`skip(condition)`**: This test *should ***never*** work* for `condition`
where `condition` is usually a certain browser like `FFOX` (for Firefox),
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
with `skip(FFOX)` since an alt-click in Firefox will not produce a download
even if a person was driving the browser.
- **`fail(condition)`**: This test *should ***eventually*** work* for `condition`
where `condition` is usually a certain browser like `FFOX` (for Firefox),
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
with `fail(CHROMIUM || WEBKIT)` since Playwright performing these actions
currently diverges from what a user would experience driving a Chromium or
WebKit.
Most of the time this should not concern you. However, if you are doing something unusual in the docs, you can build locally and test how your changes will look in practice:
1. Clone the [microsoft/playwright.dev](https://github.com/microsoft/playwright.dev) repo.
1. Follow [the playwright.dev README instructions to "roll docs"](https://github.com/microsoft/playwright.dev/#roll-docs) against your local `playwright` repo with your changes in progress.
1. Follow [the playwright.dev README instructions to "run dev server"](https://github.com/microsoft/playwright.dev/#run-dev-server) to view your changes.
Headless execution is supported for all browsers on all platforms. Check out [system requirements](https://playwright.dev/docs/intro#system-requirements) for details.
@ -46,7 +46,6 @@ npx playwright install
You can optionally install only selected browsers, see [install browsers](https://playwright.dev/docs/cli#install-browsers) for more details. Or you can install no browsers at all and use existing [browser channels](https://playwright.dev/docs/browsers).
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet) and [Xamarin](https://github.com/xamarin).
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)) of a security vulnerability, please report it to us as described below.
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/security.md/definition), please report it to us as described below.
## Reporting Security Issues
**Please do not report security vulnerabilities through public GitHub issues.**
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://msrc.microsoft.com/create-report).
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/security.md/msrc/create-report).
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc).
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/security.md/msrc/pgp).
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc).
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc).
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
@ -28,7 +28,7 @@ Please include the requested information listed below (as much as you can provid
This information will help us triage your report more quickly.
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://microsoft.com/msrc/bounty) page for more details about our active programs.
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/security.md/msrc/bounty) page for more details about our active programs.
## Preferred Languages
@ -36,6 +36,6 @@ We prefer all communications to be in English.
## Policy
Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd).
Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/security.md/cvd).
This project uses GitHub issues to track bugs and feature requests. Please search the [existing issues][gh-issues] before filing new ones to avoid duplicates. For new issues, file your bug or feature request as a new issue using corresponding template.
For help and questions about using this project, please see the [docs site for Playwright][docs].
Join our community [Discord Server][discord-server] to connect with other developers using Playwright and ask questions in our 'help-playwright' forum.
## Microsoft Support Policy
Support for Playwright is limited to the resources listed above.
@ -14,8 +14,6 @@ A few examples of problems this can catch include:
The following examples rely on the [`com.deque.html.axe-core/playwright`](https://mvnrepository.com/artifact/com.deque.html.axe-core/playwright) Maven package which adds support for running the [axe accessibility testing engine](https://www.deque.com/axe/) as part of your Playwright tests.
<!-- TOC -->
## Disclaimer
Automated accessibility tests can detect some common accessibility problems such as missing or invalid properties. But many accessibility problems can only be discovered through manual testing. We recommend using a combination of automated testing, manual accessibility assessments, and inclusive user testing.
@ -72,22 +70,24 @@ For example, you can use [`AxeBuilder.include()`](https://github.com/dequelabs/a
`AxeBuilder.analyze()` will scan the page *in its current state* when you call it. To scan parts of a page that are revealed based on UI interactions, use [Locators](./locators.md) to interact with the page before invoking `analyze()`:
@ -135,7 +135,7 @@ If the element in question is used repeatedly in many pages, consider [using a t
### Disabling individual scan rules
If your application contains many different pre-existing violations of a specific rule, you can use [`AxeBuilder.disableRules()`](https://github.com/dequelabs/axe-core-maven-html/blob/develop/playwright/README.md#axebuilderdisablerulesliststring-rules) to temporarily disable individual rules until you're able to fix the issues.
If your application contains many different preexisting violations of a specific rule, you can use [`AxeBuilder.disableRules()`](https://github.com/dequelabs/axe-core-maven-html/blob/develop/playwright/README.md#axebuilderdisablerulesliststring-rules) to temporarily disable individual rules until you're able to fix the issues.
You can find the rule IDs to pass to `disableRules()` in the `id` property of the violations you want to suppress. A [complete list of axe's rules](https://github.com/dequelabs/axe-core/blob/master/doc/rule-descriptions.md) can be found in `axe-core`'s documentation.
@ -160,38 +160,40 @@ This approach avoids the downsides of using `AxeBuilder.exclude()` at the cost o
Here is an example of using fingerprints based on only rule IDs and "target" selectors pointing to each violation:
@ -14,7 +14,7 @@ A few examples of problems this can catch include:
The following examples rely on the [`@axe-core/playwright`](https://npmjs.org/@axe-core/playwright) package which adds support for running the [axe accessibility testing engine](https://www.deque.com/axe/) as part of your Playwright tests.
:::note Disclaimer
:::note[Disclaimer]
Automated accessibility tests can detect some common accessibility problems such as missing or invalid properties. But many accessibility problems can only be discovered through manual testing. We recommend using a combination of automated testing, manual accessibility assessments, and inclusive user testing.
For manual assessments, we recommend [Accessibility Insights for Web](https://accessibilityinsights.io/docs/web/overview/?referrer=playwright-accessibility-testing-js), a free and open source dev tool that walks you through assessing a website for [WCAG 2.1 AA](https://www.w3.org/WAI/WCAG21/quickref/?currentsidebar=%23col_customize&levels=aaa) coverage.
@ -147,7 +147,7 @@ If the element in question is used repeatedly in many pages, consider [using a t
### Disabling individual scan rules
If your application contains many different pre-existing violations of a specific rule, you can use [`AxeBuilder.disableRules()`](https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md#axebuilderdisablerulesrules-stringarray) to temporarily disable individual rules until you're able to fix the issues.
If your application contains many different preexisting violations of a specific rule, you can use [`AxeBuilder.disableRules()`](https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md#axebuilderdisablerulesrules-stringarray) to temporarily disable individual rules until you're able to fix the issues.
You can find the rule IDs to pass to `disableRules()` in the `id` property of the violations you want to suppress. A [complete list of axe's rules](https://github.com/dequelabs/axe-core/blob/master/doc/rule-descriptions.md) can be found in `axe-core`'s documentation.
@ -167,7 +167,7 @@ test('should not have any accessibility violations outside of rules with known i
### Using snapshots to allow specific known issues
If you would like to allow for a more granular set of known issues, you can use [Snapshots](./test-snapshots.md) to verify that a set of pre-existing violations has not changed. This approach avoids the downsides of using `AxeBuilder.exclude()` at the cost of slightly more complexity and fragility.
If you would like to allow for a more granular set of known issues, you can use [Snapshots](./test-snapshots.md) to verify that a set of preexisting violations has not changed. This approach avoids the downsides of using `AxeBuilder.exclude()` at the cost of slightly more complexity and fragility.
Do not use a snapshot of the entire `accessibilityScanResults.violations` array. It contains implementation details of the elements in question, such as a snippet of their rendered HTML; if you include these in your snapshots, it will make your tests prone to breaking every time one of the components in question changes for an unrelated reason:
Playwright performs a range of actionability checks on the elements before making actions to ensure these actions
behave as expected. It auto-waits for all the relevant checks to pass and only then performs the requested action. If the required checks do not pass within the given `timeout`, action fails with the `TimeoutError`.
For example, for [`method: Page.click`], Playwright will ensure that:
- element is [Attached] to the DOM
For example, for [`method: Locator.click`], Playwright will ensure that:
- locator resolves to exactly one element
- element is [Visible]
- element is [Stable], as in not animating or completed animation
- element [Receives Events], as in not obscured by other elements
@ -17,72 +17,75 @@ For example, for [`method: Page.click`], Playwright will ensure that:
Here is the complete list of actionability checks performed for each action:
Some actions like [`method: Page.click`] support `force` option that disables non-essential actionability checks,
for example passing truthy `force` to [`method: Page.click`] method will not check that the target element actually
Some actions like [`method: Locator.click`] support `force` option that disables non-essential actionability checks,
for example passing truthy `force` to [`method: Locator.click`] method will not check that the target element actually
receives click events.
## Assertions
You can check the actionability state of the element using one of the following methods as well. This is typically
not necessary, but it helps writing assertive tests that ensure that after certain actions, elements reach
actionable state:
Playwright includes auto-retrying assertions that remove flakiness by waiting until the condition is met, similarly to auto-waiting before actions.
- [`method: ElementHandle.isChecked`]
- [`method: ElementHandle.isDisabled`]
- [`method: ElementHandle.isEditable`]
- [`method: ElementHandle.isEnabled`]
- [`method: ElementHandle.isHidden`]
- [`method: ElementHandle.isVisible`]
- [`method: Page.isChecked`]
- [`method: Page.isDisabled`]
- [`method: Page.isEditable`]
- [`method: Page.isEnabled`]
- [`method: Page.isHidden`]
- [`method: Page.isVisible`]
- [`method: Locator.isChecked`]
- [`method: Locator.isDisabled`]
- [`method: Locator.isEditable`]
- [`method: Locator.isEnabled`]
- [`method: Locator.isHidden`]
- [`method: Locator.isVisible`]
| Assertion | Description |
| :- | :- |
| [`method: LocatorAssertions.toBeAttached`] | Element is attached |
| [`method: LocatorAssertions.toBeChecked`] | Checkbox is checked |
| [`method: LocatorAssertions.toBeDisabled`] | Element is disabled |
| [`method: LocatorAssertions.toBeEditable`] | Element is editable |
| [`method: LocatorAssertions.toBeEmpty`] | Container is empty |
| [`method: LocatorAssertions.toBeEnabled`] | Element is enabled |
| [`method: LocatorAssertions.toBeFocused`] | Element is focused |
| [`method: LocatorAssertions.toBeHidden`] | Element is not visible |
| [`method: LocatorAssertions.toBeInViewport`] | Element intersects viewport |
| [`method: LocatorAssertions.toBeVisible`] | Element is visible |
| [`method: LocatorAssertions.toContainText`] | Element contains text |
| [`method: LocatorAssertions.toHaveAttribute`] | Element has a DOM attribute |
| [`method: LocatorAssertions.toHaveClass`] | Element has a class property |
| [`method: LocatorAssertions.toHaveCount`] | List has exact number of children |
| [`method: LocatorAssertions.toHaveCSS`] | Element has CSS property |
| [`method: LocatorAssertions.toHaveId`] | Element has an ID |
| [`method: LocatorAssertions.toHaveJSProperty`] | Element has a JavaScript property |
| [`method: LocatorAssertions.toHaveText`] | Element matches text |
| [`method: LocatorAssertions.toHaveValue`] | Input has a value |
| [`method: LocatorAssertions.toHaveValues`] | Select has options selected |
| [`method: PageAssertions.toHaveTitle`] | Page has a title |
| [`method: PageAssertions.toHaveURL`] | Page has a URL |
| [`method: APIResponseAssertions.toBeOK`] | Response has an OK status |
<br/>
## Attached
Element is considered attached when it is [connected](https://developer.mozilla.org/en-US/docs/Web/API/Node/isConnected) to a Document or a ShadowRoot.
Learn more in the [assertions guide](./test-assertions.md).
## Visible
Element is considered visible when it has non-empty bounding box and does not have `visibility:hidden` computed style. Note that elements of zero size or with `display:none` are not considered visible.
Element is considered visible when it has non-empty bounding box and does not have `visibility:hidden` computed style.
Note that according to this definition:
* Elements of zero size **are not** considered visible.
* Elements with `display:none`**are not** considered visible.
* Elements with `opacity:0`**are** considered visible.
## Stable
@ -90,18 +93,27 @@ Element is considered stable when it has maintained the same bounding box for at
## Enabled
Element is considered enabled unless it is a `<button>`, `<select>`, `<input>` or `<textarea>` with a `disabled` property.
Element is considered enabled when it is **not disabled**.
Element is **disabled** when:
- it is a `<button>`, `<select>`, `<input>`, `<textarea>`, `<option>` or `<optgroup>` with a `[disabled]` attribute;
- it is a `<button>`, `<select>`, `<input>`, `<textarea>`, `<option>` or `<optgroup>` that is a part of a `<fieldset>` with a `[disabled]` attribute;
- it is a descendant of an element with `[aria-disabled=true]` attribute.
## Editable
Element is considered editable when it is [enabled] and does not have `readonly` property set.
Element is considered editable when it is [enabled] and is **not readonly**.
Element is **readonly** when:
- it is a `<select>`, `<input>` or `<textarea>` with a `[readonly]` attribute;
- it has an `[aria-readonly=true]` attribute and an aria role that [supports it](https://w3c.github.io/aria/#aria-readonly).
## Receives Events
Element is considered receiving pointer events when it is the hit target of the pointer event at the action point. For example, when clicking at the point `(10;10)`, Playwright checks whether some other element (usually an overlay) will instead capture the click at `(10;10)`.
For example, consider a scenario where Playwright will click `Sign Up` button regardless of when the [`method: Page.click`] call was made:
For example, consider a scenario where Playwright will click `Sign Up` button regardless of when the [`method: Locator.click`] call was made:
- page is checking that user name is unique and `Sign Up` button is disabled;
- after checking with the server, the disabled `Sign Up` button is replaced with another one that is now enabled.
@ -110,4 +122,3 @@ For example, consider a scenario where Playwright will click `Sign Up` button re
@ -16,9 +16,7 @@ A few examples where it may come in handy:
All of that could be achieved via [APIRequestContext] methods.
The following examples rely on the [`Microsoft.Playwright.NUnit`](./test-runners.md) package which creates a Playwright and Page instance for each test.
<!-- TOC -->
The following examples rely on the [`Microsoft.Playwright.MSTest`](./test-runners.md) package which creates a Playwright and Page instance for each test.
## Writing API Test
@ -34,22 +32,19 @@ The following example demonstrates how to use Playwright to test issues creation
GitHub API requires authorization, so we'll configure the token once for all tests. While at it, we'll also set the `baseURL` to simplify the tests.
@ -167,41 +161,47 @@ public class TestGitHubAPI : PlaywrightTest
These tests assume that repository exists. You probably want to create a new one before running tests and delete it afterwards. Use `[SetUp]` and `[TearDown]` hooks for that.
```csharp
using System.Text.Json;
using Microsoft.Playwright;
using Microsoft.Playwright.MSTest;
namespace PlaywrightTests;
[TestClass]
public class TestGitHubAPI : PlaywrightTest
{
// ...
// ...
[TestInitialize]
public async Task SetUpAPITesting()
{
await CreateAPIRequestContext();
await CreateTestRepository();
}
[SetUp]
public async Task SetUpAPITesting()
{
await CreateAPIRequestContext();
await CreateTestRepository();
}
private async Task CreateTestRepository()
{
var resp = await Request.PostAsync("/user/repos", new()
{
DataObject = new Dictionary<string,string>()
{
["name"] = REPO,
},
});
await Expect(resp).ToBeOKAsync();
}
private async Task CreateTestRepository()
{
var resp = await Request.PostAsync("/user/repos", new()
{
DataObject = new Dictionary<string,string>()
{
["name"] = REPO,
},
});
Assert.True(resp.Ok);
}
[TestCleanup]
public async Task TearDownAPITesting()
{
await DeleteTestRepository();
await Request.DisposeAsync();
}
[TearDown]
public async Task TearDownAPITesting()
{
await DeleteTestRepository();
await Request.DisposeAsync();
}
private async Task DeleteTestRepository()
{
var resp = await Request.DeleteAsync("/repos/" + USER + "/" + REPO);
Assert.True(resp.Ok);
}
private async Task DeleteTestRepository()
{
var resp = await Request.DeleteAsync("/repos/" + USER + "/" + REPO);
await Expect(resp).ToBeOKAsync();
}
}
```
@ -210,36 +210,34 @@ public class TestGitHubAPI : PlaywrightTest
Here is the complete example of an API test:
```csharp
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Text.Json;
using Microsoft.Playwright.NUnit;
using Microsoft.Playwright;
using NUnit.Framework;
using Microsoft.Playwright.MSTest;
namespace PlaywrightTests;
[TestFixture]
[TestClass]
public class TestGitHubAPI : PlaywrightTest
{
static string REPO = "test-repo-2";
static string USER = Environment.GetEnvironmentVariable("GITHUB_USER");
@ -16,8 +16,6 @@ A few examples where it may come in handy:
All of that could be achieved via [APIRequestContext] methods.
<!-- TOC -->
## Writing API Test
[APIRequestContext] can send all kinds of HTTP(S) requests over network.
@ -196,6 +194,7 @@ public class TestGitHubAPI {
These tests assume that repository exists. You probably want to create a new one before running tests and delete it afterwards. Use `@BeforeAll` and `@AfterAll` hooks for that.
```java
public class TestGitHubAPI {
// ...
void createTestRepository() {
@ -225,6 +224,7 @@ These tests assume that repository exists. You probably want to create a new one
disposeAPIRequestContext();
closePlaywright();
}
}
```
### Complete test example
@ -375,24 +375,28 @@ public class TestGitHubAPI {
}
```
See experimental [JUnit integration](./junit.md) to automatically initialize Playwright objects and more.
## Prepare server state via API calls
The following test creates a new issue via API and then navigates to the list of all issues in the
project to check that it appears at the top of the list. The check is performed using [LocatorAssertions].
@ -16,8 +16,6 @@ A few examples where it may come in handy:
All of that could be achieved via [APIRequestContext] methods.
<!-- TOC3 -->
## Writing API Test
[APIRequestContext] can send all kinds of HTTP(S) requests over network.
@ -48,6 +46,24 @@ export default defineConfig({
});
```
**Proxy configuration**
If your tests need to run behind a proxy, you can specify this in the config and the `request` fixture
will pick it up automatically:
```js title="playwright.config.ts"
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
proxy: {
server: 'http://my-proxy:8080',
username: 'user',
password: 'secret'
},
}
});
```
### Writing tests
Playwright Test comes with the built-in `request` fixture that respects configuration options like `baseURL` or `extraHTTPHeaders` we specified and is ready to send some requests.
@ -313,7 +329,7 @@ test('context request will share cookie storage with its browser context', async
Note that since you don't need Playwright to install web browsers when testing Android, you can omit browser download via setting the following environment variable when installing Playwright:
```bash js
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm i -D playwright
```
## async method: Android.connect
* since: v1.28
- returns: <[AndroidDevice]>
@ -208,6 +202,12 @@ Prevents automatic playwright driver installation on attach. Assumes that the dr
Optional device serial number to launch the browser on. If not specified, it will
throw if multiple devices are connected.
### option: Android.launchServer.host
* since: v1.45
- `host`<[string]>
Host to use for the web socket. It is optional and if it is omitted, the server will accept connections on the unspecified IPv6 address (::) when IPv6 is available, or the unspecified IPv4 address (0.0.0.0) otherwise. Consider hardening it with picking a specific interface.
@ -136,7 +136,7 @@ Launches Chrome browser on the device, and returns its persistent context.
### option: AndroidDevice.launchBrowser.pkg
* since: v1.9
- `command` <[string]>
- `pkg` <[string]>
Optional package name to launch instead of default Chrome for Android.
@ -177,7 +177,9 @@ Launches a process in the shell on the device and returns a socket to communicat
### param: AndroidDevice.open.command
* since: v1.9
- `command`<[string]> Shell command to execute.
- `command`<[string]>
Shell command to execute.
## async method: AndroidDevice.pinchClose
* since: v1.9
@ -445,7 +447,7 @@ Either a predicate that receives an event or an options object. Optional.
* since: v1.9
- returns: <[AndroidWebView]>
This method waits until [AndroidWebView] matching the [`option: selector`] is opened and returns it. If there is already an open [AndroidWebView] matching the [`option: selector`], returns immediately.
This method waits until [AndroidWebView] matching the [`param: selector`] is opened and returns it. If there is already an open [AndroidWebView] matching the [`param: selector`], returns immediately.
All responses returned by [`method: APIRequestContext.get`] and similar methods are stored in the memory, so that you can later call [`method: APIResponse.body`]. This method
discards all stored responses, and makes [`method: APIResponse.body`] throw "Response disposed" error.
All responses returned by [`method: APIRequestContext.get`] and similar methods are stored in the memory, so that you can later call [`method: APIResponse.body`].This method discards all its resources, calling any method on disposed [APIRequestContext] will throw an exception.
### option: APIRequestContext.dispose.reason
* since: v1.45
- `reason`<[string]>
The reason to be reported to the operations interrupted by the context disposal.
## async method: APIRequestContext.fetch
* since: v1.16
- returns: <[APIResponse]>
Sends HTTP(S) request and returns its response. The method will populate request cookies from the context and update
context cookies from the response. The method will automatically follow redirects. JSON objects can be passed directly to the request.
context cookies from the response. The method will automatically follow redirects.
**Usage**
JSON objects can be passed directly to the request:
The common way to send file(s) in the body of a request is to encode it as form fields with `multipart/form-data` encoding. You can achieve that with Playwright API like this:
The common way to send file(s) in the body of a request is to upload them as form fields with `multipart/form-data` encoding, by specifiying the `multipart` parameter:
```js
// Open file as a stream and pass it to the request:
await request.PostAsync("https://example.com/api/findBook", new() { Form = formData });
```
The common way to send file(s) in the body of a request is to upload them as form fields with `multipart/form-data` encoding. You can achieve that with Playwright API like this:
The common way to send file(s) in the body of a request is to upload them as form fields with `multipart/form-data` encoding. Use [FormData] to construct request body and pass it to the request as `multipart` parameter:
```js
// Open file as a stream and pass it to the request:
@ -97,11 +96,17 @@ In case this browser is connected to, clears all created contexts belonging to t
browser server.
:::note
This is similar to force quitting the browser. Therefore, you should call [`method: BrowserContext.close`] on any [BrowserContext]'s you explicitly created earlier with [`method: Browser.newContext`] **before** calling [`method: Browser.close`].
This is similar to force-quitting the browser. To close pages gracefully and ensure you receive page close events, call [`method: BrowserContext.close`] on any [BrowserContext] instances you explicitly created earlier using [`method: Browser.newContext`] **before** calling [`method: Browser.close`].
:::
The [Browser] object itself is considered to be disposed and cannot be used anymore.
### option: Browser.close.reason
* since: v1.40
- `reason`<[string]>
The reason to be reported to the operations interrupted by the browser closure.
## method: Browser.contexts
* since: v1.8
- returns: <[Array]<[BrowserContext]>>
@ -127,16 +132,16 @@ System.out.println(browser.contexts().size()); // prints "1"
```python async
browser = await pw.webkit.launch()
print(len(browser.contexts())) # prints `0`
print(len(browser.contexts)) # prints `0`
context = await browser.new_context()
print(len(browser.contexts())) # prints `1`
print(len(browser.contexts)) # prints `1`
```
```python sync
browser = pw.webkit.launch()
print(len(browser.contexts())) # prints `0`
print(len(browser.contexts)) # prints `0`
context = browser.new_context()
print(len(browser.contexts())) # prints `1`
print(len(browser.contexts)) # prints `1`
```
```csharp
@ -197,7 +202,7 @@ Browser browser = playwright.firefox().launch(); // Or 'chromium' or 'webkit'.
Playwright has ability to mock clock and passage of time.
## event: BrowserContext.close
* since: v1.8
- argument: <[BrowserContext]>
@ -94,13 +112,19 @@ Emitted when Browser context gets closed. This might happen because of one of th
* Browser application is closed or crashed.
* The [`method: Browser.close`] method was called.
### option: BrowserContext.close.reason
* since: v1.40
- `reason`<[string]>
The reason to be reported to the operations interrupted by the context closure.
## event: BrowserContext.console
* since: v1.34
* langs:
- alias-java: consoleMessage
- argument: <[ConsoleMessage]>
Emitted when JavaScript within the page calls one of console API methods, e.g. `console.log` or `console.dir`. Also emitted if the page throws an error or a warning.
Emitted when JavaScript within the page calls one of console API methods, e.g. `console.log` or `console.dir`.
The arguments passed into `console.log` and the page are available on the [ConsoleMessage] event handler argument.
@ -196,7 +223,7 @@ also fire for popup pages. See also [`event: Page.popup`] to receive events abou
The earliest moment that page is available is when it has navigated to the initial url. For example, when opening a
popup with `window.open('http://example.com')`, this event will fire when the network request to "http://example.com" is
done and its response has started loading in the popup.
done and its response has started loading in the popup. If you would like to route/listen to this network request, use [`method: BrowserContext.route`] and [`event: BrowserContext.request`] respectively instead of similar methods on the [Page].
- `url` ?<[string]> either url or domain / path are required. Optional.
- `domain` ?<[string]> either url or domain / path are required Optional.
- `path` ?<[string]> either url or domain / path are required Optional.
- `url` ?<[string]> Either url or domain / path are required. Optional.
- `domain` ?<[string]> For the cookie to apply to all subdomains as well, prefix domain with a dot, like this: ".example.com". Either url or domain / path are required. Optional.
- `path` ?<[string]> Either url or domain / path are required Optional.
- `expires` ?<[float]> Unix time in seconds. Optional.
A permission or an array of permissions to grant. Permissions can be one of the following values:
* `'geolocation'`
* `'midi'`
* `'midi-sysex'` (system-exclusive midi)
* `'notifications'`
* `'camera'`
* `'microphone'`
* `'background-sync'`
* `'ambient-light-sensor'`
A list of permissions to grant.
:::danger
Supported permissions differ between browsers, and even between different versions of the same browser. Any permission may stop working after an update.
:::
Here are some permissions that may be supported by some browsers:
It is possible to examine the request to decide the route action. For example, mocking all requests that contain some post data, and leaving all other requests as is:
```js
await context.route('/api/**', route => {
await context.route('/api/**', async route => {
if (route.request().postData().includes('my-string'))
A glob pattern, regex pattern or predicate receiving [URL] to match while routing.
When a [`option: baseURL`] via the context options was provided and the passed URL is a path,
When a [`option: Browser.newContext.baseURL`] via the context options was provided and the passed URL is a path,
it gets merged via the [`new URL()`](https://developer.mozilla.org/en-US/docs/Web/API/URL/URL) constructor.
### param: BrowserContext.route.handler
@ -1241,6 +1272,99 @@ When set to `minimal`, only record information necessary for routing from HAR. T
Optional setting to control resource content management. If `attach` is specified, resources are persisted as separate files or entries in the ZIP archive. If `embed` is specified, content is stored inline the HAR file.
## async method: BrowserContext.routeWebSocket
* since: v1.48
This method allows to modify websocket connections that are made by any page in the browser context.
Note that only `WebSocket`s created after this method was called will be routed. It is recommended to call this method before creating any pages.
**Usage**
Below is an example of a simple handler that blocks some websocket messages.
See [WebSocketRoute] for more details and examples.
Only WebSockets with the url matching this pattern will be routed. A string pattern can be relative to the [`option: Browser.newContext.baseURL`] context option.
Browser websocket endpoint which can be used as an argument to [`method: BrowserType.connect`] to establish connection
to the browser.
Note that if the listen `host` option in `launchServer` options is not specified, localhost will be output anyway, even if the actual listening address is an unspecified address.
This method attaches Playwright to an existing browser instance. When connecting to another browser launched via `BrowserType.launchServer` in Node.js, the major and minor version needs to match the client version (1.2.3 → is compatible with 1.2.x).
This method attaches Playwright to an existing browser instance created via `BrowserType.launchServer` in Node.js.
:::note
The major and minor version of the Playwright instance that connects needs to match the version of Playwright that launches the browser (1.2.3 → is compatible with 1.2.x).
:::
### param: BrowserType.connect.wsEndpoint
* since: v1.10
- `wsEndpoint`<[string]>
A browser websocket endpoint to connect to.
A Playwright browser websocket endpoint to connect to. You obtain this endpoint via `BrowserServer.wsEndpoint`.
### option: BrowserType.connect.headers
* since: v1.11
@ -152,6 +156,10 @@ The default browser context is accessible via [`method: Browser.contexts`].
Connecting over the Chrome DevTools Protocol is only supported for Chromium-based browsers.
:::
:::note
This connection is significantly lower fidelity than the Playwright protocol connection via [`method: BrowserType.connect`]. If you are experiencing issues or attempting to use advanced functionality, you probably want to use [`method: BrowserType.connect`].
:::
**Usage**
```js
@ -337,6 +345,15 @@ use a temporary directory instead.
Host to use for the web socket. It is optional and if it is omitted, the server will accept connections on the unspecified IPv6 address (::) when IPv6 is available, or the unspecified IPv4 address (0.0.0.0) otherwise. Consider hardening it with picking a specific interface.
Accurately simulating time-dependent behavior is essential for verifying the correctness of applications. Learn more about [clock emulation](../clock.md).
Note that clock is installed for the entire [BrowserContext], so the time
in all the pages and iframes is controlled by the same clock.
## async method: Clock.fastForward
* since: v1.45
Advance the clock by jumping forward in time. Only fires due timers at most once. This is equivalent to user closing the laptop lid for a while and
reopening it later, after given time.
**Usage**
```js
await page.clock.fastForward(1000);
await page.clock.fastForward('30:00');
```
```python async
await page.clock.fast_forward(1000)
await page.clock.fast_forward("30:00")
```
```python sync
page.clock.fast_forward(1000)
page.clock.fast_forward("30:00")
```
```java
page.clock().fastForward(1000);
page.clock().fastForward("30:00");
```
```csharp
await page.Clock.FastForwardAsync(1000);
await page.Clock.FastForwardAsync("30:00");
```
### param: Clock.fastForward.ticks
* since: v1.45
- `ticks`<[long]|[string]>
Time may be the number of milliseconds to advance the clock by or a human-readable string. Valid string formats are "08" for eight seconds, "01:00" for one minute and "02:34:10" for two hours, 34 minutes and ten seconds.
## async method: Clock.install
* since: v1.45
Install fake implementations for the following time-related functions:
* `Date`
* `setTimeout`
* `clearTimeout`
* `setInterval`
* `clearInterval`
* `requestAnimationFrame`
* `cancelAnimationFrame`
* `requestIdleCallback`
* `cancelIdleCallback`
* `performance`
Fake timers are used to manually control the flow of time in tests. They allow you to advance time, fire timers, and control the behavior of time-dependent functions. See [`method: Clock.runFor`] and [`method: Clock.fastForward`] for more information.
### option: Clock.install.time
* langs: js, java
* since: v1.45
- `time`<[long]|[string]|[Date]>
Time to initialize with, current system time by default.
### option: Clock.install.time
* langs: python
* since: v1.45
- `time`<[float]|[string]|[Date]>
Time to initialize with, current system time by default.
### option: Clock.install.time
* langs: csharp
* since: v1.45
- `time`<[string]|[Date]>
Time to initialize with, current system time by default.
## async method: Clock.runFor
* since: v1.45
Advance the clock, firing all the time-related callbacks.
**Usage**
```js
await page.clock.runFor(1000);
await page.clock.runFor('30:00');
```
```python async
await page.clock.run_for(1000);
await page.clock.run_for("30:00")
```
```python sync
page.clock.run_for(1000);
page.clock.run_for("30:00")
```
```java
page.clock().runFor(1000);
page.clock().runFor("30:00");
```
```csharp
await page.Clock.RunForAsync(1000);
await page.Clock.RunForAsync("30:00");
```
### param: Clock.runFor.ticks
* since: v1.45
- `ticks`<[long]|[string]>
Time may be the number of milliseconds to advance the clock by or a human-readable string. Valid string formats are "08" for eight seconds, "01:00" for one minute and "02:34:10" for two hours, 34 minutes and ten seconds.
## async method: Clock.pauseAt
* since: v1.45
Advance the clock by jumping forward in time and pause the time. Once this method is called, no timers
are fired unless [`method: Clock.runFor`], [`method: Clock.fastForward`], [`method: Clock.pauseAt`] or [`method: Clock.resume`] is called.
Only fires due timers at most once.
This is equivalent to user closing the laptop lid for a while and reopening it at the specified time and
For best results, install the clock before navigating the page and set it to a time slightly before the intended test time. This ensures that all timers run normally during page loading, preventing the page from getting stuck. Once the page has fully loaded, you can safely use [`method: Clock.pauseAt`] to pause the clock.
```js
// Initialize clock with some time before the test time and let the page load
// naturally. `Date.now` will progress as the timers fire.
await page.clock.install({ time: new Date('2024-12-10T08:00:00') });
Resumes timers. Once this method is called, time resumes flowing, timers are fired as usual.
## async method: Clock.setFixedTime
* since: v1.45
Makes `Date.now` and `new Date()` return fixed fake time at all times,
keeps all the timers running.
Use this method for simple scenarios where you only need to test with a predefined time. For more advanced scenarios, use [`method: Clock.install`] instead. Read docs on [clock emulation](../clock.md) to learn more.
Sets system time, but does not trigger any timers. Use this to test how the web page reacts to a time shift, for example switching from summer to winter time, or changing time zones.
@ -70,9 +70,9 @@ Upon successful cancellations, `download.failure()` would resolve to `'canceled'
## async method: Download.createReadStream
* since: v1.8
* langs: java, js, csharp
- returns: <[null]|[Readable]>
- returns: <[Readable]>
Returns readable stream for current download or `null` if download failed.
Returns a readable stream for a successful download, or throws for a failed/canceled download.
## async method: Download.delete
* since: v1.8
@ -93,10 +93,9 @@ Get the page that the download belongs to.
## async method: Download.path
* since: v1.8
- returns: <[null]|[path]>
- returns: <[path]>
Returns path to the downloaded file in case of successful download. The method will
wait for the download to finish if necessary. The method throws when connected remotely.
Returns path to the downloaded file for a successful download, or throws for a failed/canceled download. The method will wait for the download to finish if necessary. The method throws when connected remotely.
Note that the download's file name is a random GUID, use [`method: Download.suggestedFilename`]
Note that since you don't need Playwright to install web browsers when testing Electron, you can omit browser download via setting the following environment variable when installing Playwright:
```bash js
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 npm i -D playwright
```
**Supported Electron versions are:**
* v12.2.0+
* v13.4.0+
Some files were not shown because too many files have changed in this diff
Show more
