Fixes https://github.com/microsoft/playwright/issues/30160
### Description:
This pull request introduces the ability to specify custom locations for
test steps in Playwright. By enabling the provision of arbitrary
locations to the test.step method, it resolves the limitation where
helper methods obfuscate the original call site, providing more accurate
and meaningful location data in test reports.
### Motivation:
To enhance the utility and clarity of test reports in Playwright.
Specifically, it addresses the need to trace test steps back to their
precise location in the code, which is especially important when steps
are abstracted in helper functions. This feature is crucial for
maintaining accurate documentation and facilitating debugging processes.
### Changes:
Added functionality to pass a custom location object to test.step.
### Expected Outcome:
This PR is expected to significantly improve the precision and
usefulness of diagnostic data in test reports by allowing specific
locations within helper functions to be accurately documented. It
facilitates better tracking of test executions and simplifies the
debugging process, making it easier for developers to understand and
address issues within complex tests.
### References:
Closes https://github.com/microsoft/playwright/issues/30160 -
"[Feature]: allow to pass arbitrary location to test.step"
**Code Check**
I conducted tests on this new feature by integrating it into some
existing test codes, and it worked well. I will attach the code used for
testing and a screenshot showing the successful outcome.
<details>
<summary>toggle dropdown</summary>
<div markdown="1">
```
import type { Location } from '../../../packages/playwright/types/testReporter'
...
test('should respect the back button', async ({ page }) => {
await page.locator('.todo-list li .toggle').nth(1).check();
await checkNumberOfCompletedTodosInLocalStorage(page, 1);
...
await test.step('Showing active items', async () => {
await page.getByRole('link', { name: 'Active' }).click();
}, {location});
```
<img width="1109" alt="image"
src="https://github.com/user-attachments/assets/359feafa-0949-4c71-9426-46debef21bdd">
</div>
</details>
Related to https://github.com/microsoft/playwright/issues/19621
Adds some instrumentation to collect timings for `APIRequestContext`
requests and adds them to the HAR trace. Doesn't yet expose them via an
API, but makes our `Duration` field in the trace viewer show a nice
duration:
<img width="1392" alt="Screenshot 2024-09-14 at 11 46 04"
src="https://github.com/user-attachments/assets/8020382d-9494-4634-9cfd-22b6f4a1d770">
I'm gonna add it to our API in a separate PR.
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Closes https://github.com/microsoft/playwright/issues/32578.
Adds a buffer mode that can be toggled by pressing <kbd>b</kbd>. When
engaged, changed test files are collected and shown on screen. The test
run is then kicked off by pressing <kbd>Enter</kbd>.
This changes the signal to start a test run from <kbd>Cmd+s</kbd> to a
<kbd>Enter</kbd> press in the test terminal. It should help users with
auto-save and make it easier to run on long-running tests. It feels very
similar to running `npx playwright test`, but without having to write a
filter.
https://github.com/user-attachments/assets/71e16139-9427-4e90-b523-8d218f09ed9d
Closes https://github.com/microsoft/playwright/issues/32632. A side
effect of Remix's hydration implementation is that it throws away the
entire DOM. This is broadly discussed in
https://github.com/remix-run/remix/issues/4822 - there might be a fix in
coming React versions, but who knows.
Besides breaking browser extensions, this also deletes our toolbar!
This PR fixes it by periodically checking in on `x-pw-glass`, and
remounting it if it was unmounted. Hacky but effective!
Adds a copy-to-clipboard button for each annotation so that text can be
copied easily.
This re-uses the existing `CopyToClipboard` component and adds a `small`
variant that can be used inline. The icon size and colour have been
chosen to avoid being overwhelming when used inline.
Related to #30141
I opted not to introduce the hover behaviour from #30749 as it's less
discoverable, but can understand why that might be favourable. Certainly
open to suggestions 😄
<img width="379" alt="Screenshot 2024-07-22 at 3 23 53 PM"
src="https://github.com/user-attachments/assets/3b9998cf-2e8d-40c9-9c8a-64eab3a9ed2e">
Changes `expect.extend` behaviour so that it doesn't mutate the global
instance and behaves closer to what users expect. This is formally a
breaking change, and I had to remove a test that asserts the breaking
behaviour.
TODO:
- [x] decide wether this is a separate method or a flag for
`expect.extend`
- [x] figure out if we need to change docs
What was happening?
- When we use CT, we go over the test files, look at the imports using
`tsxTransform.ts` and store them inside a map, these we feed into the
import registry which we build using Vite and have access inside the
browser
- In case of an inline component in the same file as where the test file
is, this is not happening.
- jsx-runtime via babel kicks in, transforms every JSX component in
something like that:
```
{
__pw_type: 'jsx',
type: [Function: MyInlineComponent],
props: { value: 'Max' },
key: undefined
}
```
this then gets passed into `wrapObject` which maps any function from the
Node.js side into expose function calls so they work inside the browser.
The assumption for `wrapObject` was to do it mostly for callbacks. So it
does for `type` - which is actually our component. We then pass this to
the React render function, which calls back the exposed function but we
never return anything, so it mounts `undefined`.
---
While there have been experiments from certain vendors to get the
'client only' code inside a server side file, we should throw for now to
not confuse users. We might revisit this in the future since Babel / TSX
doesn't support it outside of the box.
Fixes https://github.com/microsoft/playwright/issues/32167
We now hopefully align with `moduleResolution: bundler` tsconfig option,
allowing directory imports in every scenario, and allowing proper module
imports when not going through the type mapping.
This regressed in #32078. Fixes#32480, fixes#31811.
Those were just workarounds for browser-specific bugs, they should be
fixed upstream.
* individual mouse down/up/down/up events don't trigger dblclick event
in Firefox
* setContent throws when document.open/write is called in the utility
context in Firefox
Closes https://github.com/microsoft/playwright/issues/23964.
Trace snapshots are a best-effort snapshots of the browser DOM, but we
can't guarantee them to be exactly what the browser showed. One example
of this is `canvas` elements, where you just can't see their contents.
That makes snapshots useful, but not perfect.
For those cases where the snapshot doesn't show everything, this PR
introduces a new setting to show a screenshot instead. You won't be able
to scroll or inspect the DOM or select a locator anymore. But if the
snapshot was missing something, or displaying something wrong, you can
now check the screenshot instead.
Instead of plumbing it through a custom unspecified config field, make
it a part of plugin interface.
Additionally, use task runner for starting/stopping dev server.
This pull request introduces initial support for the WebDriver BiDi
protocol in Playwright. The primary goal of this PR is not to fully
implement BiDi but to experiment with the current state of the
specification and its implementation. We aim to identify the biggest
gaps and challenges that need to be addressed before considering BiDi as
the main protocol for Playwright.
In the `visit` method, we currently cache the rendered HTML for every
walked node. This re-use works well for traces that consist mostly of
references to earlier snapshots.
But for traces that don't share much, this is a large memory overhead
and leads to the memory crash documented in
https://github.com/microsoft/playwright/issues/32336. For the algocracks
amongst you, the current memory usage for an html tree $h$ is
$\mathcal{O}(|h| * \text{height}(h))$.
This PR removes that cache from the nodes and replaces it with a
snapshot-level cache, fixing the memory crash.
Traces *without* reference should not see a performance impact from
this.
Traces *with* references will have slower initial rendering, but
re-rendering maintains speed because of the snapshot-level cache.
Closes https://github.com/microsoft/playwright/issues/32336
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Max Schmitt <max@schmitt.mx>
After API review we decided to revert it:
* VSCode extension and UI mode users already get the (runtime) error if
the tag is not prefixed
* The typescript error message is not very nice
* The type change would break those clients that generate tests with
tags passed as string
This reverts commit 90e7b9ebac.
Closes https://github.com/microsoft/playwright/issues/32076.
This PR rewrites `watchMode.ts` to use `TestServer` under the hood. It's
essentially a complete rewrite, so don't pay too much attention on the
old implementation. Note that there's no changes to tests, so all
behaviour we have specced out there still works.
To make this work without a superfluous WebSocket connection, I had to
refactor `TestServerConnection` a little. Originally, I pulled this into
a [separate PR](https://github.com/microsoft/playwright/pull/32132), but
then realised how small the refactoring is. So it's in this PR now. Let
me know if you'd like to land it separately.
The `'should support custom matchers'` test asserts that the
functionality works, but it was a type error. This PR updates the types
so that it's allowed.
Closes https://github.com/microsoft/playwright/issues/32408
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Our CI operates on shallow clones. In vcs.ts, we perform a check for
shallow clones in `process.cwd()` instead of the test directory. This
makes the test in
3c208aeeff/tests/playwright-test/only-changed.spec.ts (L201)
failing in CI, but only for PRs. The fix is to perform the check on. the
test directory.
Closes https://github.com/microsoft/playwright/issues/32331
We're already passing the `outputDir` param to the UI, but the UI isn't
passing it back to the TestServer. This PR fixes that. I've added it to
`listTests`, which is requires to that
`TestServerDispatcher#_ignoredProjectOutputs` is populated with the
correct output dir. And i've added it to `runGlobalSetup`, which is what
the bug report was about.
Closes https://github.com/microsoft/playwright/issues/32256
We were expecting all errors to be of type `ExpectError`, but apparently
`expect` propagates rejections in the polling functions right through.
So we also need to handle that case.
I wonder if we have more cases of this. Would it make sense to enable
`useUnknownInCatchVariables` in TypeScript?
Closes https://github.com/microsoft/playwright/issues/32180
I was briefly wondering if we should output a log line a la "no tests
found", but my understanding is that that's the reporters job - so I
didn't change anything in that regard.
Closes https://github.com/microsoft/playwright/issues/32159. I
originally set out to enable Strict Mode for our React UI, but found a
way better thing: Enabling the lint rules we had already installed!
`eslint-plugin-react` is already in of our `package.json`, and this PR
enables it and fixes some of the reported issues. Most of them are
around the `key` prop which is mostly about performance, but there's
also fixes for misspelled `data-testid` props.
`x-unknown` is used as a placeholder for "no content-type" in the har.
We should not send it to the browser, because it is meaningfully
different from not sending `Content-Type` header. For example, Chromium
refuses to interpret stylesheets served with `x-unknown` content type.
Fixes https://github.com/microsoft/playwright-java/issues/1651.
partial fix for:
https://github.com/microsoft/playwright/issues/31927#issuecomment-2267065378
The options object wasn't treated as partial, unlike in other
frameworks, which led to the `component.update({ props: {} })` type
being selected instead the `component.update(<Component prop={} />)`
during jsx usage.
When two fixtures have different time slots, timeout in the first one
should not prevent the second one from tearing down.
Similarly, timeout in afterEach hook should not prevent fixture
teardowns.
This allows any time slot that has a legitimate timeout of zero to be
updated later on. See test for an example.
Previously, setting timeout to zero at any moment was considered a
"debug mode" and any subsequent timeouts were ignored.
Follow-up to https://github.com/microsoft/playwright/pull/32120
I made some changes suggested by @yury-s in the previous PR that make a
lot of sense:
- added an example to the documentation
- improved tests
- check params on the client and server end
- reverted to non-English characters being used as params
Closes https://github.com/microsoft/playwright/issues/32070. We were
applying `additionalFileMatcher` not just to `filteredProjectSuites`,
but also to `projectSuites`. `projectSuites` is where we take dependency
projects from, though - so `--only-changed` led to empty dependency
projects, resulting in the reported bug.
The fix is to only apply `additionalFileMatcher` on
`filteredProjectSuites`.
Supercedes https://github.com/microsoft/playwright/pull/31915, closes
https://github.com/microsoft/playwright/issues/31811.
When TypeScript resolves a specifier via path mapping, it does not
interpret `package.json`. If path mapping resolves to a directory, it
only looks at the `index.js` file in that directory if it's in CommonJS
mode.
We need to mirror this in our `esmLoader.ts`.
---------
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Also considered an alternative to not perform the locator handler check
during one-shot, but that would be somewhat against the promise of the
locator handler that is supposed to run **before** every expect check.
Fixes#32089.
Closes https://github.com/microsoft/playwright/issues/32050
When keyboarding through the action view, the UI continues showing the
hovered action. This makes keyboard nav hard to use.
The fix is to reset the higlighted action on keyboard navigation. This
is what we do when the mouse pointer leaves an action, and what I think
is reasonable.
Add Linux Mint 22 (Ubuntu based distro) version detection for dependency
installation
Linux Mint 22.x -> Ubuntu 24.04
(also see original Linux Mint support PR #28085)
Introduce `--tsconfig` to specify a single config to be used for all
imported files, instead of looking up tsconfig for each file separately.
Fixes#12829.
Update the documentation to match actual behavior.
The actual behavior today:
* Default mode is `full` when `recordHar` is passed to
`browser.newContext`
* Default mode is `minimal` when calling `context.routeFromHAR` and
`page.routeFromHAR`
Reference https://github.com/microsoft/playwright/issues/31983
Ideally we generate the timestamp when the Event gets created. This
patch adds a best-effort logic, since we can't override the constructor
of natively created events, e.g. `MouseEvent`.
Fixes https://github.com/microsoft/playwright/issues/31924
* Hide 'Status Code:' field for interrupted requests that don't have it.
* Clear up previously selected body when showing aborted requests.
* Highlight interrupted requests in red.
We populate `localStorage` using an init script. Currently, this script
isn't just run for the UI though, but also for all iframes. So we're
resetting `localStorage` every time the UI loads an iframe.
This hasn't been a problem in the past, because the only consumer of
`localStorage`, `Settings`, only read from `localStorage` once and kept
most state in `useState` afterwards. With
https://github.com/microsoft/playwright/pull/31911, this is no longer
true, so the bug starts biting us!
The fix is to ensure the init script isn't run on iframes.
This allows a dynamic import of a TS file to be processed by Babel.
For some reason, Playwright used to revert the CJS transforms. However,
ESM loader and transforms are always active, so CJS should be too.
When two attachments have the same content sha1, we used the first one's
name for the downloaded file, no matter which one the user clicked to
download. Now we pass the name explicitly.
References #31912.
Pulled out from https://github.com/microsoft/playwright/pull/31900
I stumbled over `React.Children`, because it's the first time I saw that
used. https://react.dev/reference/react/Children lists `React.Children`
it as "Legacy" and mentions it's uncommon. Also, the fact that SplitView
only displays its first two children, and all others are silently
discarded, can be a surprise to some.
By separating things out into `sidebar` and `main`, not only do we give
the two elements names (otherwise one needs to remember that sidebar is
always the first child), but we also prevent any "third children" from
being dropped.
Addresses https://github.com/microsoft/playwright/issues/31863. This PR
is chonky, but the individual commits should be easy to review. If
they're not, i'm happy to break them out into individual PRs.
There's two main things this does:
1. Remove some unused imports
2. Add a `clsx`-inspired helper function for classname templating
I wasn't able to replace `ReactDOM.render` with `ReactDOM.createRoot`.
This is the new recommended way starting with React 18, and the existing
one is going to be deprecated at some point. But it somehow breaks our
tests, i'll have to investigate that separately.
- Update attachments tab margins.
- Make sure to pass `&download` in attachment urls. This makes them
downloadable, regressed in #28727.
- Do not additionally list image diffs as screenshots.
Fixes#31912.
Closes https://github.com/microsoft/playwright/issues/22211
Currently, when the server notifies the UI about changed files, the UI
determines what files to re-run based on an old test list. By listing
tests before that, we make sure that the test list is up-to-date, and
that added tests are included in the next run.
I've also removed the `listChanged` event as discussed in the team sync.
The event isn't used anywhere and fires in exactly the same cases where
`testFilesChanged` fired, so i've folded them into one another. This allowed simplifying `Watcher`.
Motivation: When using client-certificates on a website on port `443`,
we would normalise the user input with `new URL` but still generate a
"bad" representation of the "origin" internally, since the just do
concatenated "host:port".
(The origin doesn't contain the port in case of :443)
We use `clientCertificatesToTLSOptions` in two places:
a) for APIRequestContext, there we pass one from the URL constructor
over and
b) from the socks proxy, there we **now** also pass a "good one" over.
Test plan: We don't want to run the tests on port :443, so only manually
validated the fix.
Relates https://github.com/microsoft/playwright/issues/31906
- Update copy to clipboard button.
- Reveal test source in the Source tab instead of external editor.
- New button to reveal in the external editor in the Source tab.
- Move the Pick Locator button next to snapshot tabs.
Upon calling `browser.close()` or dropping remote connection, make sure
to reject api calls before resolving `browser.close()` and firing a
`disconnected` event.
This change aligns the order guarantee with non-remote case.
Resolves https://github.com/microsoft/playwright/issues/31847 by adding
playwright config's `baseURL` value to the `context-options` trace
event, and showing that in the Trace Viewer.
Because the added property is optional, I didn't increment the trace
format version.
I've also considered pulling the `baseURL` from the existing
`browser.newContext` step to get around modifying the trace format, but
that felt pretty hacky.
https://github.com/user-attachments/assets/ecaef747-727d-4937-9ca3-1605ca9907b9
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Introduces an `--only-changed [base ref]` option.
`playwright test --only-changed` filters the test run to only run test
suites that have uncommitted changes.
`playwright test --only-changed=foo` runs only tests that were changed
since commit `foo`.
In pull request CI, this can be used to run changed tests first and fail
fast: `--only-changed=$GITHUB_BASE_REF`.
During local development, it can be used to quickly filter down to the
touched set of tests suites.
In some rare usecases, this can also help to cut down on CI usage for
pull requests. Tread with caution though.
File dependencies are taken into account to ensure that if you touched a
utility file, all relevant tests are still executed.
Closes https://github.com/microsoft/playwright/issues/15075
Broken out of https://github.com/microsoft/playwright/pull/31727 as per
@dgozman's
[request](https://github.com/microsoft/playwright/pull/31727#discussion_r1685793229).
The PR goal is to remove the `suite` argument from the Component
testing's Vite Plugin. `suite` is used to enrich Vite's dependency graph
with information about dependencies between test suites and helper
files. It essentially merges the Vite graph with the
`compilationCache.ts > fileDependencies` graph, and then writes the
result back into `compilationCache.ts > externalDependencies`.
By refactoring this to make the connection on the reading end in
`collectAffectedTestFiles`, we can drop the `suite` parameter.
We didn't yet have a test that depended on the dependency graph being
connected correctly between `fileDependencies` and
`externalDepedencies`, so I've [extended an existing
test](53a539938b)
to capture that.
This reverts commit 0aa2f06f68.
Discussed the new permission in the API review and decided not to
proceed with the feature as we are not ready to commit to supporting it
yet:
* the API is Chromium specific
* the API is still experimental
* there is no clarity to what extend the screen manipulation APIs will
work in old headless which is our main test environment
We'll keep an eye on the demand for the feature and may get back to
implementing it in the future.
Reference: https://github.com/microsoft/playwright/issues/27198
Fixes misspellings identified by the [check-spelling
action](https://github.com/marketplace/actions/check-spelling).
The misspellings have been reported at
https://github.com/jsoref/playwright/actions/runs/10015023629#summary-27685777352
The action will report that the changes in this PR would make it happy:
https://github.com/jsoref/playwright/actions/runs/10015023971#summary-27685778305
---
I understand that the commit messages will need to be reworded to match
house style. For the time being, these are merely noting the changes
they contain so that when I rebase or need to drop things, I can. --
I've already rebased once as someone fixed one of the items that my
draft work was going to fix.
---
## Testing
* The tests _mostly_ passed when I managed to trigger them, but there
were a handful of things that I didn't quite understand
* There are a large number of warnings relating to a bad interaction
between any workflow that uses this local action
b535139b32/.github/actions/run-test/action.yml (L74-L80)
and the action it calls -- I've opened Azure/login#474 asking them to
refactor their action so that it doesn't cause so much noise while
running this repository's tests
* I'm vaguely curious as to why this repository has a `branch`
constraint for its `pull_request` events in its workflows -- that
constraint gave me a number of additional headaches while trying to
prepare this branch for this PR.
---------
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Previously, only the "actual" attachment was created, pointing to the
file in `test-results`. Now, the "expected" attachment pointing to the
file in `__screenshots__` is also created. This will help any reporters
that would like to know the "expected" path, for example to do a manual
accept/decline of the baseline.
Fixes#30693.
The following actions keep `noWaitAfter` option: `click`, `selectOption`
and `press`.
All other actions that used to have `noWaitAfter` now behave like it was
set to true, not waiting for follow-up navigations. In the docs, this
option is marked as completely ignored.
A small logic change was made to compensate for this behavior: when
waiting for the `hitTargetInterceptor`, we now race it against
navigations to avoid stalling when navigation stalls. Previously,
waiting for the interceptor was disabled when `noWaitAfter` was passed,
and since it's impossible to pass this option now, we mitigate by never
stalling instead.
Fixes#31469.
... unless an array of file-system-friendly parts is provided.
Motivation: attachment name is used as a file system path when
downloading attachments, so we keep them fs-friendly.
References #30693.
When used in a terminal, the `list` reporter prints out information
about test steps to help debugging. In non-TTY environments like GitHub
Actions, currently it doesn't.
This PR changes that, so that in non-TTY environments you'll see the
"step end" messages appearing, but not the "step begin" messages. This
is a good middleground, because it helps the user understand test
progress, without being too verbose.
Closes https://github.com/microsoft/playwright/issues/31674
Retaining traces in the following scenarios:
- browser crash;
- manual `browser.close()`;
- implicit `browser.close()` from the `browser` fixture upon test end.
This does not affect the library, where `browser.close()` will not
retain the trace and will close the browser as fast as possible.
References #31541, #31535, #31537.
Redirects are always autoresumed, so the will always receive extra info
with raw headers. We only want to make raw headers available immediately
when there is a route.
Reference https://github.com/microsoft/playwright/issues/31351
* Revert harTracer change from
aeba083da0
to make sure that har.Entry._monotonicTime always represents request
start time. The issue from the corresponding report was due to HEAD and
GET request sent for the same URL, that use case is still addressed as
we match by url + method
* Adjust resources monotonic time as well when several contexts are
shown in the trace viewer.
Fixes https://github.com/microsoft/playwright/issues/31133
Partially fixes https://github.com/microsoft/playwright/issues/31337 by
supporting ignoring node_modules on windows.
When I debug the function it gets a unix style path filename on windows,
so the function never ignores node_modules. The ignore path globs are
expected to use the unix path seperator and I've tested this fix works
on windows and I assume that since mac uses unix style, it also works
there (this is a pretty standard glob construct (chokidar points at any
match https://github.com/micromatch/anymatch and anymatch has this exact
example in their readme.md)
Signed-off-by: Luke Page <137174537+lukpsaxo@users.noreply.github.com>
Fixes https://github.com/microsoft/playwright/issues/31355
All changes were done with the Android Studio upgrade assistant. It
updates it to the latest Gradle to make it compatible with recent Java
while keeping the `targetSdkVersion` unchanged.
**Investigation**
~~We use `nonStallingEvaluateInExistingContext` as of today, which does
`eval()` inside (from our utilityScript) which breaks for some sites. It
causes a hang, since the returned `Promise` of `eval()` hangs. We don't
know as of today why this happens. Without wrapping it ini `eval()` it
does not hang.~~
~~Workaround: Do a plain Runtime.evaluate instead.~~
workaround: Only wait on main frame.
Relates https://github.com/microsoft/playwright/issues/28995 (keeping it
open until they confirm that it helps)
Historically, this plugin was important to translate dynamic imports
into require calls so that we can intercept them and transpile.
This is not needed anymore with ESM loader enabled by default, so we can
avoid this transformation and support dynamic imports of ESM-only
packages/files.
Fixes#17075, fixes#23255, fixes#31140, references #23662.
This allows snapshots of file:/// pages with external stylesheets,
images, etc to be rendered correctly in the trace viewer. (Otherwise, it
tries to request the file:/// URIs directly and the requests get blocked
by the browser.)
Fixes#31112.
* Column widths are now stored on in the NetworkPanel context, this way
they are not reset after selecting an empty range (and changing position
of the NetworkGridView in the component tree).
* Column widths values are now preserved if column set changes (e.g.
selecting entries from a single context and then from multiple
contexts).
When merging blob reports test ids are patched to make sure there is no
collision when merging reports that might have overlapping test ids.
However, even if you were merging reports that had no overlapping ids,
all test ids will be modified, which is an undesirable side effect.
This PR only modify test ids when the same test id has already been used
in a previous blob report.
----
This change is also part of
https://github.com/microsoft/playwright/pull/30962
The spirit of this change is reverting #23153. Since that time, we have
moved tracing and `artifactsDir` lifetime into the test runner, so the
reason for revert is mitigated.
Fixes#30287, fixes#30718, fixes#30959.
When a signal arrives late enough, after the last action was committed
in 5 seconds, we should still account for it. This includes downloads,
popups and dialogs, but not navigations.
Exposed by a flaky test "should record open in a new tab with url".
This includes two major changes:
- reuse `SerializedFS` for live test runner tracing;
- merge scheduled `appendFile` operations into a single `fs` call.
In some cases, this improves performance of UI mode by 61% and
performance of `trace: on` mode by 38%. Note that performance
improvement on the average test will not be as noticeable.
References #30875, #30635.
Chromium's `DevTools listening on` message sometimes arrives before
Playwright is finished connecting to Node. Without this patch, it would
miss the message and fail to connect.
- Documents `PLAYWRIGHT_FORCE_TTY` and `FORCE_COLOR` across terminal
reporters.
- New `PLAYWRIGHT_LIST_PRINT_STEPS`. Removes undocumented test-only
`PW_TEST_DEBUG_REPORTERS_PRINT_STEPS`.
- Replaces `PLAYWRIGHT_HTML_REPORT` with `PLAYWRIGHT_HTML_OUTPUT_DIR`
and `PW_TEST_HTML_REPORT_OPEN` with `PLAYWRIGHT_HTML_OPEN` for
consistency, supports older versions for backwards compatibility.
- New `PLAYWRIGHT_HTML_HOST`, `PLAYWRIGHT_HTML_PORT` and
`PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL`.
- New `PLAYWRIGHT_JUNIT_STRIP_ANSI` and
`PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME`.
- Removes `PW_HTML_REPORT` that was set for unknown reason.
This is a follow up #29564
I did a deep dive on a redirect issue I observed in my infrastructure
and originally attributed to some configuration mistakes on my part.
I have code hosted on `example.com/code` and use subdomain proxying.
This leads to the uimode being exposed on
`example.com/code/proxy/{{port}}`.
Clicking on the open uimode link shown by vscode redirected with a 302
to `example.com/proxy/{{port}}`
The absolute redirect url overruled the relative path handling reverse
proxies rely on.
This PR turns the absolute into a relative url to avoid this issue.
Previously, terminal reporters consulted `process.stdout.isTTY`. Now it
is possible to control the tty behavior:
- `PLAYWRIGHT_FORCE_TTY=0` or `PLAYWRIGHT_FORCE_TTY=false` to disable
TTY;
- `PLAYWRIGHT_FORCE_TTY=1` or `PLAYWRIGHT_FORCE_TTY=true` to enable TTY,
defaults to 100 columns when real columns are unavailable;
- `PLAYWRIGHT_FORCE_TTY=<number>` to force enable TTY and set the
columns.
Fixes#29422.
Previously, terminating worker always had a 30 seconds force exit.
Now, regular worker termination assumes that process will eventually
finish tearing down all the fixtures and exits. However, the
self-destruction routine keeps the 30 seconds timeout to avoid zombies.
Fixes#30504.
Implements feature requested in
https://github.com/microsoft/playwright/issues/30457
The test runner treats flaky tests as failures when the flag is enabled,
but still reports flaky tests as flaky in the reporting interface. It
feels like something worth discussing as this behaviour makes sense to
me, but looked a bit odd to @BJSS-russell-pollock when I ran this past
him.
Closes#30457.
Matching bu `apiName@wallTime` fails when two actions start at the same
time, e.g. two parallel api requests. Moreover, it results in trace
actions that have parent set to themselves, which in turn causes
infinite loop in the trace viewer. To avoid this problems we write
stepId explicitly to the library trace and use those step ids to find
corresponding test runner steps.
The stepId is passed via zone in case of expect, because the protocol
step is quite deep in the call chain after or explicitly in case of API
call steps created by the test runner instrumentation.
Similarly to page.close, we pass test-runner specific reason to
facilitate better error messages.
```
1) a.test.ts:10:11 › test
Error: apiRequestContext.fetch: Fixture { request } from beforeAll cannot be reused in a test.
- Recommended fix: use a separate { request } in the test.
- Alternatively, manually create APIRequestContext in beforeAll and dispose it in afterAll.
See https://playwright.dev/docs/api-testing#sending-api-requests-from-ui-tests for more details.
9 |
10 | test('test', async () => {
> 11 | await context.fetch('http://example.com');
| ^
12 | });
13 |
```
Closes#29260.
