Otherwise, we might split the `beforeAll`-grouped test group into
`workers` parts instead of `shard.total` parts as the user would expect.
Fixes#33077.
The diff is now shown inline in the errors list.
There are 2 possible failures of toHaveScreenshot
* Previous and actual snapshot mismatch. In this case html report will
show diff between Actual/Previous and have Expected as a separate
screenshot.
* Actual/Previous are equal but they differ from the expected. In this
case html report only contains Actual/Expected images and the diff.
Reference: https://github.com/microsoft/playwright/issues/32341
<img width="1039" alt="image"
src="https://github.com/user-attachments/assets/b458f986-cc25-4721-862c-0cc2c1b01a42">
Fixes https://github.com/microsoft/playwright/issues/30160
### Description:
This pull request introduces the ability to specify custom locations for
test steps in Playwright. By enabling the provision of arbitrary
locations to the test.step method, it resolves the limitation where
helper methods obfuscate the original call site, providing more accurate
and meaningful location data in test reports.
### Motivation:
To enhance the utility and clarity of test reports in Playwright.
Specifically, it addresses the need to trace test steps back to their
precise location in the code, which is especially important when steps
are abstracted in helper functions. This feature is crucial for
maintaining accurate documentation and facilitating debugging processes.
### Changes:
Added functionality to pass a custom location object to test.step.
### Expected Outcome:
This PR is expected to significantly improve the precision and
usefulness of diagnostic data in test reports by allowing specific
locations within helper functions to be accurately documented. It
facilitates better tracking of test executions and simplifies the
debugging process, making it easier for developers to understand and
address issues within complex tests.
### References:
Closes https://github.com/microsoft/playwright/issues/30160 -
"[Feature]: allow to pass arbitrary location to test.step"
**Code Check**
I conducted tests on this new feature by integrating it into some
existing test codes, and it worked well. I will attach the code used for
testing and a screenshot showing the successful outcome.
<details>
<summary>toggle dropdown</summary>
<div markdown="1">
```
import type { Location } from '../../../packages/playwright/types/testReporter'
...
test('should respect the back button', async ({ page }) => {
await page.locator('.todo-list li .toggle').nth(1).check();
await checkNumberOfCompletedTodosInLocalStorage(page, 1);
...
await test.step('Showing active items', async () => {
await page.getByRole('link', { name: 'Active' }).click();
}, {location});
```
<img width="1109" alt="image"
src="https://github.com/user-attachments/assets/359feafa-0949-4c71-9426-46debef21bdd">
</div>
</details>
Closes https://github.com/microsoft/playwright/issues/32578.
Adds a buffer mode that can be toggled by pressing <kbd>b</kbd>. When
engaged, changed test files are collected and shown on screen. The test
run is then kicked off by pressing <kbd>Enter</kbd>.
This changes the signal to start a test run from <kbd>Cmd+s</kbd> to a
<kbd>Enter</kbd> press in the test terminal. It should help users with
auto-save and make it easier to run on long-running tests. It feels very
similar to running `npx playwright test`, but without having to write a
filter.
https://github.com/user-attachments/assets/71e16139-9427-4e90-b523-8d218f09ed9d
Changes `expect.extend` behaviour so that it doesn't mutate the global
instance and behaves closer to what users expect. This is formally a
breaking change, and I had to remove a test that asserts the breaking
behaviour.
TODO:
- [x] decide wether this is a separate method or a flag for
`expect.extend`
- [x] figure out if we need to change docs
We now hopefully align with `moduleResolution: bundler` tsconfig option,
allowing directory imports in every scenario, and allowing proper module
imports when not going through the type mapping.
This regressed in #32078. Fixes#32480, fixes#31811.
Instead of plumbing it through a custom unspecified config field, make
it a part of plugin interface.
Additionally, use task runner for starting/stopping dev server.
After API review we decided to revert it:
* VSCode extension and UI mode users already get the (runtime) error if
the tag is not prefixed
* The typescript error message is not very nice
* The type change would break those clients that generate tests with
tags passed as string
This reverts commit 90e7b9ebac.
Closes https://github.com/microsoft/playwright/issues/32076.
This PR rewrites `watchMode.ts` to use `TestServer` under the hood. It's
essentially a complete rewrite, so don't pay too much attention on the
old implementation. Note that there's no changes to tests, so all
behaviour we have specced out there still works.
To make this work without a superfluous WebSocket connection, I had to
refactor `TestServerConnection` a little. Originally, I pulled this into
a [separate PR](https://github.com/microsoft/playwright/pull/32132), but
then realised how small the refactoring is. So it's in this PR now. Let
me know if you'd like to land it separately.
The `'should support custom matchers'` test asserts that the
functionality works, but it was a type error. This PR updates the types
so that it's allowed.
Closes https://github.com/microsoft/playwright/issues/32408
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Closes https://github.com/microsoft/playwright/issues/32331
We're already passing the `outputDir` param to the UI, but the UI isn't
passing it back to the TestServer. This PR fixes that. I've added it to
`listTests`, which is requires to that
`TestServerDispatcher#_ignoredProjectOutputs` is populated with the
correct output dir. And i've added it to `runGlobalSetup`, which is what
the bug report was about.
Closes https://github.com/microsoft/playwright/issues/32256
We were expecting all errors to be of type `ExpectError`, but apparently
`expect` propagates rejections in the polling functions right through.
So we also need to handle that case.
I wonder if we have more cases of this. Would it make sense to enable
`useUnknownInCatchVariables` in TypeScript?
Closes https://github.com/microsoft/playwright/issues/32180
I was briefly wondering if we should output a log line a la "no tests
found", but my understanding is that that's the reporters job - so I
didn't change anything in that regard.
When two fixtures have different time slots, timeout in the first one
should not prevent the second one from tearing down.
Similarly, timeout in afterEach hook should not prevent fixture
teardowns.
This allows any time slot that has a legitimate timeout of zero to be
updated later on. See test for an example.
Previously, setting timeout to zero at any moment was considered a
"debug mode" and any subsequent timeouts were ignored.
Closes https://github.com/microsoft/playwright/issues/32070. We were
applying `additionalFileMatcher` not just to `filteredProjectSuites`,
but also to `projectSuites`. `projectSuites` is where we take dependency
projects from, though - so `--only-changed` led to empty dependency
projects, resulting in the reported bug.
The fix is to only apply `additionalFileMatcher` on
`filteredProjectSuites`.
Supercedes https://github.com/microsoft/playwright/pull/31915, closes
https://github.com/microsoft/playwright/issues/31811.
When TypeScript resolves a specifier via path mapping, it does not
interpret `package.json`. If path mapping resolves to a directory, it
only looks at the `index.js` file in that directory if it's in CommonJS
mode.
We need to mirror this in our `esmLoader.ts`.
---------
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Introduce `--tsconfig` to specify a single config to be used for all
imported files, instead of looking up tsconfig for each file separately.
Fixes#12829.
