Commit graph

550 commits

Author SHA1 Message Date
Yury Semikhatsky 8ec981c394
chore: move github reporter formatting out of base (#33213) 2024-10-22 11:45:12 -07:00
Yury Semikhatsky aebceb345e
chore: expose expect error details on TestError (#33183) 2024-10-21 11:15:55 -07:00
Max Schmitt b1fb4f16a7
chore: hide 'markdown' reporter (#33140) 2024-10-18 23:00:05 +02:00
Pavel Feldman 2e8e7a66cd
chore: implement tree w/o list (#33169) 2024-10-18 13:50:43 -07:00
Simon Knott 0d63df4875
feat(test runner): allow multiple global setups (#32955)
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2024-10-18 11:03:00 +02:00
Pavel Feldman 29c84a33c3
chore: compute aria text consistently with the role accumulated text (#33157) 2024-10-17 17:06:18 -07:00
Dmitry Gozman aa952c1b03
fix(html report): generate test snippets when test dir is non-root (#33162) 2024-10-17 08:33:15 -07:00
Dmitry Gozman 2d150eec25
fix: correct types for things like test.describe.only (#33142) 2024-10-17 03:34:05 -07:00
Pengoose d10a5e5693
feat(testType): add support for test.fail.only method (#33001) 2024-10-16 06:47:23 -07:00
Pavel Feldman b421bd8b0d
chore: add a basic snapshot generator test (#33123) 2024-10-15 15:21:45 -07:00
Pavel Feldman a38ff6e0d8
chore: experimental toMatchAriaSnapshot (#33014) 2024-10-14 14:07:19 -07:00
Dmitry Gozman ecd147ce43
fix(test runner): when sharding with beforeAll, use shards total instead of workers (#33083)
Otherwise, we might split the `beforeAll`-grouped test group into
`workers` parts instead of `shard.total` parts as the user would expect.

Fixes #33077.
2024-10-14 13:46:06 -07:00
Yury Semikhatsky b9cce598dd
fix(screenshot): show image diff inline in errors list (#32997)
The diff is now shown inline in the errors list.

There are 2 possible failures of toHaveScreenshot
* Previous and actual snapshot mismatch. In this case html report will
show diff between Actual/Previous and have Expected as a separate
screenshot.
* Actual/Previous are equal but they differ from the expected. In this
case html report only contains Actual/Expected images and the diff.

Reference: https://github.com/microsoft/playwright/issues/32341

<img width="1039" alt="image"
src="https://github.com/user-attachments/assets/b458f986-cc25-4721-862c-0cc2c1b01a42">
2024-10-10 16:49:17 -07:00
Dmitry Gozman 25dd9b5cd4
feat: config.build.tsconfig (#33026)
Allows to specify `tsconfig` in the configuration file, which applies to
test files but not the config file itself.

Fixes #32808.
2024-10-10 01:37:46 -07:00
Pavel Feldman 8f3353865d
fix(ui): bring back the headed param (#33030)
Fixes https://github.com/microsoft/playwright/issues/33023
2024-10-09 17:46:54 -07:00
Dmitry Gozman c2cbf26497
docs: refresh timeouts doc (#33025)
After changes a few releases ago, specify that `afterEach` hooks are
included in a separate timeout.

Fixes #32851.
2024-10-09 05:10:10 -07:00
Simon Knott 892d67ffef
feat(test runner): allow stopping testrun with escape (#32584)
Closes https://github.com/microsoft/playwright/issues/32579
2024-10-08 11:39:54 -07:00
Max Schmitt d0f2170e21
Revert "fix(test): do not allow mixing tests from different types (#29284)" (#33002)
This reverts commit 4784139bb0.

Closes https://github.com/microsoft/playwright/issues/29734
2024-10-08 16:00:40 +02:00
Mark 04bf425268
feat(base-reporter): Add tags to test output (#32930) 2024-10-07 17:14:46 -07:00
Simon Knott 40670e6ffd
fix(emulate media): document "no-preference" as deprecated (#32881)
Closes https://github.com/microsoft/playwright/issues/32862.

`prefers-color-scheme: no-preference` was removed from browsers. This PR
marks it as deprecated in our docs and removes all mentions.

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2024-10-04 16:49:32 +02:00
Simon Knott 208a54529d
fix(ct-react): support shorthand fragment notation (#32900)
Closes https://github.com/microsoft/playwright/issues/32853

Vite turns the shorthand fragment notation `<></>` into `import {
Fragment } from "react"; <Fragment></Fragment>`. On the Node.js side of
things, this `react` import resolves to our mock version of React, which
currently mocks `Fragment` as `{}`. Currently, we pass that straight to
`React.createElement`, which throws an error.

The fix is to make our `Fragment` mock detectable with a tag, and when
we render it replace it with the real `__pwReact.Fragment`.
2024-10-02 11:19:09 +02:00
Dmitry Gozman 541ce9f0bb
chore: mark 1.49.0-next (#32873) 2024-09-30 04:24:57 -07:00
Yury Semikhatsky 5b85c71722
fix(junit): env variable should take precedence over config (#32842)
Fixes https://github.com/microsoft/playwright/issues/32826
2024-09-26 15:29:09 -07:00
Yury Semikhatsky e8d08e8d1e
docs: more prominent note on tags being read-only (#32836)
Reference https://github.com/microsoft/playwright/issues/32828
2024-09-26 09:41:38 -07:00
Max Schmitt 3b86a9c0e4
docs(markdown): parse :::note's text as children (#32510) 2024-09-26 15:16:26 +02:00
Dmitry Gozman a2bdb2fd79
docs: linkify params and options (#32823)
References https://github.com/microsoft/playwright/issues/32590.
2024-09-26 05:13:00 -07:00
Dmitry Gozman ff954b58eb
fix(test runner): cleanup DEBUG_COLORS usage (#32764)
`DEBUG_COLORS` we default to `1`, but we should not do that when it is
already defined to some value supplied by the user.

Closes #32543.
2024-09-26 03:15:46 -07:00
Yury Semikhatsky 597642d269
chore: pass cli config overrides straight to test server (#32818) 2024-09-25 19:45:59 -07:00
Yury Semikhatsky b6783bb18b
fix(ui-mode): respect --tsconfig option (#32798)
Fixes https://github.com/microsoft/playwright/issues/32797
2024-09-24 19:59:15 -07:00
Pavel Feldman 755edfba5b
chore: fallback expect.extend to legacy (#32795) 2024-09-24 14:00:13 -07:00
Max Schmitt 12ecd476dd
fix(watch): cancel waitForCommand when files change (#32761)
Fixes https://github.com/microsoft/playwright/issues/32758
2024-09-23 14:43:28 +02:00
Pavel Feldman dfb3fdf217
chore: iterate towards recording into trace (3) (#32718) 2024-09-20 13:08:33 -07:00
Dmitry Gozman ace8cb2427
fix(test runner): page.pause() should enable debug mode (#32714)
Fixes #32706.
2024-09-20 00:41:16 -07:00
Max Schmitt 2b2a57abad
chore: enable @babel/plugin-syntax-import-attributes all the time (#32713) 2024-09-19 16:51:35 +02:00
Max Schmitt 523ec83cad
chore: move Location type from testReporter.d.ts to test.d.ts (#32687) 2024-09-18 16:57:11 +02:00
Simon Knott 443f72dcbe
feat(watch): hide show-report prompt (#32666)
Closes https://github.com/microsoft/playwright/issues/32665
2024-09-18 08:17:13 +02:00
오소현 8761dafc73
feat(test runner): allow to pass arbitrary location to test.step (#32504)
Fixes https://github.com/microsoft/playwright/issues/30160

### Description:
This pull request introduces the ability to specify custom locations for
test steps in Playwright. By enabling the provision of arbitrary
locations to the test.step method, it resolves the limitation where
helper methods obfuscate the original call site, providing more accurate
and meaningful location data in test reports.

### Motivation:
To enhance the utility and clarity of test reports in Playwright.
Specifically, it addresses the need to trace test steps back to their
precise location in the code, which is especially important when steps
are abstracted in helper functions. This feature is crucial for
maintaining accurate documentation and facilitating debugging processes.

### Changes:
Added functionality to pass a custom location object to test.step.

### Expected Outcome:
This PR is expected to significantly improve the precision and
usefulness of diagnostic data in test reports by allowing specific
locations within helper functions to be accurately documented. It
facilitates better tracking of test executions and simplifies the
debugging process, making it easier for developers to understand and
address issues within complex tests.

### References:
Closes https://github.com/microsoft/playwright/issues/30160 -
"[Feature]: allow to pass arbitrary location to test.step"

**Code Check**
I conducted tests on this new feature by integrating it into some
existing test codes, and it worked well. I will attach the code used for
testing and a screenshot showing the successful outcome.

<details>
<summary>toggle dropdown</summary>
<div markdown="1">

```
import type { Location } from '../../../packages/playwright/types/testReporter'
...
test('should respect the back button', async ({ page }) => {
    await page.locator('.todo-list li .toggle').nth(1).check();
    await checkNumberOfCompletedTodosInLocalStorage(page, 1);
...
    await test.step('Showing active items', async () => {
      await page.getByRole('link', { name: 'Active' }).click();
    }, {location});
```

<img width="1109" alt="image"
src="https://github.com/user-attachments/assets/359feafa-0949-4c71-9426-46debef21bdd">
</div>
</details>
2024-09-17 08:11:21 -07:00
Max Schmitt 507e515cb2
chore: remove unused @babel/parser (#32654) 2024-09-17 16:14:24 +02:00
Simon Knott f6219e6e79
Revert "feat(tracing): add .pwtrace to trace file extension" (#32648)
Reverts microsoft/playwright#32581
Relates
https://github.com/microsoft/playwright/issues/32226#issuecomment-2351164727
2024-09-17 15:32:30 +02:00
Simon Knott ec2ae1ed2d
feat(watch mode): buffer mode (#32631)
Closes https://github.com/microsoft/playwright/issues/32578.

Adds a buffer mode that can be toggled by pressing <kbd>b</kbd>. When
engaged, changed test files are collected and shown on screen. The test
run is then kicked off by pressing <kbd>Enter</kbd>.

This changes the signal to start a test run from <kbd>Cmd+s</kbd> to a
<kbd>Enter</kbd> press in the test terminal. It should help users with
auto-save and make it easier to run on long-running tests. It feels very
similar to running `npx playwright test`, but without having to write a
filter.



https://github.com/user-attachments/assets/71e16139-9427-4e90-b523-8d218f09ed9d
2024-09-17 08:45:44 +02:00
Dmitry Gozman ce06a81aa6
feat: make npx playwright clear-cache public (#32638) 2024-09-16 12:54:20 -07:00
Max Schmitt 21d162c945
feat(client-certificates): add support for proxies (#32611)
Fixes https://github.com/microsoft/playwright/issues/32370
2024-09-16 17:57:33 +02:00
Dmitry Gozman 268357238a
fix(expect): respect custom message in expect.poll (#32603)
Fixes #32582.
2024-09-16 00:10:06 -07:00
Simon Knott aeb4d182f7
feat(tracing): add .pwtrace to trace file extension (#32581)
Closes https://github.com/microsoft/playwright/issues/32226

I've updated every mention of `.trace.zip` except for the release notes.
2024-09-14 10:17:07 +02:00
Simon Knott 9bb1c86f93
feat(test runner): don't run tests on --watch start (#32583)
Closes https://github.com/microsoft/playwright/issues/32580.
2024-09-13 17:24:38 +02:00
Max Schmitt 9e99c86f00
chore: unhide merge-reports command (#32605) 2024-09-13 16:13:23 +02:00
Simon Knott ef4be6afff
feat(test runner): make expect.extend immutable (#32366)
Changes `expect.extend` behaviour so that it doesn't mutate the global
instance and behaves closer to what users expect. This is formally a
breaking change, and I had to remove a test that asserts the breaking
behaviour.

TODO:
- [x] decide wether this is a separate method or a flag for
`expect.extend`
- [x] figure out if we need to change docs
2024-09-12 19:56:38 +02:00
Dmitry Gozman 29a0f49e9b
chore(test runner): simplify code around running tasks (#32557)
This avoids complex boilerplate around `onConfigure`/`onEnd`/`onExit`
and managing the resulting status.
2024-09-11 13:09:00 -07:00
Max Schmitt 7458c3292c
fix(test-runner): do not consider retries for maxFailures (#32533)
Fixes https://github.com/microsoft/playwright/issues/26350
2024-09-11 20:35:26 +02:00
Yury Semikhatsky a4bd551597
docs: TestInfo.titlePath does not include project (#32548) 2024-09-10 16:52:12 -07:00