Filip/playwright
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
12085 commits 34 branches 141 tags 223 MiB
Commit graph

1187 commits

Author SHA1 Message Date
Pavel Feldman 5d9e08ac61
chore: box soft steps (#28749) 
Fixes https://github.com/microsoft/playwright/issues/28259
 2023-12-21 12:16:03 -08:00
Yury Semikhatsky 6d583a2dbe
fix: exit from merge-reports in type: "module" projects (#28725) 
Fixes https://github.com/microsoft/playwright/issues/28699
 2023-12-19 16:49:25 -08:00
Yury Semikhatsky dcaa5758f0
fix: respect .only in --list mode (#28721) 
Fixes https://github.com/microsoft/playwright/issues/28709
 2023-12-19 12:24:54 -08:00
George Griffiths 4c848c0daf
feat(expect): Allow expect timeout to be configured outside of the runner (#28716) 
Fixes #28702
 2023-12-19 10:55:05 -08:00
Yury Semikhatsky 9c845365f7
chore: revert TestProject.botName for this release (#28670) 
This is essentially a revert of f88288d71d

Reference https://github.com/microsoft/playwright/issues/27284
 2023-12-15 09:48:34 -08:00
Pavel Feldman d242ff67ef
chore: follow up to the stylePath review (#28636) 2023-12-14 08:19:24 -08:00
Pavel Feldman afe90d648e
fix: do not generate api call steps inside named expects (#28609) 
Fixes: https://github.com/microsoft/playwright/issues/28528
 2023-12-13 09:06:02 -08:00
Max Schmitt 297cfdfc5f
chore: use ReadonlyArray for input parameters (#28564) 2023-12-12 16:22:48 -08:00
Dmitry Gozman d2dc8eb1e3
fix(components): make sure defineConfig(c1, c2) works (#28608) 2023-12-12 14:01:01 -08:00
Yury Semikhatsky 411abdb752
feat: add fileName option to blob reporter (#28525) 
Reference https://github.com/microsoft/playwright/issues/27284
 2023-12-06 20:44:06 -08:00
Yury Semikhatsky 2adfbf24a9
chore: roll stable test runner (#28523) 2023-12-06 14:21:01 -08:00
Yury Semikhatsky f88288d71d
feat: config.botName for describing environment in the reports (#28507) 
Reference https://github.com/microsoft/playwright/issues/27284
 2023-12-06 13:34:16 -08:00
Pavel Feldman 0a7a10d0f6
feat(vrt): allow providing screenshot style (#28229) 2023-11-30 17:42:45 -08:00
Pavel Feldman 92a0d24069
fix: restore timeout error name (#28408) 
Fixes https://github.com/microsoft/playwright/issues/28404
 2023-11-29 17:28:17 -08:00
Pavel Feldman 022b36332d
chore: do not add to the internal action logs (#28365) 
Fixes https://github.com/microsoft/playwright/issues/28319
 2023-11-27 16:43:47 -08:00
Dmitry Gozman f61e445f2b
Revert "chore(test runner): remove fake skipped test results (#27762)" (#28360) 
This reverts commit 210168e36d.

Fixes #28321.
 2023-11-27 16:08:20 -08:00
Yury Semikhatsky dc8ecc3ca4
fix(merge): normalize path separators when merging across platforms (#28227) 2023-11-27 12:43:56 -08:00
Dmitry Gozman 62d4c3fe02
fix(defineConfig): do not add an empty project list to project-less configs (#28224) 
Otherwise, merging two configs without `projects` property will create a
config with an empty project list, which is semantically different and
always leads to "No tests found".
 2023-11-17 13:36:50 -08:00
Pavel Feldman 2bd7d67adc
chore: render testInfo errors in the Errors tab (#28179) 
Fixes https://github.com/microsoft/playwright/issues/28056
 2023-11-16 11:37:57 -08:00
Dmitry Gozman 19cfd0cc5e
chore: roll stable-test-runner to 1.40.0-beta-1700102862000 (#28192) 2023-11-16 10:14:56 -08:00
Pavel Feldman 25b9c4eb4a
chore: do not lose error name for js errors (#28177) 2023-11-15 18:27:32 -08:00
Dmitry Gozman bf4c315b09
fix(types): explicit ExpectMatcherState type, optional Expect arg (#28119) 
Fixes #28035.
 2023-11-13 18:37:50 -08:00
Yury Semikhatsky 53a78a315e
fix(merge): preserve original "rootDir" by default (#27963) 
Fixes https://github.com/microsoft/playwright/issues/27877
 2023-11-03 13:49:47 -07:00
Pavel Feldman e84dd4d708
fix(ui): do not fail on clashing groups (#27943) 
Fixes https://github.com/microsoft/playwright/issues/27929
 2023-11-02 20:50:08 -07:00
Ben Scott c59483c5fb
fix(tsconfig): handle tsconfig paths without a baseUrl (#27846) 2023-11-02 11:27:49 -07:00
Max Schmitt 5f366088be
chore: get rid of rimraf package (#27790) 
This seems more reliable nowadays as rimraf.

https://github.com/microsoft/playwright/issues/27712

---------

Signed-off-by: Max Schmitt <max@schmitt.mx>
 2023-10-26 19:41:00 +02:00
Yury Semikhatsky 86c78c1e0f
fix(html): do not update total time when filtering tests (#27808) 
![image](https://github.com/microsoft/playwright/assets/9798949/491ef34d-6674-4242-a6f4-dcef048a18f5)


Fixes #27758
 2023-10-26 08:20:14 -07:00
Pavel Feldman 47733b04fb
chore: do not select after hooks automatically (#27805) 2023-10-25 17:05:06 -07:00
Dmitry Gozman c8134bca5d
feat(html): show top-level errors (#27763) 
Drive-by:
- extract `TestErrorView`;
- replace `data-test-id` with `data-testid` and `getByTestId()`.

---

<img width="1001" alt="top-level errors in html report"
src="https://github.com/microsoft/playwright/assets/9881434/2d6c0c52-8df1-46a9-b3fd-06ddc6f16796">
 2023-10-24 09:35:07 -07:00
Dmitry Gozman 210168e36d
chore(test runner): remove fake skipped test results (#27762) 
Fixes #27455.
 2023-10-24 09:31:30 -07:00
Dmitry Gozman b8678ef902
chore(test runner): simplify some dispatcher logic (#27732) 
- remove `onlyStartedTests` in favor of explicit branch with comments;
- produce one "test not found" error per test instead of a single large
error;
- extract `_failTestWithErrors` from `_massSkipTestsFromRemaining`.
 2023-10-23 09:15:13 -07:00
Dmitry Gozman 85112be25c
fix(test runner): properly handle uncaught errors in test.fail() (#27734) 
Before this fix, unhandled error during test.fail():
- marks this test as "interrupted";
- fails next test in the file with "fatal error".

After this fix:
- marks this test as "failed as expected";
- restarts worker for the next test.
 2023-10-23 09:14:31 -07:00
Dmitry Gozman d67515f6c1
chore(test runner): do not produce some of the fake skipped test results (#27730) 2023-10-20 17:01:46 -07:00
Pavel Feldman f46c889449
chore: fix total progress reporting w/ deps (#27716) 2023-10-19 20:07:47 -07:00
Pavel Feldman d003945a7b
chore(ui): do not watch output folders (#27694) 2023-10-19 15:53:57 -07:00
Dmitry Gozman 6e62a11643
fix(trace): EPERM on windows (#27693) 
When merging trace files, we sometimes left open read streams from the
zip, which prevents it from being removed.

Fixes #27286.
 2023-10-19 11:14:17 -07:00
Sander 562938b271
fix(ct): baseURL playwright config (#27689) 
closes: https://github.com/microsoft/playwright/issues/27627
 2023-10-18 13:53:58 -07:00
Dmitry Gozman fd82b2b3fa
feat(json report): add expected/unexpected/skipped/flaky stats (#27685) 
Fixes #27498.
 2023-10-18 12:55:31 -07:00
Pavel Feldman a54dbfdadf
chore: plumb the target close reason when test fails (#27640) 2023-10-16 20:32:13 -07:00
Max Schmitt fd2fbe9d2f
fix: merge{Tests,Expects} via ESM imports (#27630) 
Backport to 1.39.X?

Fixes https://github.com/microsoft/playwright/issues/27617
 2023-10-17 00:09:44 +02:00
Pavel Feldman f212fd1a83
chore: unify target closed errors (#27540) 2023-10-12 11:05:34 -07:00
Pavel Feldman 393bd36e0a
chore: composed->merge (#27555) 2023-10-11 13:56:27 -07:00
Dmitry Gozman 3a43aaa700
chore: roll stable-test-runner to 1.39.0-beta-1697048429000 (#27553) 2023-10-11 13:42:04 -07:00
Pavel Feldman fd6bf8aa2c
chrome: improve error messages on vscode side (#27521) 2023-10-09 19:55:24 -07:00
Pavel Feldman 11a4b3f7f5
chore: remove parsed stack trace (#27496) 2023-10-09 17:04:16 -07:00
Pavel Feldman 70dbb9d83a
feat(trace): allow navigating from error to source (#27464) 2023-10-05 14:59:59 -07:00
Jim Hays dcc8dcca73
Fix various typos (Fixes #27396) (#27391) 
Fixes https://github.com/microsoft/playwright/issues/27396
 2023-10-04 19:56:42 -07:00
Dmitry Gozman daba77644c
feat: composedExpect (#27432) 
Allows to merge multiple expects with custom matchers added by
`expect.extend()`.
 2023-10-04 15:01:25 -07:00
Yury Semikhatsky ac48a47d33
feat: toHaveAttribute without value (#27418) 
This time not doing it in other languages due to unjustified generator
complexity.

Fixes #27341
 2023-10-04 09:27:28 -07:00
Dmitry Gozman 65ce4cd213
feat: expose composedTest() instead of test._extendTest() (#27414) 2023-10-03 13:26:30 -07:00
Pavel Feldman d136b0aeb6
feat(step): allow boxing steps (#27334) 2023-09-29 09:44:00 -07:00
Sander 2032b64ee6
feat(ct): allow baseUrl and host config (#27295) 
closes: https://github.com/microsoft/playwright/issues/27283
 2023-09-26 16:32:51 -07:00
Yury Semikhatsky d198784f1a
fix(blob): use reporters from the merge config (#27301) 
Fixes https://github.com/microsoft/playwright/issues/27281
 2023-09-26 11:41:17 -07:00
Pavel Feldman aed86c98a8
chore(ct): allow using component as a property (#27272) 2023-09-25 17:00:52 -07:00
Dmitry Gozman 66eb3043f5
feat(json report): expose startTime and duration (#27290) 
Fixes #27183.
 2023-09-25 10:58:55 -07:00
Pavel Feldman a6a0257c88
feat(expect): allow chaining expects (#27248) 2023-09-22 12:12:17 -07:00
Dmitry Gozman 49fd9500fe
fix: handle worker process start failure (#27249) 
Worker process start failure is reported as a test error and skips other
tests from the group.
If happened during stop (e.g. from a Ctrl+C) before worker has fully
initialized, this error is ignored.

Drive-by: send SIGINT in tests to the whole tree, to better emulate
Ctrl+C behavior.
 2023-09-22 10:57:35 -07:00
Pavel Feldman 9fe037fb63
chore: populate matcherResult in web assertions (#27133) 
Ref https://github.com/microsoft/playwright/issues/26929
 2023-09-16 14:24:10 -07:00
Dmitry Gozman 3bcf9687e6
fix(test runner): make sure auto worker fixtures run before auto test fixtures (#27131) 
Fixes #27114.
 2023-09-15 14:51:31 -07:00
Pavel Feldman 527d260cbf
chore: hide tracing calls from traces (#27096) 2023-09-15 12:55:49 -07:00
Max Schmitt 0d44405762
fix: custom expect matchers on Locator/Page/APIResponse instance (#27117) 
Fixes https://github.com/microsoft/playwright/issues/27113
 2023-09-15 18:05:44 +02:00
Yury Semikhatsky ebf6a08290
fix: list tests only once (#27103) 
Fixes #27087
 2023-09-14 20:00:30 -07:00
Max Schmitt 26cbe69442
test: use intermediate merge report for JSON repoter (#27092) 2023-09-15 04:36:15 +02:00
David Paquette 3170963f42
fix: render JUnit attachment paths relative to outputFile (#27024) 2023-09-14 12:58:09 -07:00
Dmitry Gozman 603861c48d
fix: revert toMatchSnapshot deprecation (#27089) 
- This reverts commit
8348f66107
landed in #26718.
- Update docs to point towards `toHaveScreenshot`.
 2023-09-14 12:52:41 -07:00
Max Schmitt aa929bff3b
test: unflake 'filter should update stats' test (#27085) 
It's not possible to calculate the total test duration of the tests in
the UI, since our `msToString` function is lossy. This patch unflakes
the
[test](https://github.com/microsoft/playwright/actions/runs/6183832112/job/16787806162?pr=27074#step:7:254)
when it took longer than a second to start, by writing the test-ids to
the DOM, getting them into the Node.js process, and calculating their
test duration sum based on the JSON report.

Drive-by: I found a bug that we used the JSON report from the main
test-runner process rather than from the merge process, so the test IDs
were still old. (Without that change the `useIntermediateMergeReport`
tests were not passing.
 2023-09-14 19:51:05 +02:00
Max Schmitt 4431a8366d
fix: collect stdio from forked processes (#27074) 
A summary about the attempts we tried in
https://github.com/microsoft/playwright/pull/26931 for
https://github.com/microsoft/playwright/issues/26859 and
https://github.com/microsoft/playwright/issues/24591.

1. attempt: collect all the stdio from the worker host.
- this made it hard to capture the live console logs for the live trace
2. attempt: collect all the stdio from the worker host, but keep
proxying `process.{stdout,stderr}.write`.
- this solved the live console logs problem, since we could still listen
on them
- we found out about a race that the stdout/stderr streams can still
buffer data while the test already had finished
- we created a best effort flush implementation but didn't like the
complexity it introduced to the project
3. attempt: keep things like they are but also listen to stdio from the
worker host
- this solves the race of console.log in the test (things stay like they
are)
    - this keeps live trace working (things stay like they are)
- a known pitfall is that it could result in logs which don't end up in
the reporter onStdOut/onStdErr, but it is already a significant
improvement over how it was before.
      - before: they were not getting displayed
- after: they are getting displayed and there is a low likelihood that
they don't end up in the reporter API if the write happens slightly
before a test finished.

Closes #26931
Fixes #26859
Fixes #24591
 2023-09-14 18:34:26 +02:00
Dmitry Gozman f9298cfff6
fix(tracing): unclash trace names between test and hooks (#27063) 
Fixes #27048.
 2023-09-13 19:56:40 -07:00
Andrey Lushnikov 45d2a36453
chore: roll stable test runner to 1.38 beta (#27066) 2023-09-13 16:01:06 -07:00
Pavel Feldman 02c72e545b
feat(onEnd): allow overriding the exit code (#27010) 
Fixes: https://github.com/microsoft/playwright/issues/26858
 2023-09-12 13:37:30 -07:00
Max Schmitt d0945192a4
feat: make it possible to filter by column in HTML report (#27020) 2023-09-12 22:13:15 +02:00
Max Schmitt 01ab703400
test: restore useIntermediateMergeReport=true test mode in html tests (#26989) 
This got accidentally removed in
https://github.com/microsoft/playwright/pull/26799.
 2023-09-11 09:14:05 -07:00
Yury Semikhatsky 2feae015aa
chore: remove details from markdown reports (#26961) 
- remove error details from the reports
- collapse flaky tests by default
- limit comment to 65365 character

GitHub API has comment length limit 65536 chars:
```
Unhandled error: HttpError: Validation Failed: {"resource":"IssueComment","code":"unprocessable","field":"data","message":"Body is too long (maximum is 65536 characters)"}
```
 2023-09-08 17:49:34 -07:00
Dmitry Gozman 186f86905c
chore: make @playwright/test depend on playwright (#26946) 2023-09-08 14:23:35 -07:00
Pavel Feldman 167c35ca66
chore: store scroll positions in trace viewer (#26938) 2023-09-07 17:14:39 -07:00
Pavel Feldman a339bead09
chore: implement faster ansi2html format for console (#26826) 2023-09-01 09:09:47 -07:00
Pavel Feldman 4948920437
fix(ui): do not log from beforeAll twice (#26799) 
Fixes https://github.com/microsoft/playwright/issues/26790
 2023-08-31 17:34:15 -07:00
Dmitry Gozman fa286de0b3
fix(test runner): extract FailureTracker helper (#26757) 
This way we can reuse it for:
- tracking `maxFailures` across phases;
- tracking failures for runner;
- tracking failures for `runJob` helper class later on.

Fixes #26344.
 2023-08-31 15:32:29 -07:00
Dmitry Gozman 6d85ba1494
fix: remove connectInsteadOfLaunching (#26828) 2023-08-31 15:29:04 -07:00
Pavel Feldman c209d7e708
chore: more network panel polish (#26780) 2023-08-29 22:20:28 -07:00
Pavel Feldman 34c6197f9e
chore: include start/endTime and duration in onEnd report callback (#26760) 
Fixes https://github.com/microsoft/playwright/issues/23637
 2023-08-29 10:56:21 -07:00
Pavel Feldman 38f1e62642
chore: use custom expect message in trace step (#26752) 2023-08-28 14:48:24 -07:00
Dmitry Gozman 8348f66107
feat: deprecate toMatchSnapshot for buffers (#26718) 
These are usually used for screenshots instead of toHaveScreenshot.
String-based snapshots are left as is.

<img width="836" alt="deprecated"
src="https://github.com/microsoft/playwright/assets/9881434/d02818da-7eea-47fc-8ea8-4104b0dd49b5">

---

Also deemphasized toMatchSnapshot throughout all docs.

Fixes #26612.
 2023-08-25 18:20:01 -07:00
Pavel Feldman 376f2850b3
chore: flaky tests fixes (#26714) 2023-08-25 12:43:50 -07:00
Max Schmitt 9ae77a71fd
feat: print message if maxFailures has reached (#26322) 
Fixes https://github.com/microsoft/playwright/issues/24239
 2023-08-25 19:06:49 +02:00
Pavel Feldman e7bd1864a8
chore: do not add plugins to config twice (#26670) 2023-08-24 16:19:57 -07:00
Dmitry Gozman c970179551
fix(resolver): allow importing packages with non-index main script (#26692) 
Regressed in https://github.com/microsoft/playwright/pull/23254.

Fixes #26650.
 2023-08-24 14:09:00 -07:00
Dmitry Gozman c90c943154
fix(test.fail): expect() failure should not skip future tests (#26663) 
We used to stop the worker that would skip future tests. Regressed in
https://github.com/microsoft/playwright/pull/11850.

Fixes #26435.
 2023-08-24 08:32:14 -07:00
Dmitry Gozman 81cc39ea6e
feat(expect): narrow down available assertions for Page/Locator/APIResponse (#26658) 
Fixes #26381.
 2023-08-23 13:14:39 -07:00
Dmitry Gozman 218955c155
fix(test runner): make sure static annotations are reported for skipped tests (#26634) 
Fixes #26397.
 2023-08-23 08:40:12 -07:00
Pavel Feldman f4f9e526a2
chore: tolerate css imports (#26626) 
Fixes https://github.com/microsoft/playwright/issues/24580
 2023-08-23 08:32:23 -07:00
Pavel Feldman 00e6540799
feat(ui): show test trace events live (#26619) 2023-08-22 15:46:41 -07:00
Pavel Feldman 65aa062ea1
fix(console): make format console message w/o args (#26620) 
Fixes https://github.com/microsoft/playwright/issues/26600
 2023-08-22 14:29:35 -07:00
Pavel Feldman 1ceaa923ea
chore: create artifacts dir in the test runner (#26594) 2023-08-21 19:41:10 -07:00
Pavel Feldman c3c3c7f53c
chore: decorate console message sources (#26588) 2023-08-21 16:05:27 -07:00
Pavel Feldman f83d81956d
chore: make console stream live in ui mode (#26562) 2023-08-21 10:59:37 -07:00
Richard Blažo bcc30bc71e
feat: add title for before and after hooks (#26523) 2023-08-21 09:50:22 -07:00
Pavel Feldman 41c312cd04
chore: fix ui mode to show screenshots (#26563) 2023-08-20 14:47:18 -07:00
Pavel Feldman 09bb866333
chore: format console message from page (#26555) 2023-08-19 16:13:42 -07:00
Max Schmitt 75ed251c9e
fix: download of attachments in UI Mode (#26407) 
Fixes https://github.com/microsoft/playwright/issues/26326.
 2023-08-17 10:57:28 +02:00