Commit graph

1335 commits

Author SHA1 Message Date
Simon Knott 5bdeb38a80
disallow --only-changed in UI mode 2024-07-23 11:01:31 +02:00
Simon Knott a5d7ca17ae
remove CI smartypants 2024-07-23 11:01:30 +02:00
Simon Knott ab05bbf9c8
untracked and tracked files need to be treated differently 2024-07-23 11:01:29 +02:00
Simon Knott 359edf9c0f
treat edge case of calling it in subdirectory 2024-07-23 11:01:29 +02:00
Simon Knott f672d6dcf1
add test about component tests 2024-07-23 11:01:28 +02:00
Simon Knott 822c4cb2ad
remove BASE_REF in CI 2024-07-23 11:01:28 +02:00
Simon Knott 9fb7f13bfc
set local commit identity 2024-07-23 11:01:28 +02:00
Simon Knott 5d2da71fb8
also check CI env var 2024-07-23 11:01:27 +02:00
Simon Knott 5b758230d4
add nice error message 2024-07-23 11:01:27 +02:00
Simon Knott 5cb24de3ac
add test for watch mode 2024-07-23 11:01:26 +02:00
Simon Knott 0cbfd3470a
test more complex dependency graph 2024-07-23 11:01:26 +02:00
Simon Knott 72d7c86290
make it understand dependency structure 2024-07-23 11:01:26 +02:00
Simon Knott 824bea2aa0
refactor tests 2024-07-23 11:01:25 +02:00
Simon Knott d907883518
detect base ref from environment variables 2024-07-23 11:01:25 +02:00
Simon Knott 2d9a299d66
allow comparison against base commit 2024-07-23 11:01:24 +02:00
Simon Knott 30447b14f4
support changed files 2024-07-23 11:01:24 +02:00
Simon Knott da65da5d79
feat(test runner): --only-changed 2024-07-23 11:01:22 +02:00
Simon Knott 1408a45595
chore(ct): remove suite dependency by connecting dependency graphs at read time, not write time (#31794)
Broken out of https://github.com/microsoft/playwright/pull/31727 as per
@dgozman's
[request](https://github.com/microsoft/playwright/pull/31727#discussion_r1685793229).

The PR goal is to remove the `suite` argument from the Component
testing's Vite Plugin. `suite` is used to enrich Vite's dependency graph
with information about dependencies between test suites and helper
files. It essentially merges the Vite graph with the
`compilationCache.ts > fileDependencies` graph, and then writes the
result back into `compilationCache.ts > externalDependencies`.

By refactoring this to make the connection on the reading end in
`collectAffectedTestFiles`, we can drop the `suite` parameter.

We didn't yet have a test that depended on the dependency graph being
connected correctly between `fileDependencies` and
`externalDepedencies`, so I've [extended an existing
test](53a539938b)
to capture that.
2024-07-23 10:19:58 +02:00
Dmitry Gozman 056997c41f
fix(toHaveScreenshot): attach "expected" when writing a missing expectation (#31745)
Previously, only the "actual" attachment was created, pointing to the
file in `test-results`. Now, the "expected" attachment pointing to the
file in `__screenshots__` is also created. This will help any reporters
that would like to know the "expected" path, for example to do a manual
accept/decline of the baseline.

Fixes #30693.
2024-07-18 02:42:44 -07:00
Yury Semikhatsky 3f15fe8518
feat(reporter): links in attachment names, attachments name only (#31714)
* Allow calling `test.info().attach('My text');` without options (no
path nor body).
* Highlight links in attachment names:

<img width="992" alt="image"
src="https://github.com/user-attachments/assets/770e7876-3e43-4434-8cf1-194ad6ae5819">

Fixes https://github.com/microsoft/playwright/issues/31284
2024-07-17 09:30:49 -07:00
Dmitry Gozman f4399f7f06
fix(toHaveScreenshot): sanitize attachment names and paths (#31712)
... unless an array of file-system-friendly parts is provided.

Motivation: attachment name is used as a file system path when
downloading attachments, so we keep them fs-friendly.

References #30693.
2024-07-17 07:08:43 -07:00
Simon Knott 8eab28d858
fix(list reporter): print step ends in non-TTY mode (#31703)
When used in a terminal, the `list` reporter prints out information
about test steps to help debugging. In non-TTY environments like GitHub
Actions, currently it doesn't.

This PR changes that, so that in non-TTY environments you'll see the
"step end" messages appearing, but not the "step begin" messages. This
is a good middleground, because it helps the user understand test
progress, without being too verbose.

Closes https://github.com/microsoft/playwright/issues/31674
2024-07-17 13:36:37 +02:00
Max Schmitt ed6abf86c7
fix(expect): throw unsupported error when using this.equals() in expect (#31723) 2024-07-17 13:22:00 +02:00
Max Schmitt e11c0c0cbb
fix(connect): annotate internal api calls correctly (#31715) 2024-07-17 09:00:47 +02:00
Dmitry Gozman 6ee8f1de2d
Revert "chore: move artifacts recording to TestLifecycleInstrumentation (#30935)" (#31686)
This reverts commit ba5b460444.
2024-07-15 07:01:51 -07:00
Dmitry Gozman 074cc7d467
Revert "feat(trace): record trace upon browser closure (#31563)" (#31677)
This reverts commit bc27ca225e. Considered
too risky.
2024-07-15 01:08:51 -07:00
Dmitry Gozman a1f82b0bb6
fix(trace): do not corrupt test runner actions when no library trace is present (#31564)
Recent logic that matches either by `stepId` or by `apiName`+`wallTime`
did not account for "no library trace" scenario.
2024-07-10 09:12:06 -07:00
Pavel Feldman f374f8db38
chore: follow up to the attachments preview change (#31598) 2024-07-09 09:58:59 -07:00
Álvaro Martínez 00131c1e3f
feat(trace-viewer): display text attachments in ui mode (#31215) 2024-07-08 11:16:14 -07:00
Dmitry Gozman bc27ca225e
feat(trace): record trace upon browser closure (#31563)
Retaining traces in the following scenarios:
- browser crash;
- manual `browser.close()`;
- implicit `browser.close()` from the `browser` fixture upon test end.

This does not affect the library, where `browser.close()` will not
retain the trace and will close the browser as fast as possible.

References #31541, #31535, #31537.
2024-07-06 11:34:34 -07:00
Max Schmitt 1132667ffe
chore: bump TypeScript to v5.5 (#31532) 2024-07-05 16:31:08 +02:00
Dmitry Gozman 1c69d3e175
chore: update flakiness metadata for better ui presentation (#31528) 2024-07-03 08:39:53 -07:00
Pavel Feldman f46ae15500
test(clock): fix clock mode bots (#31472) 2024-06-28 11:46:28 -07:00
Pavel Feldman da441347e2
fix(runner): do not run beforeEach hooks upon skip modifier (#31426)
Fixes https://github.com/microsoft/playwright/issues/31425
2024-06-25 10:47:37 -07:00
Pavel Feldman 122818c62c
feat: allow boxing and titling fixtures, simulate context fixture deps (#31423)
Fixes https://github.com/microsoft/playwright/issues/31411
2024-06-24 21:43:43 -07:00
Max Schmitt 47fb9a080d
fix(test-runner): don't add slow annotation twice (#31414) 2024-06-24 23:34:17 +02:00
Dmitry Gozman 6ae9adfa4f
test: add "clock" to the metadata for dashboard traceability (#31370) 2024-06-18 14:56:08 -07:00
Dmitry Gozman 2ae2fb421c
chore: roll stable test runner to 1.45.0-beta-1718411373000 (#31326) 2024-06-14 19:43:35 -07:00
Dmitry Gozman b9106a4c42
fix(test runner): do not use @babel/plugin-transform-dynamic-import (#31285)
Historically, this plugin was important to translate dynamic imports
into require calls so that we can intercept them and transpile.

This is not needed anymore with ESM loader enabled by default, so we can
avoid this transformation and support dynamic imports of ESM-only
packages/files.

Fixes #17075, fixes #23255, fixes #31140, references #23662.
2024-06-13 06:18:44 -07:00
Yury Semikhatsky f1475fa644
chore: trim multiline step titles to first line (#31269)
Fixes https://github.com/microsoft/playwright/issues/31266
2024-06-12 08:24:12 -07:00
Max Schmitt 2d7bbe4d73
test: unflake should pass "key" attribute from JSX in variable (#31141) 2024-06-04 17:57:57 +02:00
Pavel Feldman 8bfd0eb6e4
chore: introduce clock test mode (#31110) 2024-05-31 14:44:26 -07:00
Mathias Leppich 5708148496
fix(merge-reports): only change test ids when needed (#31061)
When merging blob reports test ids are patched to make sure there is no
collision when merging reports that might have overlapping test ids.
However, even if you were merging reports that had no overlapping ids,
all test ids will be modified, which is an undesirable side effect.

This PR only modify test ids when the same test id has already been used
in a previous blob report.

----

This change is also part of
https://github.com/microsoft/playwright/pull/30962
2024-05-30 11:29:20 -07:00
Dmitry Gozman ba5b460444
chore: move artifacts recording to TestLifecycleInstrumentation (#30935)
The spirit of this change is reverting #23153. Since that time, we have
moved tracing and `artifactsDir` lifetime into the test runner, so the
reason for revert is mitigated.

Fixes #30287, fixes #30718, fixes #30959.
2024-05-29 18:05:17 -07:00
Yury Semikhatsky a11a6d9874
fix(blob): make sure tele reporters do not override ids (#31056)
Fixes https://github.com/microsoft/playwright/issues/31023
2024-05-28 16:37:11 -07:00
amankagithub 0e0b426e47
feat(html): filter test cases by annotation text in HTML report (#30751) 2024-05-24 13:12:16 -07:00
Yury Semikhatsky 9884c851ff
feat(expect): expose expect timeout (#30969)
Fixes https://github.com/microsoft/playwright/issues/30583
2024-05-24 08:56:43 -07:00
Yury Semikhatsky a106428114
feat(config): respectGitIgnore option (#30972)
Fixes https://github.com/microsoft/playwright/issues/30553
2024-05-23 12:05:02 -07:00
Dmitry Gozman eed6a10f3f
fix(tracing): speed up trace recording (#30946)
This includes two major changes:
- reuse `SerializedFS` for live test runner tracing;
- merge scheduled `appendFile` operations into a single `fs` call.

In some cases, this improves performance of UI mode by 61% and
performance of `trace: on` mode by 38%. Note that performance
improvement on the average test will not be as noticeable.

References #30875, #30635.
2024-05-22 15:19:32 -07:00
Pavel Feldman 964fe66ccc
chore: hint at unroute for handle errors (#30949) 2024-05-22 08:54:19 -07:00