Filip/playwright
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
12917 commits 34 branches 141 tags 223 MiB
Commit graph

1324 commits

Author SHA1 Message Date
jonghoonpark a6488c4a28
fix(html-reporter): add filter for anonymous describe (#30621) 
related issue: https://github.com/microsoft/playwright/issues/30475

## Motivation:
On https://github.com/microsoft/playwright/issues/30475, we found that
anonymous describe is rendered in html report

## Modification:
Make filter for anonymous describe

## Result:
anonymous describe will be filtered out.
Not render empty describe
Close https://github.com/microsoft/playwright/issues/30475 issue
 2024-05-02 09:54:44 -07:00
georg.dev 8173cdc485
fix(reporters): improve detection of output folder clashes (#30607) 
When comparing `outputDir` and html-reporter `outputFolder`, we now make
sure that both paths end with a forward-slash.

Fixes #28677

---------

Co-authored-by: Georg Unterholzner <georg.unterholzner@dynatrace.com>
 2024-05-02 08:32:57 -07:00
Yury Semikhatsky 3b7c4fac22
chore: add common env vars for junit and json reporters (#30611) 2024-05-01 10:16:49 -07:00
Yury Semikhatsky 6c827121bf
chore: roll stable test runner to 1.44 beta (#30600) 2024-04-29 18:35:21 -07:00
Pavel Feldman fb90797d73
fix(last-run): remove globalOutputDir (#30571) 2024-04-29 09:02:54 -07:00
Pavel Feldman b5dee9ecb7
feat(html): allow ctrl+clicking status (#30556) 2024-04-26 10:50:20 -07:00
Tan Li Hau 194479d90e
fix: pass key attribute from jsx to component test (#30426) 
When using the `key` attribute in jsx inside the test modules, it is not
serialised and passed to the browser in component test
 2024-04-26 09:49:44 -07:00
Yury Semikhatsky 3643fd456b
feat: outputFile option and PLAYWRIGHT_BLOB_OUTPUT_FILE env for blob (#30559) 
Reference https://github.com/microsoft/playwright/issues/30091
 2024-04-26 09:33:53 -07:00
Yury Semikhatsky b5aca9fca8
test: relax "should print if globalTimeout is reached" expectation (#30558) 2024-04-25 18:40:58 -07:00
Yury Semikhatsky 9a1b34a4b0
Revert "fix: shut down workers before reporter.onEnd (#30329)" (#30551) 
This reverts commit 3cea17abb6.

Reference https://github.com/microsoft/playwright/issues/30550

The test is preserved but marked as fixme.
 2024-04-25 14:21:54 -07:00
Dmitry Gozman dc0665210f
fix(reporters): properly determine flaky status for serial mode (#30529) 
There are plenty of edge cases in this area:
- interrupted test run;
- did not run because of serial mode failure;
- failed before `test.skip()` call (e.g. in `beforeEach`) in one of the
retries;
- and more...

Related issues: #28322, #28321, #27455, #17652.
Prior changes: #27762, #26385, #28360, probably more.

There is still some duplication between `outcome()` and similar logic in
`base.ts`, which might be deduped in a follow-up.

Fixes #28322.
 2024-04-25 13:39:14 -07:00
Yury Semikhatsky 5502a16e1d
fix(junit): merged report should preserve total duration (#30525) 
Fixes https://github.com/microsoft/playwright/issues/30518
 2024-04-25 13:34:17 -07:00
Yury Semikhatsky 714235d6c8
fix(merge): include command hash into the report name (#30528) 
Reference https://github.com/microsoft/playwright/issues/30091
 2024-04-25 13:34:00 -07:00
Pavel Feldman 4fa1030af6
chore: hide skipped tests by default (#30546) 
Fixes https://github.com/microsoft/playwright/issues/30540
 2024-04-25 12:48:41 -07:00
Yury Semikhatsky b074932a44
feat(blob): PLAYWRIGHT_BLOB_FILE_NAME env variable (#30530) 
Reference https://github.com/microsoft/playwright/issues/30091
 2024-04-25 09:45:38 -07:00
Pavel Feldman a2eb43b335
feat(runner): allow running last failed tests (#30533) 
Fixes: https://github.com/microsoft/playwright/issues/30506
 2024-04-25 08:15:27 -07:00
Dmitry Gozman a6f0a89169
test: unflake "should work with custom reporter" (#30526) 2024-04-24 20:03:39 -07:00
Pavel Feldman ff3d3ae8f2
fix(ui): print config and global setup errors (#30531) 
Fixes: https://github.com/microsoft/playwright/issues/30513
 2024-04-24 18:54:48 -07:00
Yury Semikhatsky f1f3929a67
chore: get rid of ConfigInWorker, use FullConfig instead (#30517) 
Addressing API review feedback.
 2024-04-24 14:14:05 -07:00
Pavel Feldman e91d372544
fix(ct): allow importing components from node_modules (#30493) 
Fixes https://github.com/microsoft/playwright/issues/29854
 2024-04-24 12:32:28 -07:00
Yury Semikhatsky 73c12f1f77
fix: increase global timeout for a test (#30491) 
1s is not enought on CI to start running the tests sometimes.
 2024-04-23 16:10:30 -07:00
Pavel Feldman 4514b7e3ed
chore: use original test ids in html report (#30486) 
Fixes https://github.com/microsoft/playwright/issues/30430
 2024-04-23 14:28:27 -07:00
Dmitry Gozman b52cd7e4dc
test: make sure skipped tests in serial mode are reported (#30484) 
This is a regression test for #28321, in preparation for changing some
skipped/flaky test status logic.
 2024-04-23 10:51:27 -07:00
Pavel Feldman f5ca5241f3
feat(esm): support import attributes (#30482) 
Fixes https://github.com/microsoft/playwright/issues/30473
 2024-04-23 10:13:01 -07:00
Dmitry Gozman 18dcd6adff
test: make expectations for some reporter tests readable (#30470) 2024-04-22 18:29:26 -07:00
Dmitry Gozman b0fbe058ae
feat: TestProject.ignoreSnapshots (#30466) 2024-04-22 16:16:38 -07:00
Dmitry Gozman b9e5a934ee
test: make expectations readable for more step reporting tests (#30468) 2024-04-22 15:30:51 -07:00
Yury Semikhatsky 1d786c804d
chore: generate globalTimeout and maxFailures errors in runner (#30467) 
Generate 'global timeout' and 'max failures' errors in the runner. Avoid
reading `config.globalTimeout` and `config.maxFailures` in the base
reporters.

Reference https://github.com/microsoft/playwright/issues/29768
 2024-04-22 14:00:16 -07:00
Playwright Service dd1a9f5e96
feat(chromium): roll to r1115 (#30449) 2024-04-20 18:42:22 +02:00
Yury Semikhatsky 8c181f7e2b
fix: rebuild project tree from scratch when listing tests (#30407) 
Instead of filtering tests assuming there are no two projects with same
name we always rebuild test tree from scratch and restore previos test
results in the list mode.

Fixes https://github.com/microsoft/playwright/issues/30396
 2024-04-18 16:49:07 -07:00
Yury Semikhatsky cda1c945af
test: web assertion inside expect.poll (#30391) 
Reference https://github.com/microsoft/playwright/issues/30322
 2024-04-16 17:37:25 -07:00
Yury Semikhatsky 3bdbe4284e
fix(steps): make expect.toPass and expect.poll step containers (#30389) 
Fixes https://github.com/microsoft/playwright/issues/30322
 2024-04-16 16:18:37 -07:00
Yury Semikhatsky 73fce8fb98
chore: replace Zones with AsyncLocalStorage (#30381) 
Reference https://github.com/microsoft/playwright/issues/30322
 2024-04-16 12:51:20 -07:00
Yury Semikhatsky 3cea17abb6
fix: shut down workers before reporter.onEnd (#30329) 2024-04-16 09:49:11 -07:00
Yury Semikhatsky 6ad48fb594
chore: make step test expectations readable (#30362) 
Reference https://github.com/microsoft/playwright/issues/30322
 2024-04-13 20:07:18 -07:00
Yury Semikhatsky c4418875d3
chore: do skip failing test (#30361) 2024-04-12 15:47:13 -07:00
Yury Semikhatsky dc1077c5a2
test: steps inside expect.toPass (#30359) 
Reference https://github.com/microsoft/playwright/issues/30322
 2024-04-12 14:51:10 -07:00
Yury Semikhatsky 1ef85015f3
fix(merge): preserve static annotations on the tests that did not run (#30348) 
Fixes https://github.com/microsoft/playwright/issues/30260
 2024-04-11 16:17:57 -07:00
Max Schmitt 15b423106f
fix(ui-mode): do not loose run information after writing into testDir (#30312) 
Partially reverts https://github.com/microsoft/playwright/pull/30008
that started to reset all test results upon listing tests, including the
test that did just run and triggered re-listing.

https://github.com/microsoft/playwright/issues/30300.
 2024-04-10 19:01:51 +02:00
Max Schmitt b2ded9fed1
fix(expect): throw better received error when no element was found (#29890) 
Fixes https://github.com/microsoft/playwright/issues/29873
 2024-04-10 10:01:19 +02:00
Dmitry Gozman a6827772a5
fix(esmLoader): unref MessagePort in the main thread (#30271) 
This prevents the process from not exiting forever due to an open port.
 2024-04-06 14:25:57 -07:00
Dmitry Gozman e18a358bc6
test: check that ESM loader works without config with type:module (#30258) 2024-04-05 07:03:27 -07:00
Dmitry Gozman 5d810f1e07
chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250) 
Co-authored-by: Yury Semikhatsky <yurys@chromium.org>
 2024-04-04 11:26:03 -07:00
Dmitry Gozman d5907f4b13
feat(junit): includeProjectInTestName option (#30233) 
Fixes #30246.
 2024-04-04 11:04:51 -07:00
Yury Semikhatsky 81bcf2a53b
docs: document FullReporter API (#30186) 
* FullConfig split into FullConfig and ConfigInWorker
* FullProject split into FullProject and ProjectInWorker
* Tests can only access {Config, Project}InWorker, while reporters can
only access Full{Config,Project}. This will allow in the future to
change the reporter API independently from the tests.
* Added documentation for the 4 classes.
 2024-04-02 16:15:02 -07:00
Hansanto a950a7b86b
feat: Support intervals arg for toPass (#30037) 
Fixes #30040

# Context

In the `playwright.config` file, we can custom the default `timeout`
property for `expect.toPass` by the property:

```ts title='playwright.config.ts'
export default defineConfig({
  // ...
  expect: {
    toPass: {
      timeout: 60_000,
    },
  },
});
```

However, we can't customize the `intervals` option. So the default value
is always `[100, 250, 500, 1000]`.

# Goal

Add the possibility to customize the intervals option from the
`playwright.config` file.

```ts title='playwright.config.ts'
export default defineConfig({
  // ...
  expect: {
    toPass: {
      timeout: 60_000,
      intervals: [1000, 2000, 3000]
    },
  },
});
```
 2024-03-29 10:17:29 -07:00
Yury Semikhatsky 3001c9ac73
fix: preserve test declaration order in html and merged report (#30159) 
* Add `Suite.entries` that returns tests and suites in their declaration
order
* Exposed `Suite.type` and `TestCase.type` for discriminating between
different entry types.
* Blob report format is updated to store entries instead of separate
lists for suites and tests.
* Bumped blob format version to 2, added modernizer.

Fixes https://github.com/microsoft/playwright/issues/29984
 2024-03-29 10:12:33 -07:00
Yury Semikhatsky 16318ea715
chore: remove unused botName from types (#30172) 2024-03-28 16:16:15 -07:00
Max Schmitt 599185dd07
feat(ui-mode): show native tags in test tree (#30092) 
This brings up the question that we would show the tag name twice if its
a tag in a title. This would be aligned to how HTML report is doing it.

Fixes https://github.com/microsoft/playwright/issues/29927

---------

Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
 2024-03-26 01:06:22 +01:00
Dmitry Gozman 051afb9ce0
fix(test runner): update fixture teardown error message (#30109) 
With the recent change that gave after hooks a separate timeout, fixture
teardown does not imply that "test finished successfully, but fixture
teardown was slow".
 2024-03-25 17:04:03 -07:00
Max Schmitt 65d82797f1
fix(ct): ct ID clash on similar imports (#30108) 
This restores the [best
effort](https://github.com/microsoft/playwright/pull/29407/files#diff-8dd3534dc5013c3779edeaded71324b0dd1c1807668f3c6d9e9a1aab1c20ae91L152)
logic if its a relativ path.

Fixes https://github.com/microsoft/playwright/issues/30085

---------

Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Pavel Feldman <pavel.feldman@gmail.com>
 2024-03-25 23:50:45 +01:00
Dmitry Gozman 911d8effb9
feat(test runner): do not mock tty in the worker process (#30107) 
This was historically done to make `console.log()` have colors. However,
this makes any other code that checks `process.stdout.isTTY` incorrectly
assume real TTY support.

Node18 and Node20 now respect `FORCE_COLOR=1` in console, so our default
behavior of forcing colors in the worker process just works out of the
box. See https://github.com/nodejs/node/pull/48034.
 2024-03-25 15:31:58 -07:00
Max Schmitt 253a2f9a9c
chore: address UI Mode keyboard shortcut feedback (#30088) 
Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
 2024-03-25 19:48:20 +01:00
Dmitry Gozman 7a3c002944
fix(ct): allow importing json files (#30067) 
While we are not fully resolving imports during compilation, do not
treat `json` as a non-importable asset.

Fixes #29926.
 2024-03-25 07:41:54 -07:00
Pavel Feldman 1539cde034
chore: allow passing path to config to the test server (#30068) 2024-03-22 16:04:59 -07:00
Yury Semikhatsky a511731f55
fix(html): keep projects from different bots separate (#30051) 2024-03-22 10:02:00 -07:00
Dmitry Gozman 3e73a6ce69
feat(test runner): show multiple errors, at most one per stage (#30024) 
Previously, there was at most one "hard error", as opposite to multiple
"soft errors". This was done to preserve the historic behavior at the
time of introducing multiple `TestInfo.errors`.

With this change, every user callback that is executed `withRunnable()`
can throw an error and/or timeout, and both of these will end up in
`TestInfo.errors`.

Additionally, there is at most one "unhandled exception" error, to avoid
flooding the report with mass failures.

Drive-by: remove boolean arguments from `_failWithError()`.

Fixes #29876.
 2024-03-20 21:01:30 -07:00
Pavel Feldman 0a22a86e2e
chore: prepare to reuse test server from ui mode (5) (#30005) 2024-03-19 14:08:21 -07:00
Pavel Feldman 54aca430b0
chore: prepare to reuse test server from ui mode (4) (#29995) 2024-03-19 13:00:49 -07:00
jonghoonpark b929c0d8f7
feat(ui-mode): add key shortcuts for playwright uI test runner (#29868) 2024-03-19 12:36:42 +01:00
Dmitry Gozman be1af15d57
feat: always enable ESM loader with the new API (#29991) 
It does not require a process restart anymore, so safe to enable.

Fixes #29747.
 2024-03-18 21:54:25 -07:00
Dmitry Gozman b41b802662
fix(test runner): avoid dependency tracking colliding between esm and cjs (#29994) 
When collecting dependencies both from CJS loader and from ESM loader,
the latter would overwrite the dependencies set instead of appending.

Also make sure cts/cjs/mts/mjs are all supported equally.

References #29747.
 2024-03-18 17:17:58 -07:00
Max Schmitt a6d1fb93de
fix(trace-viewer): encode attachment filenames as UTF-8 (#29993) 
Fixes https://github.com/microsoft/playwright/issues/29967

Tested in Firefox, Chromium, and Safari. This now leads to "good
attachment names" in Chromium and Safari, for Firefox, it won't produce
attachments, it will open them inline, but this is not a regression, was
before like that already.

See here for the spec:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition#filename_2

---------

Signed-off-by: Max Schmitt <max@schmitt.mx>
 2024-03-18 21:26:45 +01:00
Yury Semikhatsky 35db70ea1d
chore: trim file names longer than 60 chars (was 100) (#29725) 
Test-specific output dir and snapshot names are trimmed to 60 chars
instead of just 100. The snapshot names are still trimmed at 100 chars
for backwards compatibility.


Reference https://github.com/microsoft/playwright/issues/29719
 2024-03-18 12:53:15 -07:00
Pavel Feldman ef4438ee99
chore: prepare to reuse test server from ui mode (2) (#29966) 2024-03-18 09:59:02 -07:00
Max Schmitt 048d6669fd
fix(reuse): reset Origin Private File System API (#29921) 
Fixes https://github.com/microsoft/playwright/issues/29901

This clears the Origin Private File System API when using VSCode
extension.
 2024-03-15 17:40:46 +01:00
Yury Semikhatsky 166d2d4fde
chore: separate results for repeated snapshot names (#29880) 
Reference #29719
 2024-03-12 16:10:43 -07:00
Yury Semikhatsky 59228f19ce
fix: tty.WriteStream method stubs for process.stdout/stderr (#29865) 
Fixes #29839
 2024-03-11 10:11:16 -07:00
Yury Semikhatsky 7f6f17d1c3
chore: simplify toMatchSnapshot names calculation (#29862) 
Reference https://github.com/microsoft/playwright/issues/29719
 2024-03-10 11:11:35 -07:00
Dmitry Gozman d214778548
chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857) 2024-03-08 15:19:36 -08:00
Dmitry Gozman 006ee7f3b0
feat: separate timeout for after hooks (#29828) 
Instead of sharing the timeout with the test, and then extending it when
test times out, we give after hooks a separate timeout.
 2024-03-06 12:31:54 -08:00
Sarkis Matinyan 0c3f60e95e
fix(ui): show stack frames in ui mode (#29560) 
#29558, #29500
 2024-03-06 12:29:35 -08:00
Lukas Bockstaller 425f737eb6
feat: exposes tags in testInfo (#29794) 
Fixes #29793.

---------

Signed-off-by: Lukas Bockstaller <lukas.bockstaller@everest-erp.com>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
 2024-03-06 08:33:06 -08:00
Dmitry Gozman 1d4bdc6898
chore(test runner): make runAsStage throw and catch errors explicitly (#29814) 2024-03-05 16:35:11 -08:00
Dmitry Gozman a3ed799cd5
fix(tsconfig): when extending, retain pathsBasePath from the original config (#29822) 
This fixes a case where we incorrectly used the final config's base path
when resolving relative path mappings in the absence of the baseUrl.

Fixes #29816.
 2024-03-05 16:34:39 -08:00
Dmitry Gozman abfd2c4e66
feat(ui mode): text filter should filter by explicit tags (#29821) 
Fixes #29815.
 2024-03-05 10:58:55 -08:00
Dmitry Gozman 5eb8fea616
chore: rewrite worker main through TestInfoImpl._runAsStage (#29644) 2024-03-04 19:36:58 -08:00
Pavel Feldman ef924c14e7
chore: do not use project id in telereporter (#29776) 2024-03-01 21:44:08 -08:00
Pavel Feldman bbcc3c1238
chore: remove private config usage from telereporter (#29771) 2024-03-01 13:14:12 -08:00
Pavel Feldman baf2cdf936
fix(ct): stop-gap for shared file import (#29744) 
Fixes: https://github.com/microsoft/playwright/issues/29739
 2024-02-29 19:13:32 -08:00
Pavel Feldman d9a00342c8
fix(HEAD): revert GET->HEAD migration, net-effect was negative (#29738) 
Fixes: https://github.com/microsoft/playwright/issues/29732
 2024-02-29 09:02:05 -08:00
Stevan Freeborn 52b803ecf5
feat(trace mode): add on-first-failure mode for traces (#29647) 
Implements the changes suggested in #29531
 2024-02-28 14:39:18 -08:00
Yury Semikhatsky d8c16c1edf
chore: roll stable runner to 1.42.0-beta-1708998235000 (#29675) 2024-02-27 08:36:03 -08:00
Yury Semikhatsky 1c8e8bea7e
chore: roll stable test runner to 1.42.0-beta-1708994059000 (#29672) 2024-02-26 17:39:25 -08:00
Pavel Feldman 9d0430fa26
chore: strengthen linting (#29669) 2024-02-26 16:52:28 -08:00
Pavel Feldman a3aea813bc
chore: implement server-based list files (#29633) 2024-02-23 08:29:44 -08:00
Dmitry Gozman f9aebda5db
fix(fixtures): attribute teardown step to the right TestInfo instance (#29523) 2024-02-20 12:35:43 -08:00
Yury Semikhatsky f3fde992eb
chore: remove --project-grep, add wildcard support (#29537) 
Reference #15128
 2024-02-16 19:18:00 -08:00
Dmitry Gozman 269a293ba1
chore(test runner): allow TestInfoImpl without a TestCase (#29534) 
This will be useful to run `beforeAll`/`afterAll` hooks with a separate
`TestInfo` instance, as well as run use helpers like
`_runAndFailOnError()` during scope teardown.
 2024-02-16 12:43:13 -08:00
Dmitry Gozman dc9cddde95
chore: do not rely on zones for predefined steps (#29485) 
This fixes some edge cases where fixtures and steps inside them were
attached to the wrong parent (see the new test).
This will also allow to replace some `runAsStep` calls with a flat list
of tasks to do that do not rely on lexical scope.
 2024-02-15 12:49:08 -08:00
Dmitry Gozman 08afb34c14
chore(test runner): make timeout error an Error (#29515) 2024-02-15 11:38:13 -08:00
Dmitry Gozman bd5403dcad
chore: remove --tag and tagFilter (#29468) 
This makes new tags filtered by the `grep`. New `TestCase.tags` API now
includes both old and new tags.
 2024-02-15 11:37:16 -08:00
Dmitry Gozman bc2d4302f5
test: fix a broken reporter test (#29514) 
It failed to print an error with missing stack, e.g. a timeout error.
 2024-02-15 09:45:48 -08:00
Pavel Feldman 217c0618df
chore: rename find-related-tests to find-related-test-files (#29483) 2024-02-13 15:40:49 -08:00
Pavel Feldman 7aef5249d8
chore: pass the private data to the runner via config (#29467) 2024-02-13 09:34:03 -08:00
Dmitry Gozman f605a5009b
fix(esmLoader): do not serialize the whole compilation cache after each import (#29464) 
Instead, send the newly created cache entry only.

References #29418.
 2024-02-12 16:39:53 -08:00
Yury Semikhatsky 7834beb932
chore: extract --project-grep option for matching with regex (#29445) 
Reference https://github.com/microsoft/playwright/issues/15128
 2024-02-10 11:19:00 -08:00
Pavel Feldman 586d14f02c
chore: add find-related-tests command (#29439) 2024-02-09 19:02:42 -08:00
Yury Semikhatsky 7a86bf4af6
chore: delete store.ts (#29423) 2024-02-08 12:55:18 -08:00
Pavel Feldman 84dea09cb9
fix(ct): resolve components used in tests during the vite build (#29407) 2024-02-07 20:39:45 -08:00