Introduces an `--only-changed [base ref]` option.
`playwright test --only-changed` filters the test run to only run test
suites that have uncommitted changes.
`playwright test --only-changed=foo` runs only tests that were changed
since commit `foo`.
In pull request CI, this can be used to run changed tests first and fail
fast: `--only-changed=$GITHUB_BASE_REF`.
During local development, it can be used to quickly filter down to the
touched set of tests suites.
In some rare usecases, this can also help to cut down on CI usage for
pull requests. Tread with caution though.
File dependencies are taken into account to ensure that if you touched a
utility file, all relevant tests are still executed.
Closes https://github.com/microsoft/playwright/issues/15075
This reverts commit 0aa2f06f68.
Discussed the new permission in the API review and decided not to
proceed with the feature as we are not ready to commit to supporting it
yet:
* the API is Chromium specific
* the API is still experimental
* there is no clarity to what extend the screen manipulation APIs will
work in old headless which is our main test environment
We'll keep an eye on the demand for the feature and may get back to
implementing it in the future.
Reference: https://github.com/microsoft/playwright/issues/27198
Fixes misspellings identified by the [check-spelling
action](https://github.com/marketplace/actions/check-spelling).
The misspellings have been reported at
https://github.com/jsoref/playwright/actions/runs/10015023629#summary-27685777352
The action will report that the changes in this PR would make it happy:
https://github.com/jsoref/playwright/actions/runs/10015023971#summary-27685778305
---
I understand that the commit messages will need to be reworded to match
house style. For the time being, these are merely noting the changes
they contain so that when I rebase or need to drop things, I can. --
I've already rebased once as someone fixed one of the items that my
draft work was going to fix.
---
## Testing
* The tests _mostly_ passed when I managed to trigger them, but there
were a handful of things that I didn't quite understand
* There are a large number of warnings relating to a bad interaction
between any workflow that uses this local action
b535139b32/.github/actions/run-test/action.yml (L74-L80)
and the action it calls -- I've opened Azure/login#474 asking them to
refactor their action so that it doesn't cause so much noise while
running this repository's tests
* I'm vaguely curious as to why this repository has a `branch`
constraint for its `pull_request` events in its workflows -- that
constraint gave me a number of additional headaches while trying to
prepare this branch for this PR.
---------
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
The following actions keep `noWaitAfter` option: `click`, `selectOption`
and `press`.
All other actions that used to have `noWaitAfter` now behave like it was
set to true, not waiting for follow-up navigations. In the docs, this
option is marked as completely ignored.
A small logic change was made to compensate for this behavior: when
waiting for the `hitTargetInterceptor`, we now race it against
navigations to avoid stalling when navigation stalls. Previously,
waiting for the interceptor was disabled when `noWaitAfter` was passed,
and since it's impossible to pass this option now, we mitigate by never
stalling instead.
Fixes#31469.
* added more explanation to the sharding section for it to be more
understandable what it is and what it's purpose.
* because from the explanation in the docs. - it sound like that a
"shard" is a machine that is provided by playwright in addition to the
current setup , like a managed machine, but in reality it is simply a
separate job in the CI, dividing tests to separate jobs is sharding,
which sounds a lot easier when you think about it this way because it
could be confusing at the beginning IMO.
Historically, this plugin was important to translate dynamic imports
into require calls so that we can intercept them and transpile.
This is not needed anymore with ESM loader enabled by default, so we can
avoid this transformation and support dynamic imports of ESM-only
packages/files.
Fixes#17075, fixes#23255, fixes#31140, references #23662.
"To learn more about generating tests check out or detailed guide on
Codegen."
Should it be 'our'? And why does it link back to itself? Should be
removed.
Signed-off-by: Xan <57809064+devxan@users.noreply.github.com>
The documentation for command line usage of `--grep` did not provide the
correct string used for comparisons. This has now been fixed to include
the project name.
Fixes#30895
- Documents `PLAYWRIGHT_FORCE_TTY` and `FORCE_COLOR` across terminal
reporters.
- New `PLAYWRIGHT_LIST_PRINT_STEPS`. Removes undocumented test-only
`PW_TEST_DEBUG_REPORTERS_PRINT_STEPS`.
- Replaces `PLAYWRIGHT_HTML_REPORT` with `PLAYWRIGHT_HTML_OUTPUT_DIR`
and `PW_TEST_HTML_REPORT_OPEN` with `PLAYWRIGHT_HTML_OPEN` for
consistency, supports older versions for backwards compatibility.
- New `PLAYWRIGHT_HTML_HOST`, `PLAYWRIGHT_HTML_PORT` and
`PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL`.
- New `PLAYWRIGHT_JUNIT_STRIP_ANSI` and
`PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME`.
- Removes `PW_HTML_REPORT` that was set for unknown reason.
Implements feature requested in
https://github.com/microsoft/playwright/issues/30457
The test runner treats flaky tests as failures when the flag is enabled,
but still reports flaky tests as flaky in the reporting interface. It
feels like something worth discussing as this behaviour makes sense to
me, but looked a bit odd to @BJSS-russell-pollock when I ran this past
him.
Closes#30457.
