Relates #3258
After a chat with @aslushnikov we add this in two iterations. The first one (this PR) is about scheduled runs for testing the latest tip-of-tree version of the browsers daily and the next PR is about opening an automated PRs once its passing (not sure if its worth to add for failing too) or sending Telegram/Slack notifications.
Current status is that Firefox (around 2 hours) and WebKit (around 3-4 hours) works.
Known issues which I fix before we merge:
- ~~Changes in `test/base.fixture.ts` will get extracted in #3453~~
Feel free to review, current blockers are before we can merge:
- ~~#3453~~
- potential git clone optimisation by aslushnikov
Everytime I sync my fork to upstream, I get failure notifications:
```
Run failed for master (3edfb2a)
Repository: rwoll/playwright
Workflow: devrelease
Duration: 10 minutes and 34.0 seconds
Finished: 2020-07-31 18:30:13 UTC
```
Since forks should never have the necessary secrets to publish
the npm and Docker packages, we limit the running of these jobs to avoid
noisy failures for contributors.
We currently tag tip-of-tree docker images with `dev` tag. It'll
be much nicer to have consistent taggin with our `npm` which tags
with `next` tag.
This patch removes the `dev` tag and starts using the `next` tag instead
for docker images. Since we haven't announced `dev` tag support
anywhere, I think it's fine to remove it rather than have both `next`
and `dev`.
* chore(test): restore api coverage checks
* try to catch the correct error on the bots
* try to trick github into taking the new code
* uninstall coverage
* dgozman comments
This patch bakes browsers into docker image.
Important observations:
- We now re-build docker image everytime we roll browsers.
- Docker image size almost doubles: from `225MB` to `496MB`.
References #2926
- Repository name has to start with `public/` to be exported to dockerhub.
- Remove `DOCKER_USERNAME` to avoid unnecessary masking in our logs.
- Publish dev versions of Docker image only when changes to docker file
happen. (this is why NPM publishing and Docker publishing for dev
releases are now separate).
- Release publishing in two separate jobs to make them independent.
References #2926
This starts publishing our Docker images using Github Actions:
- on every commit, a new image will be pushed, tagged with a SHA of the
commit, e.g.
`mcr.microsoft.com/playwright:sha-fe997eca88d6de48378b05133dadfe032bd8b0ea`
- on every release, a new image will be pushed, tagged with the release
version, e.g. `mcr.microsoft.com/playwright:v1.2.1`
**Note:** today we don't version our images and instead push them all as
`bionic`. This patch retains the tag - along with `latest`. Both
`bionic` and `latest` tags will point to the latest release.
We now commit protocol.ts files during the roll.
New utils/roll_browser.js helps with that.
This makes our installation very shallow:
- build installer;
- download browsers.
This patch:
- removes releasing from Travis CI
- sets up a new GH Action that releases @next version from tip-of-tree
Once this GH Action proves to be working, we'll setup a `publish_release.yml`
workflow that will be triggered only by **release** github events
and that will publish released version with `LATEST` tag.
NOTE: this workflow does not actually run publishing - we're doing
`--dry-run` for now to see how it works in
`//utils/publish_all_packages.sh`.
We need to commit `package-lock.json` to repository to ensure
reproducible builds.
This also starts using [`npm ci`](https://docs.npmjs.com/cli/ci.html) to
install dependencies.
Drive-by: remove non-functioning `rebase` workflow.
We need to commit `package-lock.json` to repository to ensure
reproducible builds.
This also starts using [`npm ci`](https://docs.npmjs.com/cli/ci.html) to
install dependencies.
Drive-by: remove non-functioning `rebase` workflow.
This patch re-implements matching and reporting for test runner.
Among other improvements:
- test failures now show a short snippet from test
- test failures now explicitly say what received and what was expected
- `expect.toBe()` now does text diff when gets strings as input
- `expect.toEqual` now does object diff
