Commit graph

111 commits

Author SHA1 Message Date
Andrey Lushnikov b7f6a98deb
devops: use a helper script to tag and push docker images (#3737)
It turns out that the new `docker push --all-tags` is not yet available
in the latest docker distribution on Ubuntu 20.04.

This provides a helper script to tag and publish the docker image.
2020-09-02 14:28:01 -07:00
Andrey Lushnikov df12264655
devops: use ubuntu 20.04 to build & publish docker releases (#3736)
We rely on `docker push --all-tags` flag that exists only in the
new version of docker.

The PR to docker with the flag: https://github.com/docker/cli/pull/2220
2020-09-02 11:46:24 -07:00
Andrey Lushnikov 216db2c9b3
devops: use microsoft/playwright-github-action@v1 everywhere (#3735)
Since we launch browsers as part of the `npm run build`, we need
to prepare environment.
2020-09-02 11:32:59 -07:00
Andrey Lushnikov 9d999ae1f0
devops: push all tags to docker registry (#3732)
Instead of pushing one specific tag, push repository itself.
This should push all the associated tags.
2020-09-02 10:33:54 -07:00
Yury Semikhatsky fc29623508
feat(screencast): use system ffmpeg on linux (#3724) 2020-09-02 08:47:43 -07:00
Andrey Lushnikov 47ea1e07ba
devops: another attempt to figure out docker publishing (#3721) 2020-09-01 17:32:16 -07:00
Andrey Lushnikov 1de4f7fa4c
devops: trigger docker devrelease when github action itself changes (#3719) 2020-09-01 16:59:03 -07:00
Andrey Lushnikov 2dc57c5b3d
chore: fix yaml syntax in the github action (#3718) 2020-09-01 16:54:05 -07:00
Andrey Lushnikov 1f93fb72d7
chore: fix devrelease workflow names (#3717) 2020-09-01 16:51:32 -07:00
Andrey Lushnikov ba7093cbe8
devops: try to use azure/docker-login instead of manual login (#3714)
Manual login appeared to be successful, but rejected docker push.

References #2926
2020-09-01 14:24:42 -07:00
Andrey Lushnikov ef5c87ccea
devops: switch docker publishing to a bash script (#3704)
It turns out Github Actions have docker pre-installed.

This patch unifies the codepaths that we use to build docker image
locally and to publish it from CI.

As a result, there's no extra `--prepare-context` argument in
`//docs/docker/build.sh` that was used solely on CI.

References #2926
2020-09-01 11:33:19 -07:00
Max Schmitt 1877c29834
devops: remove autoroll (#3684)
We have it in a own dedicated devops repo now.
2020-08-31 17:00:07 -07:00
Joel Einbinder 3cc91093a1
chore(testrunner): move out of the repo (#3687) 2020-08-29 08:27:32 -07:00
Pavel Feldman 555a8d0d10
fix(testrunner): include fixture teardown into timeout, add global timeout (#3685) 2020-08-28 19:27:49 -07:00
Dmitry Gozman e5ff283a40
fix(trace): only enable on separate tracing bots (#3677) 2020-08-28 13:50:06 -07:00
Pavel Feldman 6ffdd4dfa1
feat(testrunner): allow unexpected passes (#3665) 2020-08-28 00:32:00 -07:00
Pavel Feldman 15ec87dbcc
feat(testrunner): support --retries, flaky tests (#3649) 2020-08-26 16:32:47 -07:00
Pavel Feldman adc2a44113
infra: simplify test results collection (#3623) 2020-08-25 15:46:42 -07:00
Pavel Feldman 06dcf967ba
feat(testrunner): allow external reporters (#3603) 2020-08-24 19:16:20 -07:00
Joel Einbinder 012f9425bf
chore(test-runner): move into its own folder and typescript project (#3548) 2020-08-20 16:04:27 -07:00
Pavel Feldman 30f4c0c9fe
test runner: remove dependencies on playwright (#3539) 2020-08-19 17:47:24 -07:00
Max Schmitt e679b82339
fix(devops): auto roll tests (#3536) 2020-08-19 15:04:52 -07:00
Max Schmitt 8ea280090d
devops: first implementation of browser auto-roll bot (#3455)
Relates #3258

After a chat with @aslushnikov we add this in two iterations. The first one (this PR) is about scheduled runs for testing the latest tip-of-tree version of the browsers daily and the next PR is about opening an automated PRs once its passing (not sure if its worth to add for failing too) or sending Telegram/Slack notifications.

Current status is that Firefox (around 2 hours) and WebKit (around 3-4 hours) works.

Known issues which I fix before we merge:
- ~~Changes in `test/base.fixture.ts` will get extracted in #3453~~

Feel free to review, current blockers are before we can merge:
- ~~#3453~~
- potential git clone optimisation by aslushnikov
2020-08-17 11:23:46 -07:00
Pavel Feldman 262e886940
test: organize golden files under __snapshots__ folder (#3494) 2020-08-16 19:19:52 -07:00
Dmitry Gozman 9b52ca8676
chore: remove unused non-rpc code, test options, infra, bots (#3444) 2020-08-13 16:00:23 -07:00
Dmitry Gozman 84ca0120f8
test: upload output from headful bots (#3439) 2020-08-13 09:33:43 -07:00
Joel Einbinder d3677357b8
feat(testrunner): take the first argument as the test root dir (#3423) 2020-08-12 20:01:13 -07:00
Yury Semikhatsky f0fcdc8f51
test(firefox): make headful screencast tests work under xvfb (#3421) 2020-08-12 16:07:01 -07:00
Pavel Feldman a4a07c46c3
test: support --grep, --forbid-only (#3417) 2020-08-12 14:57:37 -07:00
Pavel Feldman 962ddc09c4
test: consolidate runner files (#3415) 2020-08-12 13:47:44 -07:00
Pavel Feldman 7e07634cc6
test: use mocha in ci/cd (#3406) 2020-08-12 11:48:30 -07:00
Dmitry Gozman 8bb6d73b44
feat(rpc): keep non-rpc linux bots for now (#3381) 2020-08-10 20:49:53 -07:00
Ross Wollman cbd33f9601
devops: avoid running publish on external contrib (#3257)
Everytime I sync my fork to upstream, I get failure notifications:

```
Run failed for master (3edfb2a)
Repository: rwoll/playwright
Workflow: devrelease
Duration: 10 minutes and 34.0 seconds
Finished: 2020-07-31 18:30:13 UTC
```

Since forks should never have the necessary secrets to publish
the npm and Docker packages, we limit the running of these jobs to avoid
noisy failures for contributors.
2020-07-31 14:09:36 -07:00
Andrey Lushnikov 421f6f48c7
devops: use playwright-github-action@v1 (#3221)
The playwright-github-action@v1 now points to v1.4.3, so we can
switch to it once again.
2020-07-30 17:47:39 -07:00
Andrey Lushnikov e7ddf868bf
devops: rename docker image tag for tip-of-tree images (#3222)
We currently tag tip-of-tree docker images with `dev` tag. It'll
be much nicer to have consistent taggin with our `npm` which tags
with `next` tag.

This patch removes the `dev` tag and starts using the `next` tag instead
for docker images. Since we haven't announced `dev` tag support
anywhere, I think it's fine to remove it rather than have both `next`
and `dev`.
2020-07-30 11:45:54 -07:00
Andrey Lushnikov 77b1c4b883
devops: enable Ubuntu 20.04 tests (#3178) 2020-07-29 11:36:38 -07:00
Yury Semikhatsky 79ab07bd0d
devops: install Media Pack on Windows bot (#3137) 2020-07-23 18:06:16 -07:00
Dmitry Gozman d234dac752
chore: support esm imports (#3125) 2020-07-23 15:14:36 -07:00
Joel Einbinder 773ee08e9d
chore(test): restore api coverage checks (#3068)
* chore(test): restore api coverage checks

* try to catch the correct error on the bots

* try to trick github into taking the new code

* uninstall coverage

* dgozman comments
2020-07-22 09:08:10 -07:00
Joel Einbinder 3c151d8f1d
fix(test): don't output babel's debug info on the bots (#3073) 2020-07-21 16:23:46 -07:00
Pavel Feldman b5f9985d3a
devops: make headful a matrix, collect test results (#3027) 2020-07-18 10:50:58 -07:00
Andrey Lushnikov 7d2078ef6f
devops: bake browsers into Docker image (#2990)
This patch bakes browsers into docker image.

Important observations:
- We now re-build docker image everytime we roll browsers.
- Docker image size almost doubles: from `225MB` to `496MB`.

References #2926
2020-07-17 16:51:39 -07:00
Pavel Feldman 9d9801192e
test: wire test commands to jest (#3014) 2020-07-17 16:09:27 -07:00
Pavel Feldman 424f11d165
test: convert some tests to the jest+fixtures (#2983) 2020-07-16 16:59:45 -07:00
Pavel Feldman ecc130c644
test: convert evaluation.spec to jest+fixtures (#2968) 2020-07-16 11:18:38 -07:00
Pavel Feldman 19cd96c46a
test: add the jest-circus experimental runner (#2962) 2020-07-15 16:52:41 -07:00
Andrey Lushnikov d58a57c4b2
devops: fix docker publishing (#2939)
- Repository name has to start with `public/` to be exported to dockerhub.
- Remove `DOCKER_USERNAME` to avoid unnecessary masking in our logs.
- Publish dev versions of Docker image only when changes to docker file
happen. (this is why NPM publishing and Docker publishing for dev
releases are now separate).
- Release publishing in two separate jobs to make them independent.

References #2926
2020-07-13 17:15:10 -07:00
Andrey Lushnikov b7d68d1cdb
devops: automate Docker image publishing (#2937)
This starts publishing our Docker images using Github Actions:
- on every commit, a new image will be pushed, tagged with a SHA of the
  commit, e.g.
  `mcr.microsoft.com/playwright:sha-fe997eca88d6de48378b05133dadfe032bd8b0ea`
- on every release, a new image will be pushed, tagged with the release
  version, e.g. `mcr.microsoft.com/playwright:v1.2.1`

**Note:** today we don't version our images and instead push them all as
`bionic`. This patch retains the tag - along with `latest`. Both
`bionic` and `latest` tags will point to the latest release.
2020-07-13 16:18:51 -07:00
Dmitry Gozman 2151757621
feat(rpc): run rpc tests in-process and out-of-process (#2929) 2020-07-13 15:00:20 -07:00
Dmitry Gozman 6d94c92053
feat(rpc): support no-serialization mode, run hook tests (#2925)
Added rpc_json_linux bots to excercise serialization - these
do not run hook tests.
2020-07-13 08:31:20 -07:00