Commit graph

264 commits

Author SHA1 Message Date
Andrey Lushnikov f2cc439d8b
chore: move electron back from FYI bots to CQ1 bots (#6883)
This is according to the discussion at the meeting.
2021-06-03 15:03:10 -07:00
Andrey Lushnikov b19b2dc3ad
devops: introduce manual @next NPM publishing (#6881) 2021-06-03 14:50:40 -07:00
Andrey Lushnikov 17b6f06b98
feat: install media pack on windows with npx playwright install-deps (#6836)
Chromium on Windows requires Media Pack to be installed. This patch
moves media pack installation under the `npx playwright install-deps`
umbrella.
2021-06-02 10:47:10 -07:00
Andrey Lushnikov 919d258356
feat: support npx playwright install chrome (#6835)
This will install latest-and-greatest chrome stable.
2021-06-01 23:16:55 -07:00
Andrey Lushnikov c9613b36f9
devops: introduce "FYI" test bots (#6834)
The "FYI" bots never run on PRs - even when you put a "CQ1" label.
These tests are only run on commits to master / release branches and
their results are observed on the flakiness dashboard.
2021-06-01 18:45:45 -07:00
Andrey Lushnikov cb4adb1446
feat: install chrome-beta via cli (#6831)
This patch starts introduces `npx playwright install chrome-beta`
command and switches our Chrome Beta tests to it.
2021-06-01 18:26:12 -07:00
Andrey Lushnikov 17585a369e
devops: do not run tests for docs changes (#6825) 2021-06-01 13:55:48 -07:00
Dmitry Gozman 8fefac9b8c
test: roll to folio@0.4.0-alpha21 (#6789) 2021-05-28 17:03:18 -07:00
Andrey Lushnikov c60974d922
feat: do not rely on chocolatey to install Google Chrome Beta (#6735) 2021-05-25 17:32:08 -07:00
Andrey Lushnikov 2cdf1e122c
chore: add more logging while installing browsers (#6688) 2021-05-20 15:47:22 -07:00
Andrey Lushnikov 33a505b1e5
chore: add logging for installation steps (#6565)
This patch:
- adds debug logging
- sets up our workflows to use `DEBUG=pw:install` on CI
2021-05-18 17:38:02 -07:00
Manu Chambon 2697f8380f
devops(docker): upgrade to node 16 (#6498) 2021-05-12 09:23:38 -07:00
Andrey Lushnikov a04c54ac28
devops: do not run workflows when all changes are browser-only (#6520)
Drive-by: rename primary/secondary tests to `tests 1` and `tests 2`
2021-05-11 17:25:05 -07:00
Andrey Lushnikov bf81a284dd
devops: run less tests on each PR (#6518)
With this patch, we have the following behaviors:
- For PR's, we will run only Linux tests for Chromium, Firefox and
  Webkit
- For commits to master and release-*, we'll run all the tests

To run all tests on PR, assign a "CQ1" label to the PR.
2021-05-11 17:01:02 -07:00
Dmitry Gozman b01ccc282f
test: roll to folio@0.4.0-alpha11 (#6496) 2021-05-11 06:40:06 -07:00
Dmitry Gozman 573327b7b5
test: roll to folio@0.4.0-alpha8 (#6451) 2021-05-07 15:25:55 -07:00
Andrey Lushnikov 298b7aef2a
devops: install Google Chrome Beta testers (#6389) 2021-05-06 10:07:00 -07:00
Max Schmitt 2945daca7a
devops: fixed broken GitHub Actions workflow (#6399) 2021-05-04 12:19:50 +02:00
Andrey Lushnikov 4cd5673cc7
chore: add debugging information on bots to trace unzipping (#6395)
We occasionally stuble upon unzipping error: "end of central directory
record signature not found" that's coming from the underlying
unzip library.

This tracing should help us investigate what's going on here.
2021-05-03 16:52:54 -07:00
Max Schmitt 29164a625f
devops: use Node.js 12 on Windows bots (#6377) 2021-04-30 21:35:41 +02:00
Max Schmitt a59a494e64
chore: drop support for Node.js 10 (#6371) 2021-04-30 07:49:10 -07:00
Dmitry Gozman a4561310e8
test: set 20 minutes timeout for installation tests (#6363) 2021-04-29 11:12:03 -07:00
Max Schmitt 2333eb09a6
chore: adjust GitHub template envinfo command (#6359) 2021-04-29 09:43:41 -07:00
Andrey Lushnikov 23dfaf9e9b
feat: start downloading firefox-stable channel (#6177)
This patch adds support for downloading and running Playwright with
firefox-stable channel.

References #5993
2021-04-19 21:26:33 -07:00
Max Schmitt 6296d276f2
devops(gha): remove obsolete comment (#6234) 2021-04-19 13:02:32 -07:00
Andrey Lushnikov 27ed123a50
feat: remove core dumps from bots (#6231)
Turns out we haven't been uploading them as artifacts for quite some
time at least.
2021-04-19 13:02:06 -07:00
Andrey Lushnikov 2c34eaeaf9
devops: better upload flakiness dashboard upload script (#6176)
This patch moves some duplicated logic from YAML workflows to
the dashboard uploading script.
2021-04-19 08:39:10 -07:00
Yury Semikhatsky e4ae6503d7
fix(inspector): fall back to custom executable path for UI (#6214) 2021-04-16 15:19:44 -07:00
Andrey Lushnikov ad8b43467e
devops: trigger Firefox Stable builds (#6174)
References #5993
2021-04-09 22:27:02 -07:00
Dmitry Gozman bd8433ba49
test: cleanup various testing env variables (#6155) 2021-04-09 07:59:09 -07:00
Dmitry Gozman 310692b101
test: run page tests on electron bot (#6122) 2021-04-08 10:26:26 -07:00
Dmitry Gozman 36a546999e
test: roll to folio 0.3.21-alpha (#6108) 2021-04-06 16:09:54 -07:00
Andrey Lushnikov da3ddb07c0
devops: start uploading test reports from chrome stable runs (#6092) 2021-04-05 23:15:06 -07:00
Max Schmitt 6a767d1a9c
docs(docker): use focal by default (#5746) 2021-04-05 16:09:35 -07:00
Dmitry Gozman 5afe282fbb
test: move remaining files from old test/ directory (#6081) 2021-04-05 15:51:45 -07:00
Max Schmitt af48a8a1f1
devops: use ubuntu focal on bots and docs (#5951)
- drive-by: updated NPM Action
- replaced ubuntu-latest by ubuntu-20 so its pinned

Extracted from #5746
2021-04-05 12:39:15 -07:00
Dmitry Gozman 4f7e7450e2
test: migrate last tests to new folio (#6071) 2021-04-05 09:18:56 -07:00
Dmitry Gozman f21f47889e
test: migrate more page tests to folio (#6062) 2021-04-04 19:32:14 -07:00
Dmitry Gozman 0629922774
test: migrate page tests to new folio (#6054) 2021-04-02 14:23:42 -07:00
Dmitry Gozman be79b3883b
test: bring new folio and migrate small amount of tests to it (#5994) 2021-04-01 16:35:26 -07:00
Ross Wollman ca7cd7a677
devops: skip flakiness upload for forks (#5978)
Before this change, forks face a noise from pipeline
failures trying to upload to the flakiness dashboard.

Example: https://github.com/rwoll/playwright/runs/2199170557
2021-03-29 20:05:10 -07:00
Andrey Lushnikov 7678112242
tests: do not run video tests with Chrome Stable on Linux (#5931) 2021-03-23 22:18:49 -07:00
Pavel Feldman 1b802332d2
infra: remove shards from mac builds to remove 6 bots (#5928) 2021-03-23 14:23:23 -07:00
Pavel Feldman ad5c028f37
test(android): run selected page tests on android (#5879) 2021-03-18 19:31:54 -07:00
Pavel Feldman 141583c79f
infra(chrome_stable): add more bots (#5863) 2021-03-17 17:24:01 -07:00
Andrey Lushnikov ab4629af38
devops: add trigger workflow to deprecated webkit builds (#5836) 2021-03-16 01:31:32 -07:00
Pavel Feldman ea32ad2b09
infra(channel): add edge stable bot (#5825) 2021-03-15 16:00:52 -07:00
Pavel Feldman 1dd6bd3316
infra(channel): wire release channel to all tests (#5820) 2021-03-15 08:07:57 -07:00
Pavel Feldman a96d6a7dbb
feat: allow to pick stable channel (#5817) 2021-03-13 14:02:39 -08:00
Pavel Feldman c4578f19f2
chore: organize per-browser dependencies (#5787) 2021-03-11 20:22:50 -08:00
Pavel Feldman 7fa59f6d94
infra(stable): add chrome stable bot (#5768) 2021-03-09 19:19:21 -08:00
Max Schmitt 7971bb0335
devops: verify clean tree on bots after build (#5354) 2021-02-16 13:38:09 -08:00
Andrey Lushnikov e7b431d2f7
devops: fix test triggering (#5458) 2021-02-13 15:17:53 -08:00
Andrey Lushnikov 6326d6f3ac
devops: properly trigger tests on internal test runners (#5453) 2021-02-12 15:52:48 -08:00
Andrey Lushnikov fa730bec78 devops: trigger internal tests on each commit 2021-02-12 00:41:51 -07:00
Andrey Lushnikov 9d2269dc75
devops: attempt to fix npm canary publishing 2021-02-11 18:26:25 -08:00
Andrey Lushnikov 009765d719
devops: upload flakiness dashboard for release branches too (#5392)
There are no reasons to not upload test results for release branches.
2021-02-10 09:19:10 -08:00
Andrey Lushnikov 1240dd48cb
devops: start publishing canary at midnight every day (#5343)
This patch:
- starts publishing canary NPM package at 00:10AM UTC
- canary version is published from default (`master`) branch and is 
  named with a date. E.g. for a version published on Feb 5, 2021, the
  version would be `1.8.0-alpha-feb-5-2021`
- versions from release branches are still published on every commit and have the 
  regular commit timestamp suffix
2021-02-09 14:28:04 -08:00
Andrey Lushnikov a370443a8a
devops: use repository dispatch to trigger builds for all applications (#5104) 2021-01-22 12:26:31 +03:00
Andrey Lushnikov 86775f06d9 devops: add workflow to trigger Chromium builds once revision changes 2021-01-22 12:06:32 +03:00
Dmitry Gozman 0d0a6e8f82
devops: update publish script to work for @next (#5072) 2021-01-20 13:38:03 -08:00
Dmitry Gozman 7ff86a848a
chore: add publish-driver-release workflow (#5006) 2021-01-13 17:01:46 -08:00
Dmitry Gozman 4ff7e1a419
chore: cleanup our build system (#4903)
- Consolidate our build and watch to a single build.js file.
- Update contributing docs.
- Remove unused scripts and package.json script entries.
2021-01-06 12:41:17 -08:00
Dmitry Gozman d47fb6a780
feat(cli): build driver and upload to cdn (#4841) 2021-01-05 14:04:47 -08:00
Pavel Feldman ae935a43d9
doc: further align docs w/ playwright.dev (#4866) 2021-01-03 08:47:29 -08:00
Yury Semikhatsky a721ba5a8a
devops: give unique names to test shards (#4756) 2020-12-17 18:11:31 -08:00
Pavel Feldman e02c54485c
chore(adb): make driver smaller (#4713) 2020-12-14 18:39:14 -08:00
Pavel Feldman 4799e8f20b
feat(adb): add screenshot (#4701) 2020-12-13 23:20:13 -08:00
Pavel Feldman 1596b53da2
test(adb): fix browser tests (#4700) 2020-12-13 22:33:00 -08:00
Max Schmitt f4eff4dbe0
devops: add bot to test Android (#4693) 2020-12-13 14:13:50 -08:00
Max Schmitt 4f3f6267a8
chore: add dummy package.json's for GH dependents analysis (#4666)
Relates #4320

The support suggest something like that.

(does not close because we want to verify it works before)
2020-12-10 06:35:29 -08:00
Andrey Lushnikov bab16a5ce7
devops: remove self-hosted runners (#4524)
Turns out there's a way to abuse these self-hosted runners through pull-requests that mess with the YML files.
We'll move them to the private repo.
2020-11-25 08:24:54 -08:00
Pavel Feldman e72d9a4185
chore: add websocket connection mode (#4510) 2020-11-20 15:19:39 -08:00
Andrey Lushnikov 2d16953e0a
devops: start Chromium mirroring from GHA (#4492) 2020-11-19 13:44:00 -08:00
Andrey Lushnikov e2d4379489
devops: use proper tags for self-hosted runner (#4480)
Turns out github already has a reserved bot label called `ubuntu-18.04`,
so marking our self-hosted bot this way was adding it to the pool of
machines that are used here.
2020-11-18 11:02:49 -08:00
Andrey Lushnikov 33a168e5cf
devops: build firefox on self-hosted ubuntu 18.04 runner (#4478) 2020-11-18 08:33:53 -08:00
Andrey Lushnikov 05bb149d79
devops: teach self-hosted runner to install dependencies (#4475) 2020-11-17 20:46:16 -08:00
Andrey Lushnikov 80446aaf41
devops: use labels instead of self-hosted runner name (#4474) 2020-11-17 20:01:39 -08:00
Andrey Lushnikov 497a99deef
devops: try building webkit on a self-hosted ubuntu 18.04 runner (#4473) 2020-11-17 19:40:44 -08:00
Andrey Lushnikov f9a407f10d
devops: remove bigsur buildbot (#4457) 2020-11-17 19:27:12 -08:00
Andrey Lushnikov 52147b3046
devops: fix flakiness upload on windows (#4415) 2020-11-12 09:30:09 -08:00
Andrey Lushnikov 5d39eae509
devops: actually export secrets into webkit building workflow (#4354) 2020-11-05 03:13:34 -08:00
Andrey Lushnikov db8332b811
devops: checkout, build, and upload webkit-mac-11 (#4353) 2020-11-05 02:48:46 -08:00
Andrey Lushnikov bba8c98c69
devops: try building webkit on gha (#4351) 2020-11-05 01:01:18 -08:00
Pavel Feldman 6cafdc5a0e
infra: fix the headful bit on the bots (#4261) 2020-10-27 13:14:27 -07:00
Andrey Lushnikov 163cf1befd
devops: collect test reports for all bots (#4246)
Drive-by: make sure to upload flakiness even if tests fail.
2020-10-26 17:46:13 -07:00
Andrey Lushnikov d5fbe3a662
devops: start uploading test reports to flakiness dashboard (#4239) 2020-10-26 13:46:26 -07:00
Max Schmitt ebf207b7a1
chore: add support for macOS Big Sur (11.0) (#4149)
macOS 11.0 Big Sur was released as a preview on GitHub Actions: https://github.com/actions/virtual-environments/issues/1781#event-3879790130

Closes #2789
Relates #2793
2020-10-19 10:02:35 -07:00
Pavel Feldman 9daedaca08
chore: roll test fixtures, replace trace w/ video (#4129) 2020-10-13 13:18:36 -07:00
Pavel Feldman 5804131c99
chore: bump folio to 0.3.9, use fixture timeout (#4118) 2020-10-13 12:17:23 -07:00
Dmitry Gozman 46b14bc740
chore: roll folio to 0.3.6 (#4110) 2020-10-12 09:16:02 -07:00
Dmitry Gozman fe41bbd303
test: enable videos on tracing bots (#3930) 2020-09-24 04:38:01 -07:00
Max Schmitt 967f3b7505
devops(docker): push focal images to CR (#3950)
This will end up in the following Docker tags:
- focal <- new
- bionic < old as before
- latest <- old as before, keep bionic for now until EOL is reached
- v1.4 <- old as before, is bionic
- v1.4-focal <- new, is focal

Depends on https://github.com/aslushnikov/devops.aslushnikov.com/pull/3 to ensure it works.
Relates to https://github.com/microsoft/playwright/issues/3791.
2020-09-23 15:10:43 -07:00
Pavel Feldman 2d1cabdd3f
test: roll test runner to 0.3.18 (#3949) 2020-09-22 12:52:39 -07:00
Max Schmitt becdccdf03
devops(docker): added Dockerfile for Ubuntu 20 focal (#3891)
Each Ubuntu and Debian release has a code name. Ubuntu 18(bionic), Ubuntu 20(focal). This adds the Dockerfile for Ubuntu20.
Next steps and follow up changes:
- add it to the devops site, so we are sure all tests are passing, locally they did
- deploy it to the MCR, naming needs to be clarified, probably just as "focal".

This naming schema allows us in the future to add Debian support too. But we should wait until Headless WK is fixed.

Relates #3791
Relates #2758
Closes #3338
2020-09-21 08:47:44 -07:00
Andrey Lushnikov 84a0066c9f
devops: align release publishing of docker image with dev releases (#3725)
This patch starts using the same approach to docker publishing that we
currently use to publish canary docker images.
2020-09-02 23:33:09 -07:00
Andrey Lushnikov b7f6a98deb
devops: use a helper script to tag and push docker images (#3737)
It turns out that the new `docker push --all-tags` is not yet available
in the latest docker distribution on Ubuntu 20.04.

This provides a helper script to tag and publish the docker image.
2020-09-02 14:28:01 -07:00
Andrey Lushnikov df12264655
devops: use ubuntu 20.04 to build & publish docker releases (#3736)
We rely on `docker push --all-tags` flag that exists only in the
new version of docker.

The PR to docker with the flag: https://github.com/docker/cli/pull/2220
2020-09-02 11:46:24 -07:00
Andrey Lushnikov 216db2c9b3
devops: use microsoft/playwright-github-action@v1 everywhere (#3735)
Since we launch browsers as part of the `npm run build`, we need
to prepare environment.
2020-09-02 11:32:59 -07:00
Andrey Lushnikov 9d999ae1f0
devops: push all tags to docker registry (#3732)
Instead of pushing one specific tag, push repository itself.
This should push all the associated tags.
2020-09-02 10:33:54 -07:00