Commit graph

81 commits

Author SHA1 Message Date
Andrey Lushnikov 009765d719
devops: upload flakiness dashboard for release branches too (#5392)
There are no reasons to not upload test results for release branches.
2021-02-10 09:19:10 -08:00
Dmitry Gozman 4ff7e1a419
chore: cleanup our build system (#4903)
- Consolidate our build and watch to a single build.js file.
- Update contributing docs.
- Remove unused scripts and package.json script entries.
2021-01-06 12:41:17 -08:00
Dmitry Gozman d47fb6a780
feat(cli): build driver and upload to cdn (#4841) 2021-01-05 14:04:47 -08:00
Yury Semikhatsky a721ba5a8a
devops: give unique names to test shards (#4756) 2020-12-17 18:11:31 -08:00
Pavel Feldman e02c54485c
chore(adb): make driver smaller (#4713) 2020-12-14 18:39:14 -08:00
Pavel Feldman 4799e8f20b
feat(adb): add screenshot (#4701) 2020-12-13 23:20:13 -08:00
Pavel Feldman 1596b53da2
test(adb): fix browser tests (#4700) 2020-12-13 22:33:00 -08:00
Max Schmitt f4eff4dbe0
devops: add bot to test Android (#4693) 2020-12-13 14:13:50 -08:00
Pavel Feldman e72d9a4185
chore: add websocket connection mode (#4510) 2020-11-20 15:19:39 -08:00
Andrey Lushnikov 52147b3046
devops: fix flakiness upload on windows (#4415) 2020-11-12 09:30:09 -08:00
Pavel Feldman 6cafdc5a0e
infra: fix the headful bit on the bots (#4261) 2020-10-27 13:14:27 -07:00
Andrey Lushnikov 163cf1befd
devops: collect test reports for all bots (#4246)
Drive-by: make sure to upload flakiness even if tests fail.
2020-10-26 17:46:13 -07:00
Andrey Lushnikov d5fbe3a662
devops: start uploading test reports to flakiness dashboard (#4239) 2020-10-26 13:46:26 -07:00
Max Schmitt ebf207b7a1
chore: add support for macOS Big Sur (11.0) (#4149)
macOS 11.0 Big Sur was released as a preview on GitHub Actions: https://github.com/actions/virtual-environments/issues/1781#event-3879790130

Closes #2789
Relates #2793
2020-10-19 10:02:35 -07:00
Pavel Feldman 9daedaca08
chore: roll test fixtures, replace trace w/ video (#4129) 2020-10-13 13:18:36 -07:00
Pavel Feldman 5804131c99
chore: bump folio to 0.3.9, use fixture timeout (#4118) 2020-10-13 12:17:23 -07:00
Dmitry Gozman 46b14bc740
chore: roll folio to 0.3.6 (#4110) 2020-10-12 09:16:02 -07:00
Dmitry Gozman fe41bbd303
test: enable videos on tracing bots (#3930) 2020-09-24 04:38:01 -07:00
Pavel Feldman 2d1cabdd3f
test: roll test runner to 0.3.18 (#3949) 2020-09-22 12:52:39 -07:00
Joel Einbinder 3cc91093a1
chore(testrunner): move out of the repo (#3687) 2020-08-29 08:27:32 -07:00
Pavel Feldman 555a8d0d10
fix(testrunner): include fixture teardown into timeout, add global timeout (#3685) 2020-08-28 19:27:49 -07:00
Dmitry Gozman e5ff283a40
fix(trace): only enable on separate tracing bots (#3677) 2020-08-28 13:50:06 -07:00
Pavel Feldman 6ffdd4dfa1
feat(testrunner): allow unexpected passes (#3665) 2020-08-28 00:32:00 -07:00
Pavel Feldman 15ec87dbcc
feat(testrunner): support --retries, flaky tests (#3649) 2020-08-26 16:32:47 -07:00
Pavel Feldman adc2a44113
infra: simplify test results collection (#3623) 2020-08-25 15:46:42 -07:00
Pavel Feldman 06dcf967ba
feat(testrunner): allow external reporters (#3603) 2020-08-24 19:16:20 -07:00
Joel Einbinder 012f9425bf
chore(test-runner): move into its own folder and typescript project (#3548) 2020-08-20 16:04:27 -07:00
Pavel Feldman 30f4c0c9fe
test runner: remove dependencies on playwright (#3539) 2020-08-19 17:47:24 -07:00
Max Schmitt 8ea280090d
devops: first implementation of browser auto-roll bot (#3455)
Relates #3258

After a chat with @aslushnikov we add this in two iterations. The first one (this PR) is about scheduled runs for testing the latest tip-of-tree version of the browsers daily and the next PR is about opening an automated PRs once its passing (not sure if its worth to add for failing too) or sending Telegram/Slack notifications.

Current status is that Firefox (around 2 hours) and WebKit (around 3-4 hours) works.

Known issues which I fix before we merge:
- ~~Changes in `test/base.fixture.ts` will get extracted in #3453~~

Feel free to review, current blockers are before we can merge:
- ~~#3453~~
- potential git clone optimisation by aslushnikov
2020-08-17 11:23:46 -07:00
Pavel Feldman 262e886940
test: organize golden files under __snapshots__ folder (#3494) 2020-08-16 19:19:52 -07:00
Dmitry Gozman 9b52ca8676
chore: remove unused non-rpc code, test options, infra, bots (#3444) 2020-08-13 16:00:23 -07:00
Dmitry Gozman 84ca0120f8
test: upload output from headful bots (#3439) 2020-08-13 09:33:43 -07:00
Joel Einbinder d3677357b8
feat(testrunner): take the first argument as the test root dir (#3423) 2020-08-12 20:01:13 -07:00
Yury Semikhatsky f0fcdc8f51
test(firefox): make headful screencast tests work under xvfb (#3421) 2020-08-12 16:07:01 -07:00
Pavel Feldman a4a07c46c3
test: support --grep, --forbid-only (#3417) 2020-08-12 14:57:37 -07:00
Pavel Feldman 962ddc09c4
test: consolidate runner files (#3415) 2020-08-12 13:47:44 -07:00
Pavel Feldman 7e07634cc6
test: use mocha in ci/cd (#3406) 2020-08-12 11:48:30 -07:00
Dmitry Gozman 8bb6d73b44
feat(rpc): keep non-rpc linux bots for now (#3381) 2020-08-10 20:49:53 -07:00
Andrey Lushnikov 421f6f48c7
devops: use playwright-github-action@v1 (#3221)
The playwright-github-action@v1 now points to v1.4.3, so we can
switch to it once again.
2020-07-30 17:47:39 -07:00
Andrey Lushnikov 77b1c4b883
devops: enable Ubuntu 20.04 tests (#3178) 2020-07-29 11:36:38 -07:00
Yury Semikhatsky 79ab07bd0d
devops: install Media Pack on Windows bot (#3137) 2020-07-23 18:06:16 -07:00
Dmitry Gozman d234dac752
chore: support esm imports (#3125) 2020-07-23 15:14:36 -07:00
Joel Einbinder 773ee08e9d
chore(test): restore api coverage checks (#3068)
* chore(test): restore api coverage checks

* try to catch the correct error on the bots

* try to trick github into taking the new code

* uninstall coverage

* dgozman comments
2020-07-22 09:08:10 -07:00
Joel Einbinder 3c151d8f1d
fix(test): don't output babel's debug info on the bots (#3073) 2020-07-21 16:23:46 -07:00
Pavel Feldman b5f9985d3a
devops: make headful a matrix, collect test results (#3027) 2020-07-18 10:50:58 -07:00
Pavel Feldman 9d9801192e
test: wire test commands to jest (#3014) 2020-07-17 16:09:27 -07:00
Pavel Feldman 424f11d165
test: convert some tests to the jest+fixtures (#2983) 2020-07-16 16:59:45 -07:00
Pavel Feldman ecc130c644
test: convert evaluation.spec to jest+fixtures (#2968) 2020-07-16 11:18:38 -07:00
Pavel Feldman 19cd96c46a
test: add the jest-circus experimental runner (#2962) 2020-07-15 16:52:41 -07:00
Dmitry Gozman 2151757621
feat(rpc): run rpc tests in-process and out-of-process (#2929) 2020-07-13 15:00:20 -07:00