Commit graph

32 commits

Author SHA1 Message Date
Andrey Lushnikov 7925a51149
feat: support concurrent installation of browsers (#3929)
A few details on locking registry to prohibit concurrent access:
- locking is done by creating a `__dirlock` directory in the top-level
of our registry.
- since `__dirlock` directory does not match any of browser
directories, old versions of the installer will ignore it
- in case of concurrent access, installation will wait for a lock to be
released for 10 minutes, periodically trying to grab the lock. If it
fails to do so in 10 minutes, the installation will fail.

Fixes #3912
2020-09-21 16:09:11 -07:00
Andrey Lushnikov 638c77c8e2
devops: stop bundling FFMPEG with Chromium (#3806)
- This stops bundling FFMPEG with Chromium
- Stop supporting build numbers as fractional parts for Chromium
revisions
2020-09-08 17:05:21 -07:00
Andrey Lushnikov a755d100b3
devops: encode build number together with Chromium revision (#3769)
This is an alternative approach to #3698 that was setting up a custom
mapping between chromium revisions and our mirrored builds. For example, we were
taking chromium `792639` and re-packaging it to our CDN as Chromium 1000.

One big downside of this opaque mapping was inability to quickly
understand which Chromium is mirrored to CDN.

To solve this, this patch starts treating browser revision as a fractional number,
with and integer part being a chromium revision, and fractional
part being our build number. For example, we can generate builds `792639`, `792639.1`,
`792639.2` etc, all of which will pick Chromium `792639` and re-package it to our CDN.

In the Playwright code itself, there are a handful of places that treat
browser revision as integer, exclusively to compare revision with some particular
revision numbers. This code would still work as-is, but I changed these places
to use `parseFloat` instead of `parseInt` for correctness.
2020-09-04 03:12:30 -07:00
Dmitry Gozman 2e1493a5fa
chore: move browserPaths to utils, enforce more deps (#3584) 2020-08-22 21:15:03 -07:00
Dmitry Gozman f4e8f34c96
chore: move chromium to src/server/chromium, enfore installer deps (#3582) 2020-08-22 15:46:42 -07:00
Dmitry Gozman 655013d025
chore: move shared utilities to src/utils (#3575) 2020-08-22 07:07:13 -07:00
Dmitry Gozman 58fc6b4003
chore: align some server-side methods with rpc calls (#3504)
- Never write to console on the server side - we use stdout for
  communication. This includes logPolitely and deprecate.
- Pass undefined instead of null in some BrowserContext methods.
- Use explicit _setFileChooserIntercepted instead of on/off magic.
2020-08-17 16:19:21 -07:00
Joel Einbinder 23f5ed89b0
fix(launcher): default to ubuntu20.04 for newer releases (#3400) 2020-08-12 17:50:46 -07:00
Andrey Lushnikov 40f685226b
devops: migrate //utils/check_availability.js off browser fetcher (#3418)
The script is used to check the state of chromium CDN, whereas
our browser fetcher now defaults to Playwright CDN.
2020-08-12 15:01:07 -07:00
Andrey Lushnikov 1b8128eb84
installer: start downloading Chromium archives from our CDN (#3361)
This starts downloading newer Chromium archives from our CDN, but
retains old endpoint for older Chromium revisions.

This backwards compatibility might help later on to implement
a browser bisecting tool.

References #3259
2020-08-10 11:40:30 -07:00
Yury Semikhatsky cbfdca736c
feat(launcher): check dependencies before launch on Windows (#3240) 2020-07-30 17:15:46 -07:00
Dmitry Gozman fd2e65b73c
api: export all browsers from every package (#3128)
This makes it easier to reason about our packages.
The only difference is what each package downloads.
When the browser is not downloaded, it will fail to launch.

Each browser gets a 'download' attribute in the browser.json file.
2020-07-24 16:36:00 -07:00
Andrey Lushnikov 678d16454a
devops: normalize blob names on the CDN (#3136)
This establishes a single naming for all our blobs with browser
builds that we upload to CDN: `<browser-name>-<os-version>`

- `<browser-name>` is either `firefox` or `webkit`.
- `os-version` is the OS that was used to produce the build.

References #2745
2020-07-24 00:31:18 -07:00
Andrey Lushnikov bbe7dbe9ae
feat(installer): start downloadinb Ubuntu 20.04 builds (#3126)
Start auto-detecting Ubuntu 20.04 and downloading custom webkit
build for it.

References #2745
2020-07-23 17:37:47 -07:00
Andrey Lushnikov 63689e36a6
devops: prepare buildbots to the introduction of Ubuntu 20.04 builder (#3116)
This patch:
- specializes "linux" scripts into "Ubuntu 18.04" scripts
- renames all future linux blobs on CDN:
    * `firefox-linux.zip => firefox-ubuntu-18.04.zip`
    * `minibrowser-gtk.zip => minibrowser-gtk-ubuntu-18.04.zip`
    * `minibrowser-wpe.zip => minibrowser-wpe-ubuntu-18.04.zip`
    * `minibrowser-gtk-wpe.zip => minibrowser-gtk-wpe-ubuntu-18.04.zip`
- updates downloader to deal with the new names

References #2745
2020-07-23 11:59:23 -07:00
Andrey Lushnikov ef2a6522b8
feat: support atomic browser installation - attempt 2 (#3008)
Currently, Ctrl-C while extracting browser might yield users in
a bad place.

This patch adds a marker file inside browser directory to make
sure that browser extraction completed.

Note: this was already attempted in #2489, but was eventually
reverted in #2534.

References #2660
2020-07-17 16:39:27 -07:00
Andrey Lushnikov 0b9218149f
feat: validate browser dependencies before launching on Linux (#2960)
Missing dependencies is #1 problem with launching on Linux.

This patch starts validating browser dependencies before launching
browser on Linux. In case of a missing dependency, we will abandon
launching with an error that lists all missing libs.

References #2745
2020-07-15 15:24:38 -07:00
Dmitry Gozman 5c4751d5dd
chore: generate protocol during browser roll (#2719)
We now commit protocol.ts files during the roll.
New utils/roll_browser.js helps with that.

This makes our installation very shallow:
- build installer;
- download browsers.
2020-07-01 15:22:29 -07:00
Andrey Lushnikov 922cbe6718
chore: roll https-proxy-agent to v5 (#2777)
Co-authored-by: Luke Edwards <luke.edwards05@gmail.com>
2020-06-30 17:03:01 -07:00
Mike Tobia 63924d9a26
feat(install/download-hosts): allow multiple per browser (#2452)
In addition to `PLAYWRIGHT_DOWNLOAD_HOST` env variable, this patch adds a per-browser 
configuration:

- `PLAYWRIGHT_CHROMIUM_DOWNLOAD_HOST`
- `PLAYWRIGHT_FIREFOX_DOWNLOAD_HOST`
- `PLAYWRIGHT_WEBKIT_DOWNLOAD_HOST`
2020-06-18 11:20:43 -07:00
Andrey Lushnikov 89d1b52f43
chore: revert atomic install (#2534)
This reverts 2 commits:

- "fix(installer): create tmp directory inside `browserPath` (#2498)"
  commit 946b4efa3b.

- "feat: support atomic installation of browsers (#2489)"
  commit 3de0c087bc.

This addresses installation issues we see in some CI environments.
2020-06-10 18:49:03 -07:00
Andrey Lushnikov 946b4efa3b
fix(installer): create tmp directory inside browserPath (#2498)
`fs.rename` doesn't work across partitions, so we have to have
tmp folder next to our final destination.

Fixes #2494
2020-06-08 12:11:16 -07:00
Andrey Lushnikov 3de0c087bc
feat: support atomic installation of browsers (#2489)
Currently, Ctrl-C while extracting browser might yield users in
a bad place.

This patch extracts browsers in a temp directory that is later
moved to a installer registry.
2020-06-05 16:26:32 -07:00
Andrey Lushnikov fdd48f8940
chore: remove confusing logging from registry (#2397) 2020-05-28 22:36:08 -07:00
Pavel Feldman 9895cd0a31
chore: optionally create downloads folder (#2188) 2020-05-11 14:42:13 -07:00
Pavel Feldman 793a2bf7d4
fix(firefox): do not run firefox as a part of the installation process (#2125) 2020-05-06 10:42:27 -07:00
Pavel Feldman d95891ebad
fix(install): only install browsers needed by this revision (#2112) 2020-05-04 22:24:40 -07:00
Pavel Feldman 1f0217986e
feat(firefox): cache firefox pre-compiled scripts (#2087) 2020-05-04 09:34:59 -07:00
Pavel Feldman 9f62f29946
feat(install): use shared installation folder by default (#2044) 2020-04-29 18:59:20 -07:00
Pavel Feldman 0228ba4992
feat(registry): implement download registry (#1979) 2020-04-29 17:19:21 -07:00
Pavel Feldman 6c94f604d8
feat(chromium): roll to r763809 (#2028) 2020-04-29 08:44:09 -07:00
Pavel Feldman 2cdf297245
chore: remove uncompiled download-browser (#2018) 2020-04-28 17:06:01 -07:00