Commit graph

149 commits

Author SHA1 Message Date
Dmitry Gozman d38baae01c
feat(testrunner): nested environments (#1635) 2020-04-02 13:12:22 -07:00
Pavel Feldman e241c1bef8
chore: remove web mode (#1625) 2020-04-01 14:42:47 -07:00
Arjun Attam 1f0b7bff04
docs(api): update ordering for $eval and $$eval (#1623) 2020-04-01 13:09:24 -07:00
Dmitry Gozman f87e64544c
feat(testrunner): introduce environments (#1593) 2020-04-01 10:49:47 -07:00
Joel Einbinder a853690c16
fix(types): don't export derived types (#1598)
I was playing around today with different ways of changing the way we export types for #1439. I looked at only exporting 'Parameter' types, only exporting 'Return' types, only exporting a manual list of 'important' types. They all had different pros and cons, and it was very difficult to settle on a good answer.

For now, let's not export any parameter/return types. We can whitelist some types upon user request. I'm thinking `LaunchOptions` and `AccessibilitySnapshot` could be quite useful. We can always add new types after 1.0, but we can't remove them.

The patch looks funny because this was my original intent for the types, but I didn't know I had to `export {}` to tell typescript that my .d.ts shouldn't export everything.
2020-03-31 16:19:20 -07:00
Joel Einbinder 92c5ab3b7e
fix(types): correctly infer type in $eval and $$eval (#1603) 2020-03-31 13:22:38 -07:00
Joel Einbinder 08aebc72fa
fix(types): add types for waitForEvent (#1601) 2020-03-31 09:17:54 -07:00
Dmitry Gozman b6166c9669
chore(testrunner): introduce Location class (#1585)
Drive-by: fix an edge when testing continued after termination.
2020-03-29 21:38:30 -07:00
Dmitry Gozman c49b856d6d
chore(testrunner): remove setup() helper (#1584) 2020-03-29 11:37:45 -07:00
Dmitry Gozman a41836b1f1
chore(testrunner): introduce TestRun (#1582) 2020-03-28 14:25:57 -07:00
Dmitry Gozman b85ab891a1
chore(testrunner): make most modifiers external (#1581) 2020-03-28 08:49:00 -07:00
Dmitry Gozman 4bd46bafa9
feat(testurnner): allow multiple hooks isntances and per-test hooks (#1571) 2020-03-27 22:57:22 -07:00
Dmitry Gozman 6503c83302
fix(install): speculative fix for generateChromiumProtocol (#1583) 2020-03-27 22:42:39 -07:00
Pavel Feldman 6903496f2d
fix(build): generate protocol in chromium (#1579) 2020-03-27 17:11:54 -07:00
Dmitry Gozman a042466125
chore(testrunner): remove effectiveMode and effectiveExpectation (#1569) 2020-03-27 15:35:14 -07:00
Dmitry Gozman af7a16c360
chore(testrunner): merge test spec with test, suite spec with suite (#1566) 2020-03-26 22:47:13 -07:00
Dmitry Gozman aad82e00bd
chore(testrunner): decouple UserCallback from location and timeout (#1557)
This will make it easier to change lifetimes of Test and Suite.
2020-03-26 14:43:28 -07:00
Dmitry Gozman 81bd8de00b
feat(testrunner): composable and bindable attributes and modifiers (#1547)
This allows to make things like `fit`, `skip()` and soon even `dit` be implemented externally.
2020-03-25 22:42:09 -07:00
Andrey Lushnikov 7e75cefd20
devops: restore publishing @next version (#1540)
This merges the `//utils/apply_next_version.js` in to
`//utils/update_version.js`.
2020-03-25 17:31:19 -07:00
Dmitry Gozman b61198458c
feat(testrunner): modifiers and attributes (#1538)
This generalizes test modifiers to support custom ones.
2020-03-25 14:40:57 -07:00
Dmitry Gozman 60942d0af5
chore(selectors): move selectors logic to selectors.ts (#1536)
This encapsulates selectors logic in one place, in a preparation for more complex scenarios like main-world selectors or piercing frames.

Note: we had `Page.fill should wait for visible visibilty` test, but we do not actually wait for visible in page.fill(). It happened to pass due to lucky evaluation order.

References #1316.
2020-03-25 14:08:46 -07:00
Dmitry Gozman 6ee7852f0f
chore: update release guide and helper script (#1521) 2020-03-25 13:15:29 -07:00
Dmitry Gozman a2ee7a1f2a
fix(testrunner): do not spam output after termination (#1511) 2020-03-24 14:40:59 -07:00
Pavel Feldman 222560866a
chore: add logging to the testing server (#1505) 2020-03-24 09:11:56 -07:00
Joel Einbinder 5bf9f223db
fix(docs): consider argument to be optional in evaluate (#1500) 2020-03-23 17:22:10 -07:00
Pavel Feldman c0c9b7f137
test: make debugp collect IO (take 2) (#1493) 2020-03-23 15:08:02 -07:00
Joel Einbinder 1a25a4efcf
fix(doclint): support lists in comments (#1492)
Adds logging comments to the doclint tests, and adds a new one with a bulleted list in a comment. Lists can only be used in comments where extra properties would be unexpected.
2020-03-23 14:50:32 -07:00
Dmitry Gozman 63906454a7
fix(testrunner): attribute unhandle promise reject to a single worker (#1491)
When there is a single worker, we are almost sure the error originated in that worker. Attributing it helps with context by showing last run tests.
2020-03-23 14:45:21 -07:00
Pavel 1ddf05113b Revert "test: make debugp collect IO (#1485)"
This reverts commit b1bebdad2e.
2020-03-23 13:48:22 -07:00
Pavel Feldman b1bebdad2e
test: make debugp collect IO (#1485) 2020-03-23 13:42:51 -07:00
Dmitry Gozman 16c7a5bd5c
api(eval): accept zero or one arguments in all evaluation functions (#1431) 2020-03-20 15:08:17 -07:00
Joel Einbinder 825555cddf
types: better types (#1166)
This generates typescript definitions based on the api.md, instead of autogenerating them from the typescript source code.

Now types
 - only include the public api
 - work with older versions of typescript
 - include descriptions
 - are more consistent
 - are more complete

#6
2020-03-20 01:30:35 -07:00
Andrey Lushnikov f5ecbff16e
devops: remake downloading logic (#1419)
This patch:
- removes `browserType.downloadBrowserIfNeeded()` method. The method
  turned out to be ill-behaving and cannot not be used as we'd like to (see #1085)
- adds a `browserType.setExecutablePath` method to set a browser
  exectuable.

With this patch, we take the following approach towards managing browser downloads:
- `playwright-core` doesn't download any browsers. In `playwright-core`, `playwright.chromium.executablePath()` returns `null` (same for firefox and webkit).
- clients of `playwright-core` (e.g. `playwright` and others) download browsers one way or another.
They can then configure `playwright` with executable paths and re-export the `playwright` object to their clients.
- `playwright`, `playwright-firefox`, `playwright-chromium` and `playwright-webkit` download 
browsers. Once browsers are downloaded, their executable paths are saved to a `.downloaded-browsers.json` file. This file is read in `playwright/index.js` to configure browser executable paths and re-export the API.
- special case is `install-from-github.js` that also cleans up old browsers.
2020-03-19 11:43:35 -07:00
Joel Einbinder b0749e3a6d
fix(docs): fixup and lint optionals and return values in api.md (#1376)
Currently in our API `?` means null, but sometimes it means optional. Linting optional/nulls with this patch is required for #1166 to land nicely.

Previously, return types were not being linted in `api.md`. This is fixed, along with many broken return types.

This patch considers `?` to mean nullable, and has some heuristics to determine optionality. I believe this to be the minimal patch needed to unblock #1166. After it lands, we can consider changing the api docs to hopefully remove some heuristics and strangeness.
2020-03-18 16:23:25 -07:00
Joel Einbinder 741e2d19d6
fix(docs): lint and fix all internal links in api.md
We have had a lot of churn in the api, which has caused a lot of our links to break.
2020-03-18 15:34:53 -07:00
Andrey Lushnikov a9ab9b0dc0
fix(testrunner): sourcemapify stack traces for test errors (#1409)
Fixes #1403
2020-03-17 09:04:44 -07:00
Dmitry Gozman 3960b179a1
fix(testrunner): fit.fail should run the test (#1407) 2020-03-16 19:12:52 -07:00
Andrey Lushnikov 5816ec53f7
fix(testrunner): dedup focused tests and suites by id (#1393) 2020-03-16 09:04:55 -07:00
Dmitry Gozman e7eeefe4c7
chore(testrunner): separate expectations from run mode (#1395)
Run/Focus/Skip is orthogonal to expect to Pass/Fail/Flake.
This change separates the two, in a preparation to run Fail/Flaky tests.
2020-03-15 23:10:49 -07:00
Pavel Feldman e692dd6e62
api(cdp): rename ChromiumSession to CDPSession (#1380) 2020-03-13 16:16:54 -07:00
Dmitry Gozman 0d7cb29329
test: continue running tests after crash, report crashes separately (#1362) 2020-03-12 17:32:53 -07:00
Joel Einbinder be83cba409
fix(doclint): correctly get versions on windows (#1350)
--version doesn't work on windows.
2020-03-12 14:03:01 -07:00
Andrey Lushnikov 704fe6dc73 fix(testrunner): fix reporting focused tests 2020-03-11 18:53:24 -07:00
Andrey Lushnikov 1cd00bd068
feat(testrunner): allow filtering by name and show all focused tests (#1354) 2020-03-11 18:30:43 -07:00
Max Schmitt a24cce8b74
devops: fix protocol generation with root on Linux (#1327)
Currently it was leading to an error if I tried to build it with a root user on a Linux environment. So in the end the type generation was just skipped:

```
22:51:55.757 $ node install-from-github.js
22:51:55.813 Building playwright...
22:52:14.094 chromium downloaded to /11994741/playwright/.local-chromium/linux-747023 22:52:14.123 Failed to launch browser!
22:52:14.123 [0310/215214.121201:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
22:52:14.123 TROUBLESHOOTING: https://github.com/Microsoft/playwright/blob/master/docs/troubleshooting.md
```

Maybe we should also exit the script with an error code if the protocol generation was not successful.
2020-03-10 15:21:08 -07:00
Yury Semikhatsky 16d5a9cb24
tests(runner): support DEBUGP for timing out tests (#1324) 2020-03-10 14:48:13 -07:00
Dmitry Gozman 0ce8efab7b
test: rework testrunner workers (#1296)
This change introduces a TestWorker that can be in a certain state,
meaning it has run some beforeAll hooks of a certain test suite stack.

TestWorker can be created at any time, which allows for a number of features:
- don't run hooks for suites with no runnable tests;
- smarter test distribution (and possibility for variuos strategies);
- recovering from hook failures and test failure by creating a new worker;
- possible isolation between workers by running them in separate environments.
2020-03-10 11:30:02 -07:00
Andrey Lushnikov d542ef849f
fix(testrunner): handle uncaught errors (#1317) 2020-03-10 11:16:54 -07:00
Andrey Lushnikov 3dc48f96c0
chore: output both received value and diff for string expected results (#1287) 2020-03-07 17:29:41 -08:00
Andrey Lushnikov c881248062
docs(contributing.md): update CONTRIBUTING.md (#1286)
Fixes #1257
2020-03-07 17:29:22 -08:00