Filip/playwright
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

docs(runner): start adding runner docs (#6773)

Browse source
This commit is contained in:
Pavel Feldman 2021-05-27 09:16:50 -07:00 committed by GitHub
parent ddce546ed2
commit 4578d57901
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 518 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

181
docs/src/test-runner-configuration-js.md Normal file
View file

@ -0,0 +1,181 @@
---
id: test-runner-configuration
title: "Configuration"
---
<!-- TOC -->
## Modify options
You can modify browser launch options, context creation options and testing options either globally in the configuration file, or locally in the test file.
Playwright test runner is based on the [Folio] framework, so it supports any configuration available in Folio, and adds a lot of Playwright-specific options.
### Globally in the configuration file
You can specify different options for each browser using projects in the configuration file. Below is an example that changes some global testing options, and Chromium browser configuration.
```js
// config.ts
import { PlaywrightTestConfig } from "@playwright/test";
const config: PlaywrightTestConfig = {
// Each test is given 90 seconds.
timeout: 90000,
// Failing tests will be retried at most two times.
retries: 2,
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
// Launch options
headless: false,
slowMo: 50,
// Context options
viewport: { width: 800, height: 600 },
ignoreHTTPSErrors: true,
// Testing options
video: 'retain-on-failure',
},
},
],
};
export default config;
```
### Locally in the test file
With `test.use()` you can override some options for a file, or a `describe` block.
```js
// my.spec.ts
import { test, expect } from "@playwright/test";
// Run tests in this file with portrait-like viewport.
test.use({ viewport: { width: 600, height: 900 } });
test('my test', async ({ page }) => {
// Test code goes here.
});
```
### Available options
See the full list of launch options in [`browserType.launch()`](https://playwright.dev/docs/api/class-browsertype#browsertypelaunchoptions) documentation.
See the full list of context options in [`browser.newContext()`](https://playwright.dev/docs/api/class-browser#browsernewcontextoptions) documentation.
Available testing options:
- `screenshot: 'off' | 'on' | 'only-on-failure'` - Whether to capture a screenshot after each test, off by default.
- `off` - Do not capture screenshots.
- `on` - Capture screenshot after each test.
- `only-on-failure` - Capture screenshot after each test failure.
- `video: 'off' | 'on' | 'retain-on-failure' | 'retry-with-video'` - Whether to record video for each test, off by default.
- `off` - Do not record video.
- `on` - Record video for each test.
- `retain-on-failure` - Record video for each test, but remove all videos from successful test runs.
- `retry-with-video` - Record video only when retrying a test.
Most notable testing options from [Folio documentation][folio]:
- `reporter: 'dot' | 'line' | 'list'` - Choose a reporter: minimalist `dot`, concise `line` or detailed `list`. See [Folio reporters][folio-reporters] for more details.
- `retries: number` - Each failing test will be retried up to the certain number of times.
- `testDir: string` - Directory where test runner should search for test files.
- `timeout: number` - Timeout in milliseconds for each test.
- `workers: number` - The maximum number of worker processes to run in parallel.
## Skip tests with annotations
The Playwright test runner can annotate tests to skip under certain parameters. This is enabled by [Folio annotations][folio-annotations].
```js
test("should be skipped on firefox", async ({ page, browserName }) => {
test.skip(browserName === "firefox", "optional description for the skip");
// Test function
});
```
## Run tests in parallel
Tests are run in parallel by default, using multiple worker processes. You can control the parallelism with the `workers` option in the configuration file or from the command line.
```sh
# Run just a single test at a time - no parallelization
npx folio --workers=1
# Run up to 10 tests in parallel
npx folio --workers=10
```
```js
// config.ts
import { PlaywrightTestConfig } from "@playwright/test";
const config: PlaywrightTestConfig = {
// No parallelization on CI, default value locally.
worker: process.env.CI ? 1 : undefined,
projects: [
// Your projects go here
],
};
export default config;
```
By default, test runner chooses the number of workers based on available CPUs.
## Reporters
Playwright test runner comes with a few built-in reporters for different needs and ability to provide custom reporters. The easiest way to try out built-in reporters is to pass `--reporter` [command line option](#command-line). Built-in terminal reporters are minimalist `dot`, concise `line` and detailed `list`.
```sh
npx folio --reporter=line
npx folio --reporter=dot
npx folio --reporter=list
```
Alternatively, you can specify the reporter in the configuration file.
```js
// config.ts
import { PlaywrightTestConfig } from "@playwright/test";
const config: PlaywrightTestConfig = {
// Concise 'dot' on CI, more interactive 'list' when running locally
reporter: process.env.CI ? 'dot' : 'line',
projects: [
// Your projects go here
],
};
export default config;
```
### Export JUnit or JSON report
The Playwright test runner includes reporters that produce a JUnit compatible XML file or a JSON file with test results.
```js
// config.ts
import { PlaywrightTestConfig } from "@playwright/test";
const config: PlaywrightTestConfig = {
reporter: [
// Live output to the terminal
'list',
// JUnit compatible xml report
{ name: 'junit', outputFile: 'report.xml' },
// JSON file with test results
{ name: 'json', outputFile: 'report.json' },
]
projects: [
// Your projects go here
],
};
export default config;
```
[folio]: https://github.com/microsoft/folio
[folio-annotations]: https://github.com/microsoft/folio#annotations
[folio-cli]: https://github.com/microsoft/folio#command-line
[folio-reporters]: https://github.com/microsoft/folio#reporters

133
docs/src/test-runner-examples-js.md Normal file
View file

@ -0,0 +1,133 @@
---
id: test-runner-examples
title: "Examples"
---
<!-- TOC -->
## Multiple pages
The default `context` argument is a [BrowserContext][browser-context]. Browser contexts are isolated execution environments that can host multiple pages. See [multi-page scenarios](./multi-pages.md) for more examples.
```js
import { test } from "@playwright/test";
test("tests on multiple web pages", async ({ context }) => {
const pageFoo = await context.newPage();
const pageBar = await context.newPage();
// Test function
});
```
## Mobile emulation
`use` section in the configuration file can be used to configure mobile emulation in the default `context`.
```js
// config.ts
import { PlaywrightTestConfig } from "@playwright/test";
import { devices } from "playwright";
const config: PlaywrightTestConfig = {
timeout: 30000,
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
headless: true,
...devices["Pixel 2"],
},
},
],
};
export default config;
```
## Network mocking
Define a custom route that mocks network calls for a browser context.
```js
// In foo.spec.ts
import { test, expect } from "@playwright/test";
test.beforeEach(async ({ context }) => {
// Block any css requests for each test in this file.
await context.route(/.css/, route => route.abort());
});
test("loads page without css", async ({ page }) => {
// Alternatively, block any png requests just for this test.
await page.route(/.png/, route => route.abort());
// Test function code.
await page.goto("https://stackoverflow.com");
});
```
## Visual comparisons
The `expect` API supports visual comparisons with `toMatchSnapshot`. This uses the [pixelmatch](https://github.com/mapbox/pixelmatch) library, and you can pass `threshold` as an option.
```js
import { test, expect } from "@playwright/test";
test("compares page screenshot", async ({ page }) => {
await page.goto("https://stackoverflow.com");
const screenshot = await page.screenshot();
expect(screenshot).toMatchSnapshot(`test.png`, { threshold: 0.2 });
});
```
On first execution, this will generate golden snapshots. Subsequent runs will compare against the golden snapshots. To update golden snapshots with new actual values, run with the `--update-snapshots` flag.
```sh
# Update golden snapshots when they differ from actual
npx folio --update-snapshots
```
### Page object model
To introduce a Page Object for a particular page, create a class that will use the `page` object.
Create a `LoginPage` helper class to encapsulate common operations on the login page.
```js
// login-page.ts
import type { Page } from "playwright";
export class LoginPage {
page: Page;
constructor(page: Page) {
this.page = page;
}
async goto() {
await this.page.goto("https://example.com/login");
}
async login() {
await this.page.fill("#username", TEST_USERNAME);
await this.page.fill("#password", TEST_PASSWORD);
await this.page.click("text=Login");
}
}
```
Use the `LoginPage` class in the tests.
```js
// my.spec.ts
import { test, expect } from "@playwright/test";
import { LoginPage } from "./login-page";
test('login works', async ({ page }) => {
// Create the login page and perform operations.
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login();
// Verify it worked.
expect(await page.textContent("#user-info")).toBe("Welcome, Test User!");
});
```

198
docs/src/test-runner-intro-js.md Normal file
View file

@ -0,0 +1,198 @@
---
id: test-runner-intro
title: "Playwright Tests"
---
Playwright Test Runner was created specifically to accommodate the needs of the end-to-end testing. It does everything you would expect from the regular test runner, and more. Playwright test allows to:
- Run tests across all browsers.
- Execute tests in parallel.
- Enjoy context isolation out of the box.
- Capture videos, screenshots and other artifacts on failure.
- Integrate your POMs as extensible fixtures.
There are many more exciting features, so read on!
<!-- TOC -->
Note that you can still wire Playwright to your existing JavaScript [test runner]('./test-runners-other-js.md).
## Installation
```sh
npm i -D @playwright/test@1.0.0-alpha
```
## First test
Create `tests/foo.spec.ts` to define your test.
```js
import { test, expect } from '@playwright/test';
test('is a basic test with the page', async ({ page }) => {
await page.goto('https://playwright.dev/');
const name = await page.innerText('.navbar__title');
expect(name).toBe('Playwright');
});
```
Now run your tests:
```sh
# Assuming that test files are in the tests directory.
npx folio -c tests
```
## Test fixtures
You noticed an argument `{ page }` that the test above has access to:
```js
test('basic test', async ({ page }) => {
...
```
We call these arguments `fixtures`. Playwright Test comes loaded with those fixtures, and you can add your own fixtures as well. Here is a list of the pre-defined fixtures that you are likely to use most of the time:
- `page`: [Page] - Isolated page for this test run.
- `context`: [BrowserContext] - Isolated context for this test run. The `page` fixture belongs to this context as well. Learn how to [configure context](#modify-options) below.
- `browser`: [Browser] - Browsers are shared across tests to optimize resources. Learn how to [configure browser](#modify-options) below.
- `browserName` - The name of the browser currently running the test. Either `chromium`, `firefox` or `webkit`.
## Test and assertion features
### Focus or skip tests
```js
import { test, expect } from '@playwright/test';
// You can focus single test.
test.only('focus this test', async ({ page }) => {
// Only this test in the entire project runs.
});
// You can skip tests.
test.skip('skip this test', async ({ page }) => {
});
```
### Group tests together
```js
import { test, expect } from '@playwright/test';
test.describe('two tests', () => {
test.only('one', async ({ page }) => {
// ...
});
test.skip('two', async ({ page }) => {
// ...
});
});
```
### Use test hooks
You can use `test.beforeAll` and `test.afterAll` hooks to set up and tear down resources shared between tests.
And you can use `test.beforeEach` and `test.afterEach` hooks to set up and tear down resources for each test individually.
```js
import { test, expect } from '@playwright/test';
test.describe('feature foo', () => {
test.beforeEach(async ({ page }) => {
// Go to the starting url before each test.
await page.goto('https://my.start.url');
});
test('my test', async ({ page }) => {
// Assertions use the expect API.
expect(page.url()).toBe('https://my.start.url');
});
});
```
## Write a configuration file
Create `config.ts` to configure your tests: specify browser launch options, run tests in multiple browsers and much more. Here is an example configuration that runs every test in Chromium, Firefox and WebKit.
```js
import { PlaywrightTestConfig } from '@playwright/test';
const config: PlaywrightTestConfig = {
timeout: 30000, // Each test is given 30 seconds.
// A project per browser, each running all the tests.
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
headless: true,
viewport: { width: 1280, height: 720 },
},
},
{
name: 'webkit',
use: {
browserName: 'webkit',
headless: true,
viewport: { width: 1280, height: 720 },
},
},
{
name: 'firefox',
use: {
browserName: 'firefox',
headless: true,
viewport: { width: 1280, height: 720 },
},
}
],
};
export default config;
```
## Run the test suite
Tests can be run in single or multiple browsers, in parallel or sequentially.
```sh
# Run all tests across Chromium, Firefox and WebKit
$ npx folio --config=config.ts
# Run tests on a single browser
$ npx folio --config=config.ts --project=chromium
# Run tests sequentially
$ npx folio --config=config.ts --workers=1
# Retry failing tests
$ npx folio --config=config.ts --retries=2
# See all options
$ npx folio --help
```
Refer to the [command line documentation][folio-cli] for all options.
### Configure NPM scripts
Save the run command as an NPM script.
```json
{
"scripts": {
"test": "npx folio --config=config.ts"
}
}
```
[folio]: https://github.com/microsoft/folio
[folio-annotations]: https://github.com/microsoft/folio#annotations
[folio-cli]: https://github.com/microsoft/folio#command-line
[folio-reporters]: https://github.com/microsoft/folio#reporters

8
docs/src/test-runners-js.md
View file

@ -1,12 +1,16 @@
--- ---
id: test-runners id: test-runners
title: "Test Runners" title: "Third party runners"
--- ---
With a few lines of code, you can hook up Playwright to your favorite JavaScript test runner. With a few lines of code, you can hook up Playwright to your existing JavaScript test runner.
<!-- TOC --> <!-- TOC -->
## @playwright/test
[@playwright/test](./test-runner-intro.md) is our first-party recommended test runner to be used with Playwright. Learn more about it [here](./test-runner-intro.md).
## Jest / Jasmine ## Jest / Jasmine
For Jest, [jest-playwright](https://github.com/playwright-community/jest-playwright) can be used. However for a light-weight solution, requiring playwright directly works fine. Jest shares it's syntax with Jasmine, so this applies to Jasmine as well. For Jest, [jest-playwright](https://github.com/playwright-community/jest-playwright) can be used. However for a light-weight solution, requiring playwright directly works fine. Jest shares it's syntax with Jasmine, so this applies to Jasmine as well.