cherry-pick: ToT docs (#16413)

* docs: Remove html reporter page (#16407)

* docs: intro doc on ci (#16346)

* docs: Trace-viewer-intro (#16254)

Co-authored-by: Debbie O'Brien <debs-obrien@users.noreply.github.com>
This commit is contained in:
Max Schmitt 2022-08-10 14:52:52 +02:00 committed by GitHub
parent e93ed0ae97
commit 150a75189e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
16 changed files with 343 additions and 34 deletions

115
docs/src/ci-intro-js.md Normal file
View file

@ -0,0 +1,115 @@
---
id: ci-intro
title: "CI Github Actions"
---
When installing Playwright you are given the option to add a [GitHub Actions](https://docs.github.com/en/actions). This creates a `playwright.yml` file inside a `.github/workflows` folder containing everything you need so that your tests run on each push and pull request into the main/master branch.
**What you will learn:**
- [How to use GitHub Actions to run your tests](#github-actions)
- [How to create a repo and push to GitHub](#create-a-repo-and-push-to-github)
- [How to open the workflows](#opening-the-workflows)
- [How to view the test logs](#viewing-test-logs)
- [How to download the report from GitHub](#downloading-the-playwright-report)
- [How to view the report](#viewing-the-playwright-report)
- [How to view the trace](#viewing-the-trace)
## GitHub Actions
Tests will run on push or pull request on branches main/master. The [workflow](https://docs.github.com/en/actions/using-workflows/about-workflows) will install all dependencies, install Playwright and then run the tests. It will also create the HTML report.
```yaml
name: Playwright Tests
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: "14.x"
- name: Install dependencies
run: npm ci
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npx playwright test
- uses: actions/upload-artifact@v2
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30
```
### Create a Repo and Push to GitHub
[Create a repo on GitHub](https://docs.github.com/en/get-started/quickstart/create-a-repo) and create a new repository or push an existing repository. Follow the instructions on GitHub and don't forget to [initialize a git repository](https://github.com/git-guides/git-init) using the `git init` command so you can [add](https://github.com/git-guides/git-add), [commit](https://github.com/git-guides/git-commit) and [push](https://github.com/git-guides/git-push) your code.
<img width="861" alt="Create a Repo and Push to GitHub" src="https://user-images.githubusercontent.com/13063165/183423254-d2735278-a2ab-4d63-bb99-48d8e5e447bc.png"/>
### Opening the Workflows
Click on the **Actions** tab to see the workflows. Here you will see if your tests have passed or failed.
<img width="847" alt="Opening the Workflows" src="https://user-images.githubusercontent.com/13063165/183423584-2ea18038-cd49-4daa-a20c-2205352f0933.png"/>
On Pull Requests you can also click on the **Details** link in the [PR status check](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks).
<img width="645" alt="pr status checked" src="https://user-images.githubusercontent.com/13063165/183722462-17a985db-0e10-4205-b16c-8aaac36117b9.png" />
### Viewing Test Logs
Clicking on the workflow run will show you the all the actions that GitHub performed and clicking on **Run Playwright tests** will show the error messages, what was expected and what was received as well as the call log.
<img width="839" alt="Viewing Test Logs" src="https://user-images.githubusercontent.com/13063165/183423783-58bf2008-514e-4f96-9c12-c9a55703960c.png"/>
## HTML Report
The HTML Report shows you a full report of your tests. You can filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests.
### Downloading the HTML Report
In the Artifacts section click on the **playwright-report** to download your report in the format of a zip file.
<img width="972" alt="Downloading the HTML Report" src="https://user-images.githubusercontent.com/13063165/183437023-524f1803-84e4-4862-9ce3-1d55af0e023e.png" />
### Viewing the HTML Report
Locally opening the report will not work as expected as you need a web server in order for everything to work correctly. First, extract the zip, preferably in a folder that already has Playwright installed. Using the command line change into the directory where the report is and use `npx playwright show-report` followed by the name of the extracted folder. This will serve up the report and enable you to view it in your browser.
```bash
npx playwright show-report name-of-my-extracted-playwright-report
```
<img width="752" alt="Viewing the HTML Report" src="https://user-images.githubusercontent.com/13063165/183437645-b47dd175-2e07-4ecc-a469-27d5b150b7ed.png" />
To learn more about reports check out our detailed guide on [HTML Reporter](/test-reporters.md#html-reporter)
### Viewing the Trace
Once you have served the report using `npx playwright show-report`, click on the trace icon next to the test's file name as seen in the image above. You can then view the trace of your tests and inspect each action to try to find out why the tests are failing.
<img width="1907" alt="Viewing the Trace" src="https://user-images.githubusercontent.com/13063165/183879653-d442e6b4-14f5-4d0e-99f3-9ba19f82c7cf.png"/>
To learn more about traces check out our detailed guide on [Trace Viewer](/trace-viewer.md).
To learn more about running tests on CI check out our detailed guide on [Continuous Integration](/ci.md).
## What's Next
- [Learn how to use Web First Assertions](/test-assertions.md)
- [Learn how to use Selectors](/selectors.md)
- [Learn how to use Locators](/locators.md)

View file

@ -288,4 +288,4 @@ await page.PauseAsync();
## What's Next ## What's Next
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -93,4 +93,4 @@ As you interact with the page Codegen will generate the test for you in the newl
- [Write tests using web first assertions, page fixtures and locators](./writing-tests.md) - [Write tests using web first assertions, page fixtures and locators](./writing-tests.md)
- [See test reports](./running-tests.md#test-reports) - [See test reports](./running-tests.md#test-reports)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -1,16 +0,0 @@
---
id: html-reporter
title: "HTML Reporter"
---
The HTML Reporter shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed.
```bash
npx playwright show-report
```
<img width="739" alt="image" src="https://user-images.githubusercontent.com/13063165/178003817-3bd2f088-4173-406c-a9e9-74c89181f381.png" />
You can click on each test to get more info on the test and explore the errors section which will show you what errors there are and on what line as well as suggestions on how to fix the errors. Each step of the test can be expanded and looked into in detail.
<img width="747" alt="image" src="https://user-images.githubusercontent.com/13063165/178007807-b88e4d22-cd40-4a07-81ab-50096eb3ea1b.png" />

View file

@ -194,5 +194,5 @@ See our doc on [Test Runners](./test-runners.md) to learn more about running tes
- [Run single tests, multiple tests, headed mode](./running-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md)
- [Learn more about the NUnit and MSTest base classes](./test-runners.md) - [Learn more about the NUnit and MSTest base classes](./test-runners.md)
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)
- [Using Playwright as library](./library.md) - [Using Playwright as library](./library.md)

View file

@ -82,7 +82,7 @@ See our doc on [Running Tests](./running-tests.md) to learn more about running t
## HTML Test Reports ## HTML Test Reports
Once your test has finished running a [HTML Reporter](./html-reporter.md) will have been created which shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. You can click on each test and explore the test's errors as well as each step of the test. By default, the HTML report is opened automatically if some of the tests failed. Once your test has finished running a [HTML Reporter](./test-reporters.md#html-reporter) will have been created which shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. You can click on each test and explore the test's errors as well as each step of the test. By default, the HTML report is opened automatically if some of the tests failed.
```bash ```bash
npx playwright show-report npx playwright show-report
@ -96,4 +96,4 @@ npx playwright show-report
- [Write tests using web first assertions, page fixtures and locators](./writing-tests.md) - [Write tests using web first assertions, page fixtures and locators](./writing-tests.md)
- [Run single tests, multiple tests, headed mode](./running-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md)
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -64,4 +64,4 @@ See our doc on [Running Tests](./running-tests.md) to learn more about running t
- [Write tests using web first assertions, page fixtures and locators](./writing-tests.md) - [Write tests using web first assertions, page fixtures and locators](./writing-tests.md)
- [Run single tests, multiple tests, headed mode](./running-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md)
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -93,4 +93,4 @@ Check out our [debugging guide](./debug.md) to learn more about the [Playwright
## What's Next ## What's Next
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -90,7 +90,7 @@ Check out our [debugging guide](./debug.md) to learn more about the [Playwright
## Test Reports ## Test Reports
The [HTML Reporter](./html-reporter.md) shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed. The [HTML Reporter](././test-reporters.md#html-reporter) shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed.
```bash ```bash
npx playwright show-report npx playwright show-report
@ -105,4 +105,4 @@ You can click on each test and explore the tests errors as well as each step of
## What's Next ## What's Next
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -83,4 +83,4 @@ Check out our [debugging guide](./debug.md) to learn more about the [Playwright
## What's Next ## What's Next
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -0,0 +1,120 @@
---
id: trace-viewer-intro
title: "Trace Viewer"
---
Playwright Trace Viewer is a GUI tool that lets you explore recorded Playwright traces of your tests meaning you can go back and forward though each action of your test and visually see what was happening during each action.
**You will learn**
- How to record a trace
- How to open the HTML report
- How to open the trace viewer
## Recording a trace
Traces can be recorded using the [`property: BrowserContext.tracing`] API as follows:
```java
Browser browser = browserType.launch();
BrowserContext context = browser.newContext();
// Start tracing before creating / navigating a page.
context.tracing().start(new Tracing.StartOptions()
.setScreenshots(true)
.setSnapshots(true)
.setSources(true));
Page page = context.newPage();
page.navigate("https://playwright.dev");
// Stop tracing and export it into a zip archive.
context.tracing().stop(new Tracing.StopOptions()
.setPath(Paths.get("trace.zip")));
```
```python async
browser = await chromium.launch()
context = await browser.new_context()
# Start tracing before creating / navigating a page.
await context.tracing.start(screenshots=True, snapshots=True, sources=True)
await page.goto("https://playwright.dev")
# Stop tracing and export it into a zip archive.
await context.tracing.stop(path = "trace.zip")
```
```python sync
browser = chromium.launch()
context = browser.new_context()
# Start tracing before creating / navigating a page.
context.tracing.start(screenshots=True, snapshots=True, sources=True)
page.goto("https://playwright.dev")
# Stop tracing and export it into a zip archive.
context.tracing.stop(path = "trace.zip")
```
```csharp
await using var browser = playwright.Chromium.LaunchAsync();
await using var context = await browser.NewContextAsync();
// Start tracing before creating / navigating a page.
await context.Tracing.StartAsync(new()
{
Screenshots = true,
Snapshots = true,
Sources = true
});
var page = context.NewPageAsync();
await page.GotoAsync("https://playwright.dev");
// Stop tracing and export it into a zip archive.
await context.Tracing.StopAsync(new()
{
Path = "trace.zip"
});
```
This will record the trace and place it into the file named `trace.zip`.
## Opening the trace
You can open the saved trace using Playwright CLI or in your browser on [`trace.playwright.dev`](https://trace.playwright.dev).
```bash js
npx playwright show-trace trace.zip
```
```bash java
mvn exec:java -e -Dexec.mainClass=com.microsoft.playwright.CLI -Dexec.args="show-trace trace.zip"
```
```bash python
playwright show-trace trace.zip
```
```bash csharp
pwsh bin\Debug\netX\playwright.ps1 show-trace trace.zip
```
## Viewing the trace
View traces of your test by clicking through each action or hovering using the timeline and see the state of the page before and after the action. Inspect the log, source and network during each step of the test. The trace viewer creates a DOM snapshot so you can fully interact with it, open devtools etc.
<img width="941" alt="image" src="https://user-images.githubusercontent.com/13063165/182618490-3340cfbf-7ac9-46e2-8157-6a8ce52dca28.png" />
To learn more check out our detailed guide on [Trace Viewer](/trace-viewer.md).

View file

@ -0,0 +1,89 @@
---
id: trace-viewer-intro
title: "Trace Viewer"
---
Playwright Trace Viewer is a GUI tool that lets you explore recorded Playwright traces of your tests meaning you can go back and forward though each action of your test and visually see what was happening during each action.
**You will learn**
- [How to record a trace](/trace-viewer-intro.md#recording-a-trace)
- [How to open the HTML report](/trace-viewer-intro.md#opening-the-html-report)
- [How to open and view the trace](/trace-viewer-intro.md#viewing-the-trace)
## Recording a Trace
By default the [playwright.config](/test-configuration.md#record-test-trace) file will contain the configuration needed to create a `trace.zip` file for each test. Traces are setup to run `on-first-retry` meaning they will be run on the first retry of a failed test. Also `retires` are set to 2 when running on CI and 0 locally. This means the traces will be recorded on the first retry of a failed test but not on the first run and not on the second retry.
```js tab=js-js
// @ts-check
/** @type {import('@playwright/test').PlaywrightTestConfig} */
const config = {
retries: process.env.CI ? 2 : 0, // set to 2 when running on CI
...
use: {
trace: 'on-first-retry', // record traces on first retry of each test
},
};
module.exports = config;
```
```js tab=js-ts
import type { PlaywrightTestConfig } from '@playwright/test';
const config: PlaywrightTestConfig = {
retries: process.env.CI ? 2 : 0, // set to 2 when running on CI
...
use: {
trace: 'on-first-retry', // record traces on first retry of each test
},
};
export default config;
```
To learn more about available options to record a trace check out our detailed guide on [Trace Viewer](/trace-viewer.md).
Traces are normally run in a Continuous Integration(CI) environment as locally you can use [debugging](/debug.md) methods to debug tests. However should you want to run traces locally you can force tracing to be on with `--trace on`.
```bash
npx playwright test --trace on
```
:::note
The `trace-on` flag was introduced in Playwright v1.25. Check your `package.json` to sure you have at least this version of Playwright installed.
:::
## Opening the HTML Report
If you have a failed test then tests will run a total of 3 times. On the first retry the trace will be recorded. After the second retry the tests will stop running and a HTML report is available to view.
```bash
npx playwright show-report
```
In the HTML report click on the trace icon to directly open the trace file.
<img width="744" alt="image" src="https://user-images.githubusercontent.com/13063165/182853447-e26f4d39-b4e2-4d9b-a890-ac1838c088e1.png" />
You can also click on the test file and then click the 'Retry #1' tab which will show you a traces section in your html report. Here you can open the trace by clicking on the screenshot.
<img width="749" alt="image" src="https://user-images.githubusercontent.com/13063165/183130559-16a83a39-2f1d-4560-850c-d025fad789b3.png" />
To learn more about reporters check out our detailed guide on reporters including the [HTML Reporter](/test-reporters.md#html-reporter).
## Viewing the Trace
View traces of your test by clicking through each action or hovering using the timeline and see the state of the page before and after the action. Inspect the log, source and network during each step of the test. The trace viewer creates a DOM snapshot so you can fully interact with it, open devtools etc.
<img width="941" alt="image" src="https://user-images.githubusercontent.com/13063165/182618490-3340cfbf-7ac9-46e2-8157-6a8ce52dca28.png" />
To learn more about traces check out our detailed guide on [Trace Viewer](/trace-viewer.md).

View file

@ -52,13 +52,14 @@ await page.goto('https://playwright.dev');
await context.tracing.stop({ path: 'trace.zip' }); await context.tracing.stop({ path: 'trace.zip' });
``` ```
You can also use `trace: 'retain-on-failure'` if you do not enable retries but still want traces for failed tests.
Available options to record a trace: Available options to record a trace:
- `'off'` - Do not record a trace.
- `'on'` - Record a trace for each test.
- `'retain-on-failure'` - Record a trace for each test, but remove it from successful test runs.
- `'on-first-retry'` - Record a trace only when retrying a test for the first time. - `'on-first-retry'` - Record a trace only when retrying a test for the first time.
- `'off'` - Do not record a trace.
- `'on'` - Record a trace for each test. (not recommended as it's performance heavy)
- `'retain-on-failure'` - Record a trace for each test, but remove it from successful test runs.
You can also use `trace: 'retain-on-failure'` if you do not enable retries but still want traces for failed tests.
If you are not using Playwright Test, use the [`property: BrowserContext.tracing`] API instead. If you are not using Playwright Test, use the [`property: BrowserContext.tracing`] API instead.

View file

@ -237,4 +237,4 @@ public class UnitTest1 : PageTest
- [Run single tests, multiple tests, headed mode](./running-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md)
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -146,4 +146,4 @@ test.describe("navigation", () => {
- [Run single tests, multiple tests, headed mode](./running-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md)
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)

View file

@ -104,4 +104,4 @@ def test_main_navigation(page: Page):
- [Run single tests, multiple tests, headed mode](./running-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md)
- [Generate tests with Codegen](./codegen.md) - [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer.md) - [See a trace of your tests](./trace-viewer-intro.md)