Filip/playwright
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

reduce ci-intro to barebones, pull in template updates from ci.md

Browse source
This commit is contained in:
Simon Knott 2024-08-02 15:55:27 +02:00
parent 537075db66
commit 9c26a8955e
No known key found for this signature in database
GPG key ID: 8CEDC00028084AEC
1 changed files with 26 additions and 276 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

302
docs/src/ci-intro.md
View file

@ -1,14 +1,12 @@
--- ---
id: ci-intro id: ci-intro
title: "CI GitHub Actions" title: "Setting up CI"
--- ---
## Introduction ## Introduction
* langs: js * langs: js
Playwright tests can be run on any CI provider. In this section we will cover running tests on GitHub using GitHub actions. If you would like to see how to configure other CI providers check out our detailed [doc on Continuous Integration](./ci.md). Playwright tests can be run on any CI provider. This guide covers one way of running tests on GitHub using GitHub actions. If you would like to learn more, or how to configure other CI providers, check out our detailed [doc on Continuous Integration](./ci.md).
When [installing Playwright](./intro.md) using the [VS Code extension](./getting-started-vscode.md) or with `npm init playwright@latest` you are given the option to add a [GitHub Actions](https://docs.github.com/en/actions) workflow. This creates a `playwright.yml` file inside a `.github/workflows` folder containing everything you need so that your tests run on each push and pull request into the main/master branch.
#### You will learn #### You will learn
* langs: js * langs: js
@ -25,22 +23,18 @@ When [installing Playwright](./intro.md) using the [VS Code extension](./getting
Playwright tests can be ran on any CI provider. In this section we will cover running tests on GitHub using GitHub actions. If you would like to see how to configure other CI providers check out our detailed doc on Continuous Integration. Playwright tests can be ran on any CI provider. In this section we will cover running tests on GitHub using GitHub actions. If you would like to see how to configure other CI providers check out our detailed doc on Continuous Integration.
To add a [GitHub Actions](https://docs.github.com/en/actions) file first create `.github/workflows` folder and inside it add a `playwright.yml` file containing the example code below so that your tests will run on each push and pull request for the main/master branch.
#### You will learn #### You will learn
* langs: python, java, csharp * langs: python, java, csharp
- [How to run tests on push/pull_request](/ci-intro.md#on-pushpull_request) - [How to set up GitHub Actions](/ci-intro.md#setting-up-GitHub-Actions)
- [How to view test logs](/ci-intro.md#viewing-test-logs) - [How to view test logs](/ci-intro.md#viewing-test-logs)
- [How to view the trace](/ci-intro.md#viewing-the-trace) - [How to view the trace](/ci-intro.md#viewing-the-trace)
## Setting up GitHub Actions ## Setting up GitHub Actions
### On push/pull_request
* langs: js * langs: js
Tests will run on push or pull request on branches main/master. The [workflow](https://docs.github.com/en/actions/using-workflows/about-workflows) will install all dependencies, install Playwright and then run the tests. It will also create the HTML report. When [installing Playwright](./intro.md) using the [VS Code extension](./getting-started-vscode.md) or with `npm init playwright@latest` you are given the option to add a [GitHub Actions](https://docs.github.com/en/actions) workflow. This creates a `playwright.yml` file inside a `.github/workflows` folder containing everything you need so that your tests run on each push and pull request into the main/master branch. Here's how that file looks:
```yml js title=".github/workflows/playwright.yml" ```yml js title=".github/workflows/playwright.yml"
name: Playwright Tests name: Playwright Tests
@ -57,7 +51,7 @@ jobs:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-node@v4 - uses: actions/setup-node@v4
with: with:
node-version: 18 node-version: lts/*
- name: Install dependencies - name: Install dependencies
run: npm ci run: npm ci
- name: Install Playwright Browsers - name: Install Playwright Browsers
@ -65,17 +59,26 @@ jobs:
- name: Run Playwright tests - name: Run Playwright tests
run: npx playwright test run: npx playwright test
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v4
if: ${{ !cancelled() }} if: always()
with: with:
name: playwright-report name: playwright-report
path: playwright-report/ path: playwright-report/
retention-days: 30 retention-days: 30
``` ```
### On push/pull_request If this doesn't make sense to you, give GitHub's [Understanding GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions) guide a read. Looking at the list of steps in `jobs.test.steps`, you can see that the workflow performs these steps:
1. Clone your repository
2. Install Node.js
3. Install NPM Dependencies
4. Install Playwright Browsers
5. Run Playwright tests
6. Upload HTML report to the GitHub UI
## Setting up GitHub Actions
* langs: python, java, csharp * langs: python, java, csharp
Tests will run on push or pull request on branches main/master. The [workflow](https://docs.github.com/en/actions/using-workflows/about-workflows) will install all dependencies, install Playwright and then run the tests. To add a [GitHub Actions](https://docs.github.com/en/actions) file first create `.github/workflows` folder and inside it add a `playwright.yml` file containing the example code below so that your tests will run on each push and pull request for the main/master branch.
```yml python title=".github/workflows/playwright.yml" ```yml python title=".github/workflows/playwright.yml"
name: Playwright Tests name: Playwright Tests
@ -126,7 +129,7 @@ jobs:
with: with:
distribution: 'temurin' distribution: 'temurin'
java-version: '17' java-version: '17'
- name: Build & Install - name: Ensure browsers are installed
run: mvn -B install -D skipTests --no-transfer-progress run: mvn -B install -D skipTests --no-transfer-progress
- name: Install Playwright - name: Install Playwright
run: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps" run: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps"
@ -158,268 +161,14 @@ jobs:
run: dotnet test run: dotnet test
``` ```
### On push/pull_request (sharded) If this doesn't make sense to you, give GitHub's [Understanding GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions) guide a read.
* langs: js
GitHub Actions supports [sharding tests between multiple jobs](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs). Check out our [sharding doc](./test-sharding) to learn more about sharding and to see a [GitHub actions example](./test-sharding.md#github-actions-example) of how to configure a job to run your tests on multiple machines as well as how to merge the HTML reports. Looking at the list of steps in `jobs.test.steps`, you can see that the workflow performs these steps:
### Via Containers 1. Clone your repository
2. Install language dependencies
GitHub Actions support [running jobs in a container](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container) by using the [`jobs.<job_id>.container`](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainer) option. This is useful to not pollute the host environment with dependencies and to have a consistent environment for e.g. screenshots/visual regression testing across different operating systems. 4. Install Playwright Browsers
5. Run tests
```yml js title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Run your tests
run: npx playwright test
env:
HOME: /root
```
```yml python title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r local-requirements.txt
pip install -e .
- name: Run your tests
run: pytest
env:
HOME: /root
```
```yml java title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'
- name: Build & Install
run: mvn -B install -D skipTests --no-transfer-progress
- name: Run tests
run: mvn test
env:
HOME: /root
```
```yml csharp title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy
steps:
- uses: actions/checkout@v4
- name: Setup dotnet
uses: actions/setup-dotnet@v4
with:
dotnet-version: 8.0.x
- run: dotnet build
- name: Run your tests
run: dotnet test
env:
HOME: /root
```
### On deployment
This will start the tests after a [GitHub Deployment](https://developer.github.com/v3/repos/deployments/) went into the `success` state.
Services like Vercel use this pattern so you can run your end-to-end tests on their deployed environment.
```yml js title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npx playwright test
env:
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
```yml python title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v4
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Ensure browsers are installed
run: python -m playwright install --with-deps
- name: Run tests
run: pytest
env:
# This might depend on your test-runner
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
```yml java title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'
- name: Build & Install
run: mvn -B install -D skipTests --no-transfer-progress
- name: Install Playwright
run: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps"
- name: Run tests
run: mvn test
env:
# This might depend on your test-runner
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
```yml csharp title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v4
- name: Setup dotnet
uses: actions/setup-dotnet@v4
with:
dotnet-version: 8.0.x
- run: dotnet build
- name: Ensure browsers are installed
run: pwsh bin/Debug/net8.0/playwright.ps1 install --with-deps
- name: Run tests
run: dotnet test
env:
# This might depend on your test-runner
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
### Fail-Fast
* langs: js
Even with sharding enabled, large test suites can take very long to execute. Running changed test files first on PRs will give you a faster feedback loop and use less CI resources.
```yml js title=".github/workflows/playwright.yml"
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run changed Playwright tests
run: npx playwright test --only-changed=$GITHUB_BASE_REF
if: github.event_name == 'pull_request'
- name: Run Playwright tests
run: npx playwright test
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 30
```
## Create a Repo and Push to GitHub ## Create a Repo and Push to GitHub
@ -569,4 +318,5 @@ This step will not work for pull requests created from a forked repository becau
- [Learn how to perform Actions](./input.md) - [Learn how to perform Actions](./input.md)
- [Learn how to write Assertions](./test-assertions.md) - [Learn how to write Assertions](./test-assertions.md)
- [Learn more about the Trace Viewer](/trace-viewer.md) - [Learn more about the Trace Viewer](/trace-viewer.md)
- [Learn more about running tests on other CI providers](/ci.md) - [Learn more ways of running tests on GitHub Actions](/ci.md)
- [Learn more about running tests on other CI providers](/ci.md#github-actions) // TODO: is this link correct?