This document provides an overview of Playwright's continuous integration, testing infrastructure, and release management processes. It covers the GitHub Actions workflows that ensure code quality, the multi-stage testing strategy across browsers and platforms, and the mechanisms for publishing releases to various distribution channels.
For detailed information about specific test workflows and their configurations, see page 7.1. For release processes including npm publishing, driver distribution, and Docker image creation, see page 7.2. For the process of tracking API changes across language ports, see page 7.3.
Playwright's CI/CD infrastructure is built entirely on GitHub Actions and consists of several coordinated systems:
utils/roll_browser.js</old_str> <new_str>
utils/roll_browser.jsWorkflow File Structure
Workflow Triggers and Purposes
| Workflow | Trigger | Jobs | Key Scripts/Actions |
|---|---|---|---|
tests_primary.yml | push, pull_request on main/release-* | test_linux, test_linux_chromium_tot, test_test_runner, test_vscode_extension, test_package_installations | .github/actions/run-test |
tests_secondary.yml | push on main/release-*, pull_request labeled | test_linux, test_mac, test_win, headed_tests, transport_linux, tracing_linux, test_chromium_channels, chromium_tot, firefox_beta | .github/actions/run-test |
tests_components.yml | push, pull_request on main/release-* | test_components | npm run ct |
tests_video.yml | push on main/release-* | video_linux | PWTEST_VIDEO=1 |
infra.yml | push, pull_request on main/release-* | doc-and-lint, lint-snippets | npm run lint, utils/doclint/linting-code-snippets/cli.js |
create_test_report.yml | workflow_run completion of tests 1/2/others/MCP | merge-reports | npx playwright merge-reports --config .github/workflows/merge.config.ts |
publish_release_docker.yml | release published, workflow_dispatch | publish-docker-release | utils/docker/publish_docker.sh |
roll_browser_into_playwright.yml | repository_dispatch: roll_into_pw | roll | utils/roll_browser.js |
pr_check_client_side_changes.yml | push on main (client-side paths) | check | github.rest.issues.create on Python/Java/.NET repos |
cherry_pick_into_release_branch.yml | workflow_dispatch | roll | git cherry-pick |
Sources: .github/workflows/tests_primary.yml1-225 .github/workflows/tests_secondary.yml1-327 .github/workflows/create_test_report.yml1-71 .github/workflows/publish_release_docker.yml1-41 .github/workflows/roll_browser_into_playwright.yml1-79 .github/workflows/pr_check_client_side_changes.yml1-75 .github/workflows/cherry_pick_into_release_branch.yml1-87 .github/workflows/infra.yml1-62 .github/workflows/tests_components.yml1-45 .github/workflows/tests_video.yml1-39
Playwright uses a multi-dimensional test matrix to ensure comprehensive coverage across browsers, operating systems, Node.js versions, and execution modes.
Matrix Strategy Configuration
Primary Test Matrix (tests_primary.yml) — runs on push and pull_request:
| Job | Browsers | OS / Node | Notes |
|---|---|---|---|
test_linux | chromium, firefox, webkit | ubuntu-22.04 × Node 20, 22, 24; ubuntu-22.04-arm × Node 20 | Core browser coverage |
test_linux_chromium_tot | chromium-tip-of-tree | ubuntu-22.04 | Tracks upstream Chromium |
test_test_runner | (no browser install) | ubuntu, macOS, Windows × Node 20, 22, 24; sharded | Runs npm run ttest |
test_vscode_extension | chromium | ubuntu-latest | Clones and tests playwright-vscode |
test_package_installations | (installer tests) | ubuntu, macOS, Windows | Validates npm, yarn, pnpm installs |
Secondary Test Matrix (tests_secondary.yml) — runs on push; pull_request only when labeled:
| Job | Browsers | OS Coverage | Notes |
|---|---|---|---|
test_linux | chromium, firefox, webkit | ubuntu-24.04 | Complements primary Ubuntu coverage |
test_mac | chromium, firefox, webkit | macos-14/15 Intel + ARM64; macos-26 for WebKit | Multi-arch macOS |
test_win | chromium, firefox, webkit | windows-latest | Firefox uses --workers 1 |
headed_tests | chromium, firefox, webkit | ubuntu-24.04, macos-14-xlarge, windows-latest, ubuntu-22.04 | --headed flag |
transport_linux | chromium | ubuntu-22.04 | PWTEST_MODE=driver and service |
tracing_linux | chromium, firefox, webkit | ubuntu-22.04/24.04 | PWTEST_TRACE=1 |
test_chromium_channels | chrome, chrome-beta, msedge, msedge-beta, msedge-dev | ubuntu, macOS, Windows | Stable channel binaries |
chromium_tot | chromium-tip-of-tree | ubuntu, macOS, Windows (headed + headless) | Excludes ubuntu headless (in primary) |
firefox_beta | firefox-beta | ubuntu, Windows, macOS | Tracks Firefox Beta |
Sources: .github/workflows/tests_primary.yml30-224 .github/workflows/tests_secondary.yml27-327
The test runner tests use weighted sharding to balance execution time:
Sources: .github/workflows/tests_primary.yml86-150
Playwright's release process covers multiple distribution targets. Docker publishing is automated via publish_release_docker.yml, which triggers on GitHub release publication and uses utils/docker/publish_docker.sh to push multi-architecture images (amd64, arm64) to DockerHub and Azure Container Registry. Additional npm and driver release workflows are covered in detail in page 7.2.
Docker Release Flow
Sources: .github/workflows/publish_release_docker.yml1-41
Playwright's distributed test execution uses blob reports to aggregate results from all parallel jobs into a single unified report.
Blob Report Workflow
Implementation Details
The merge process follows these steps:
.github/actions/upload-blob-report which uploads its blob-report/ directory as a GitHub artifact with a unique namecreate_test_report.yml triggers via workflow_run event when workflows ["tests 1", "tests 2", "tests others", "MCP"] complete.github/actions/download-artifact downloads all artifacts matching namePrefix: 'blob-report' into all-blob-reports/npx playwright merge-reports --config .github/workflows/merge.config.ts ./all-blob-reports with environment variable HTML_REPORT_URL setazcopy cp with Azure CLI authentication to upload to blob storage at path run-{workflow_run.id}-{run_attempt}-{sha}/The merge configuration file .github/workflows/merge.config.ts customizes the HTML reporter to include the Azure URL and GitHub token for PR commenting.
Sources: .github/workflows/create_test_report.yml1-54 .github/workflows/tests_primary.yml184-189 .github/workflows/tests_secondary.yml36-45
Playwright automates browser version updates through the roll_browser_into_playwright.yml workflow, which processes repository_dispatch events from external browser build systems.
Browser Roll Automation Process
Workflow Configuration
The workflow is defined in .github/workflows/roll_browser_into_playwright.yml1-79 with these key elements:
on: repository_dispatch: types: [roll_into_pw]github.event.client_payload
BROWSER: Browser name (chromium, firefox, webkit)REVISION: Browser revision number (e.g., 1148)BROWSER_VERSION: Semantic browser version (e.g., 145.0)group: 'roll-browser-into-playwright-${{ browser }}-${{ revision }}' prevents duplicate rollsactions/create-github-app-token@v2) to enable CI workflow triggering on created PRRoll Script Operation
The utils/roll_browser.js script performs:
packages/playwright-core/browsers.json with new revision and browserVersionnpm run build to regenerate derived filesSources: .github/workflows/roll_browser_into_playwright.yml1-79
Test workflows integrate with an Azure-hosted flakiness dashboard to track and analyze test stability over time.
Flakiness Dashboard Architecture
Integration Mechanism
The flakiness dashboard integration is controlled by:
Environment Gating: Jobs specify environment: ${{ github.event_name == 'push' && 'allow-uploading-flakiness-results' || null }}
main or release-* branches have the environment setOIDC Authentication: Workflows declare permissions: id-token: write to enable OpenID Connect authentication
azure/login@v2 in the run-test actionBot Identification: Each job provides a unique bot-name identifying the test configuration
{browser}-{os} or {browser}-{os}-node{version}"chromium-ubuntu-22.04-node20", "firefox-ubuntu-22.04-node20"Conditional Execution: The run-test action only uploads if all three flakiness credential secrets are provided
Sources: .github/workflows/tests_primary.yml32-63 .github/workflows/tests_secondary.yml29-45 .github/workflows/tests_secondary.yml73-89 .github/actions/run-test/action.yml75-87
The infra.yml workflow provides pre-merge validation of code quality and documentation.
| Check | Command | Purpose |
|---|---|---|
| Linting | npm run lint | ESLint, TypeScript, and documentation linting |
| Clean tree | git status -s | Ensures npm run build doesn't modify tracked files |
| NPM audit | node utils/check_audit.js | Security vulnerability scanning (continue-on-error) |
| Snippet validation | node utils/doclint/linting-code-snippets/cli.js | Validates code examples in all languages |
The snippet validation step:
Sources: .github/workflows/infra.yml16-62
Playwright defines custom composite actions in .github/actions/ to reduce duplication across workflows.
Action Architecture
The run-test action orchestrates the complete test execution lifecycle:
actions/setup-node@v4enable-microphone-access actionnpm ci and npm run buildnpx playwright install --with-depsxvfb-run on Linux)utils/upload_flakiness_dashboard.sh (on push events only)upload-blob-report (on pull_request or failure)Sources: .github/actions/run-test/action.yml1-94 .github/actions/upload-blob-report/action.yml1-24 .github/actions/enable-microphone-access/action.yml1-26 .github/workflows/tests_primary.yml55-63 .github/workflows/tests_secondary.yml38-45
The cherry_pick_into_release_branch.yml workflow automates backporting commits to release branches:
The workflow:
1.25)cherry-pick(#PR): prefixSources: .github/workflows/cherry_pick_into_release_branch.yml1-87
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.