This page documents the GitHub Actions workflows that run Playwright's automated test suite on every push and pull request. It covers the two primary test workflows (tests_primary.yml, tests_secondary.yml), the reusable run-test composite action, the flakiness dashboard upload mechanism, and the create_test_report workflow that aggregates blob reports from all shards into a single HTML report.
For information about release publishing and browser rolling, see Release and Distribution. For the cross-language tracking workflow, see Cross-Language Synchronization.
Both primary test workflows share the same trigger logic: they fire on push to main or release-* branches, and on pull_request against those branches with certain path exclusions.
tests 1 (tests_primary.yml) excludes pull request paths under browser_patches/, docs/, packages/playwright/src/mcp/, and tests/mcp/.
tests 2 (tests_secondary.yml) excludes browser_patches/ and docs/ on pull requests, and additionally requires pull requests to carry a labeled event type (i.e., it only runs when a label is added to a PR).
Pull request runs use GitHub's concurrency groups keyed on github.workflow + github.head_ref, with cancel-in-progress: true, so only the latest commit per branch is tested.
Sources: .github/workflows/tests_primary.yml1-27 .github/workflows/tests_secondary.yml1-25
run-test Composite ActionAll test jobs delegate to the reusable composite action at .github/actions/run-test/action.yml It is the single point of truth for how a test run is set up and reported.
Inputs:
| Input | Required | Default | Description |
|---|---|---|---|
command | yes | — | Full shell command to execute the tests |
node-version | no | 20 | Node.js version to install |
browsers-to-install | no | '' | Space-separated list passed to npx playwright install --with-deps |
bot-name | yes | — | Unique identifier for this job; used as PW_TAG and blob report name |
shard-index | no | 0 | Numeric shard index; appended to the blob report artifact name |
shell | no | bash | Shell type; pwsh is used on Windows where needed |
flakiness-client-id | no | — | Azure OIDC credentials for flakiness dashboard upload |
flakiness-tenant-id | no | — | |
flakiness-subscription-id | no | — |
Step sequence inside the composite action:
Figure 1: run-test action step sequence
Sources: .github/actions/run-test/action.yml36-94
Key environment variable: PW_TAG is set to @<bot-name> for every run. This tag is consumed by the test runner to annotate results for the flakiness dashboard and blob report.
On Linux, the test command is wrapped in xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" to provide a virtual framebuffer for headed browser sessions.
Sources: .github/actions/run-test/action.yml58-74
This workflow runs the core cross-browser and test runner matrix on every PR and push.
Figure 2: tests_primary.yml job matrix
Sources: .github/workflows/tests_primary.yml29-225
| Job | npm Script | Description |
|---|---|---|
test_linux | npm run test -- --project=${{ matrix.browser }}-* | Browser-specific projects |
test_linux_chromium_tot | npm run test -- --project=chromium-* | Tip-of-tree Chromium |
test_test_runner | npm run ttest -- --shard N/M | Test runner self-tests, sharded |
test_vscode_extension | npm run test (inside playwright-vscode/) | VS Code extension tests |
test_package_installations | npm run itest | Package installation tests |
The test_test_runner job uses a custom PWTEST_SHARD_WEIGHTS environment variable (e.g., 58:42 for 2 shards, 44:33:23 for 3 shards) and sets PWTEST_CHANNEL=firefox-beta.
Sources: .github/workflows/tests_primary.yml89-153
This workflow runs extended platform coverage, headed mode tests, transport layer tests, tracing tests, Chromium channel variants, and Firefox Beta.
| Job | Platforms | Command | Notable env vars |
|---|---|---|---|
test_linux | ubuntu-24.04 × {chromium, firefox, webkit} | npm run test -- --project=$browser-* | — |
test_mac | macos-{14,15}-{large,xlarge} + macos-26-xlarge/webkit × browsers | npm run test -- --project=$browser-* | — |
test_win | windows-latest × {chromium, firefox, webkit} | npm run test -- --project=$browser-* [--workers 1 for firefox] | — |
headed_tests | ubuntu-24.04, macos-14-xlarge, windows-latest, ubuntu-22.04(webkit) × browsers | npm run test -- --project=$browser-* --headed | — |
transport_linux | ubuntu-22.04 × {driver, service} | npm run ctest | PWTEST_MODE=$mode |
tracing_linux | ubuntu-22.04 / ubuntu-24.04 × {chromium, firefox, webkit, chromium-tip-of-tree} | npm run test -- --project=$browser-* | PWTEST_TRACE=1 |
test_chromium_channels | ubuntu-22.04, macos-latest, windows-latest × {chrome, chrome-beta, msedge, msedge-beta, msedge-dev} | npm run ctest | PWTEST_CHANNEL=$channel |
chromium_tot | ubuntu-22.04, macos-14-large, windows-latest × {headed, headless} | npm run ctest -- $headed | PWTEST_CHANNEL=chromium-tip-of-tree |
chromium_tot_headless_shell | ubuntu-22.04 | npm run ctest | PWTEST_CHANNEL=chromium-tip-of-tree-headless-shell |
firefox_beta | ubuntu-22.04, windows-latest, macos-latest | npm run ftest | PWTEST_CHANNEL=firefox-beta |
build-playwright-driver | ubuntu-24.04 | utils/build/build-playwright-driver.sh | — |
test-package-installations-other-node-versions | ubuntu-latest × {node 20, 22, 24} | npm run itest | — |
test_channel_chromium | ubuntu/windows/macos-latest | npm run ctest | PWTEST_CHANNEL=chromium |
Sources: .github/workflows/tests_secondary.yml26-327
The transport_linux job exercises the Playwright driver protocol in two modes by setting PWTEST_MODE to either driver (in-process driver) or service (out-of-process service). Both run npm run ctest (Chromium-only tests) to validate transport layer correctness.
Sources: .github/workflows/tests_secondary.yml150-170
The headed_tests job in tests_secondary.yml passes the --headed flag, which causes browsers to open a real window. On Linux this still requires xvfb-run, which the run-test composite action handles automatically. An extra webkit / ubuntu-22.04 row is added because WebKit ships different binaries per Ubuntu major version.
Sources: .github/workflows/tests_secondary.yml126-148
Every job that calls run-test can optionally upload its results to an Azure-hosted flakiness dashboard. The upload only occurs on push events to microsoft/playwright (not on forks or PRs).
Flow:
environment is set to allow-uploading-flakiness-results when github.event_name == 'push'. This environment gate provides the OIDC secrets.run-test, after the tests complete, azure/login@v2 authenticates via OIDC using the three Azure secrets (flakiness-client-id, flakiness-tenant-id, flakiness-subscription-id)../utils/upload_flakiness_dashboard.sh ./test-results/report.json uploads the JSON report.The bot-name input (e.g., chromium-ubuntu-22.04-node20) is passed as PW_TAG=@<bot-name> to identify the result source in the dashboard.
Sources: .github/actions/run-test/action.yml75-87 .github/workflows/tests_primary.yml32-33
Playwright's sharded test runs produce per-shard blob reports (zip archives) that are collected and merged into a single HTML report by the create_test_report workflow.
Figure 3: Blob report flow from test jobs to HTML report
Sources: .github/actions/upload-blob-report/action.yml1-24 .github/workflows/create_test_report.yml1-71
upload-blob-report ActionThe upload-blob-report composite action (.github/actions/upload-blob-report/action.yml) takes:
report_dir: directory containing the blob zip files (default: blob-report)job_name: used to name the artifact blob-report-<job_name>It first runs an integrity check (unzip -t on every .zip) before calling actions/upload-artifact@v4 with a 7-day retention period.
Upload is conditional: if: ${{ !cancelled() && (github.event_name == 'pull_request' || failure()) }} — meaning blob reports are always uploaded on PRs (for the merged report), and on push only when the job fails.
Sources: .github/actions/run-test/action.yml88-93 .github/actions/upload-blob-report/action.yml12-24
create_test_report.ymlThis workflow is triggered by workflow_run completion for any of: tests 1, tests 2, tests others, or MCP. It:
blob-report into all-blob-reports/.npx playwright merge-reports --config .github/workflows/merge.config.ts ./all-blob-reports with NODE_OPTIONS=--max-old-space-size=8192.playwright-report/ to Azure Blob Storage at a path keyed by run_id, run_attempt, and sha.HTML Report (<workflow name>).The HTML_REPORT_URL environment variable is pre-computed and passed to the merge step so it can embed a correct self-referential link.
Sources: .github/workflows/create_test_report.yml1-71
Runs only on push to main or release-* (not on PRs). Runs all three browsers on both ubuntu-22.04 and ubuntu-24.04 with PWTEST_VIDEO=1. Uses the allow-uploading-flakiness-results environment unconditionally (not gated on push vs. PR).
Sources: .github/workflows/tests_video.yml1-39
Runs component tests (npm run ct) on ubuntu-latest, macos-latest, and windows-latest with Node.js 20, plus Node.js 22 and 24 on Ubuntu. Does not use the run-test composite action — it installs browsers via npx playwright install --with-deps directly and does not upload flakiness data or blob reports.
Sources: .github/workflows/tests_components.yml1-45
The following environment variables configure test behavior across the workflows:
| Variable | Source | Effect |
|---|---|---|
FORCE_COLOR=1 | workflow env | Forces ANSI color output in terminal reporters |
ELECTRON_SKIP_BINARY_DOWNLOAD=1 | workflow env | Prevents Electron binary download during npm ci |
PW_TAG | run-test action | Set to @<bot-name>; tags results for reporting |
PWTEST_CHANNEL | job env | Selects a specific browser channel (e.g., chromium-tip-of-tree, firefox-beta, chrome) |
PWTEST_MODE | job env | Selects transport mode: driver or service |
PWTEST_TRACE=1 | job env | Enables tracing for all tests in the tracing job |
PWTEST_VIDEO=1 | job env | Enables video recording for all tests |
PWTEST_SHARD_WEIGHTS | job env | Colon-separated shard weight ratios for the test runner job |
Sources: .github/workflows/tests_primary.yml24-27 .github/workflows/tests_secondary.yml17-20 .github/actions/run-test/action.yml66-74
Figure 4: Workflow and action dependency graph
Sources: .github/workflows/tests_primary.yml1-225 .github/workflows/tests_secondary.yml1-327 .github/workflows/create_test_report.yml1-71 .github/actions/run-test/action.yml1-94
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.