Skip to main content
Stably integrates seamlessly with your existing GitHub Actions workflows. Use Stably’s Playwright SDK to unlock AI-powered testing capabilities including auto-heal, AI assertions, and centralized reporting—all while maintaining 100% compatibility with your existing Playwright tests.

Prerequisites

  • A GitHub repository with an existing GitHub Actions workflow (or the ability to create one)
  • Node.js 18+ configured in your workflow
  • Playwright tests in your repository

Quick Setup

1. Install Stably Playwright SDK

Update your project dependencies to use Stably’s Playwright SDK instead of the standard Playwright package:
npm remove @playwright/test
npm install --save-dev @stablyai/playwright-test
Or using other package managers:
npm remove @playwright/test
npm install --save-dev @stablyai/playwright-test
Commit the updated package.json and lock file to your repository.

2. Update Test Imports

Replace all imports from @playwright/test with @stablyai/playwright-test in your test files:
// Before
import { test, expect } from "@playwright/test";

// After
import { test, expect } from "@stablyai/playwright-test";

3. Configure GitHub Secrets

Add your Stably API key as a GitHub secret:
  1. Get your Team API Key from Stably Settings
  2. Go to your repository’s Settings > Secrets and variables > Actions
  3. Create a new secret named STABLY_API_KEY with your API key

4. Update Your GitHub Actions Workflow

Your workflow remains the same—just ensure the STABLY_API_KEY environment variable is set:
.github/workflows/playwright.yml
- name: Run Playwright tests
  run: npx playwright test
  env:
    STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
The Stably SDK is a drop-in replacement for Playwright, so you continue using npx playwright test instead of a different command. All your existing Playwright CLI flags and options work the same way.

Running Tests in Parallel

Stably works with GitHub Actions matrix strategy to run your tests in parallel across multiple containers, reducing overall execution time and providing faster feedback.

Example: Parallel Execution with Matrix

Here’s a complete workflow that runs tests across 4 parallel containers:
.github/workflows/playwright.yml
name: Playwright Tests
on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      
      - name: Install dependencies
        run: npm ci
      
      - name: Install Playwright browsers
        run: npx playwright install --with-deps
      
      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}

How Matrix Parallelization Works

By using GitHub Actions’ matrix execution strategy:
  • Multiple containers are created automatically (4 in the example above)
  • Each container receives a unique shard of tests to execute
  • Tests run simultaneously across all containers, reducing total execution time
  • Results are collected and aggregated in the Stably dashboard
You can adjust the number of parallel jobs by modifying the shardIndex and shardTotal arrays. For example, to run 8 parallel jobs:
strategy:
  matrix:
    shardIndex: [1, 2, 3, 4, 5, 6, 7, 8]
    shardTotal: [8]

Add Stably Reporter (Optional)

To integrate with Stably’s cloud platform and view test results in the dashboard, add the Stably reporter:

1. Install the Reporter

npm install --save-dev @stablyai/playwright-reporter

2. Configure in playwright.config.ts

playwright.config.ts
export default defineConfig({
  reporter: [
    ["list"],
    ["@stablyai/playwright-reporter"],
  ],
});

3. Add Environment Variables

In your GitHub Actions workflow, add the STABLY_PROJECT_ID:
- name: Run Playwright tests
  run: npx playwright test
  env:
    STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
    STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
Find your Project ID in your project settings on the Stably Dashboard and add it to your GitHub repository secrets.
Once configured, view comprehensive results in the Stably dashboard:
  • Test execution timeline showing parallel run progress
  • Screenshots and videos for failed tests
  • AI-generated failure analysis and suggested fixes
  • Historical trends and flakiness detection
  • Detailed logs from each parallel container
All test results from parallel containers are automatically aggregated into a single unified report, making it easy to review the complete test run regardless of how many shards executed.

Complete Workflow Examples

  • Basic Setup
  • Parallel with Artifacts
  • Multiple Browsers
  • With Stably Reporter
Minimal configuration for single-threaded execution:
.github/workflows/playwright.yml
name: Playwright Tests
on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}

Next Steps

I