CI Integration

Stably integrates seamlessly into your CI/CD pipeline. Run tests automatically on every push or pull request, and use stably fix to automatically resolve failing tests with AI assistance.

Choose Your Approach

Your Own Runner

Run tests on your CI infrastructure (GitHub Actions, GitLab CI, etc.). You control the environment and pay for compute.

Stably Cloud Runner

Run tests on Stably’s scalable infrastructure. Run hundreds of tests in parallel without managing infrastructure.

Feature	Your Own Runner	Stably Cloud
Parallelization	Limited by CI plan	Up to 100+ workers
Setup	Install browsers, manage deps	Just add API key
Cost	Pay your CI provider	Pay Stably (usage-based)
Best for	Small suites, custom environments	Large suites, fast feedback

Option 1: Your Own CI Runner

Run Stably CLI commands directly in your CI environment. This approach gives you full control over the test execution environment.

Setup Requirements

Before configuring your CI workflow, you’ll need credentials:

Get Your API Key

Go to the API Keys settings page and create or copy your API key.

Find Your Project ID

Navigate to your project in the Stably dashboard. The project ID is visible in the project settings or URL.

Add Secrets to CI

Add STABLY_API_KEY and STABLY_PROJECT_ID as secrets in your CI provider.GitHub: Settings → Secrets and variables → Actions → New repository secret

Never hardcode your API key in configuration files or commit it to version control. Always use your CI provider’s secret management.

GitHub Actions Example

The recommended pattern runs tests in a parallel matrix and uses a separate fix job that triggers only when tests fail:

.github/workflows/stably.yaml

name: Run Stably Tests

on:
  push:
    branches:
      - main
  schedule:
    - cron: '0 */3 * * *'
  workflow_dispatch:

env:
  CI_BUILD_ID: stably-${{ github.run_id }}-${{ github.run_attempt }}

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 120
    strategy:
      fail-fast: false
      matrix:
        project:
          - 'e2e:smoke'
          - 'e2e:checkout'
          - 'e2e:login'
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install chromium --with-deps

      - name: Run Stably tests
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
        run: npx stably test --project=${{ matrix.project }}

  fix:
    needs: test
    if: always() && needs.test.result == 'failure'
    runs-on: ubuntu-latest
    timeout-minutes: 120
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install chromium --with-deps

      - name: Run Stably Fix
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
        run: npx stably fix

Key points:

CI_BUILD_ID ties all parallel test shards to the same run, so stably fix can find all failures.
fail-fast: false ensures all matrix jobs complete, giving the fix agent the full picture.
The fix job uses if: always() && needs.test.result == 'failure' to run only when at least one test job failed.

Other CI Platforms

The same commands work in any CI environment. See platform-specific guides:

CLI in CI guide - GitLab CI, CircleCI, Jenkins, Bitbucket Pipelines, Azure DevOps
GitLab CI guide

CLI Commands Reference

Command	Description
`npx stably install`	Installs browsers needed to run tests
`npx stably test`	Runs your Playwright tests and reports results to Stably
`npx stably fix`	Uses AI to analyze and fix failing tests

For full CLI documentation, see the CLI Reference.

How stably fix Works in CI

When you run stably fix in CI, it automatically detects which test run to fix using the CI_BUILD_ID environment variable.Behind the scenes:

The CI_BUILD_ID env var (e.g., stably-${{ github.run_id }}-${{ github.run_attempt }}) ties all parallel test shards to a single logical run
stably fix uses this ID to find all failures across matrix jobs
The AI analyzes the failures and applies targeted fixes

Why a separate fix job? When running tests in a parallel matrix (recommended for large suites), stably fix must run after all test jobs complete so it can see the full set of failures. The fix job depends on the test job and only triggers when at least one shard failed.

Option 2: Stably Cloud Runner

For teams that need to run large test suites quickly, Stably offers a scalable cloud infrastructure that can run hundreds of tests in parallel.

Why Use Stably Cloud?

Massive Parallelization

Run up to 100+ tests simultaneously. A 2-hour test suite can complete in minutes.

No Infrastructure Management

No need to provision runners, manage browser installations, or scale capacity.

Consistent Environment

Tests run in a clean, consistent environment every time—no flaky failures from CI resource contention.

Built-in Retries

Automatic retries and smart failure analysis included.

GitHub Action (Recommended)

Use the Stably GitHub Action to run tests on Stably Cloud:

.github/workflows/stably-cloud.yaml

name: Stably Cloud Tests

on:
  pull_request:
  push:
    branches:
      - main

permissions:
  pull-requests: write
  contents: write

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Run Tests on Stably Cloud
        uses: stablyai/stably-runner-action@v4
        with:
          api-key: ${{ secrets.STABLY_API_KEY }}
          project-id: ${{ secrets.STABLY_PROJECT_ID }}
          playwright-project-name: smoke

For full configuration options, see the GitHub Actions guide.

API (Non-GitHub Platforms)

If you’re using GitLab, CircleCI, Jenkins, or another CI platform, use the Stably API to trigger cloud runs:

# Trigger a cloud run
RUN_ID=$(curl -s -X POST "https://api.stably.ai/v1/projects/${STABLY_PROJECT_ID}/runs" \
  -H "Authorization: Bearer ${STABLY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "playwrightProjectName": ["smoke", "regression"],
    "branch": "'"${GITHUB_HEAD_REF:-main}"'"
  }' | jq -r '.runId')

echo "Started run: $RUN_ID"

# Poll until the run finishes
while true; do
  STATUS=$(curl -s "https://api.stably.ai/v1/projects/${STABLY_PROJECT_ID}/runs/${RUN_ID}" \
    -H "Authorization: Bearer ${STABLY_API_KEY}" | jq -r '.status')
  echo "Status: $STATUS"
  case "$STATUS" in
    PASSED|FAILED|TIMEDOUT|CANCELLED|INTERRUPTED) break ;;
  esac
  sleep 10
done

# Fail the CI step if the run didn't pass
[ "$STATUS" = "PASSED" ] || exit 1

See the API Reference for full documentation.

Start with your own runner to get familiar with Stably, then consider Stably Cloud when your test suite grows and execution time becomes a bottleneck.

Troubleshooting

stably fix fails with missing dependencies

If stably fix fails due to missing dependencies or configuration issues, you need to run stably init on your repository first. This command sets up the necessary Playwright and Stably SDK dependencies.Solution:

Run stably init locally in your repository:

npx stably init

The AI agent will install and configure the required dependencies
Commit the changes to your repository
Re-run your CI pipeline

stably init only needs to be run once per repository. After the initial setup, stably fix will work in CI without any additional configuration.

Next Steps

CLI Reference

Full CLI command reference

GitHub Actions

Detailed GitHub Actions setup

Test Groups

Organize tests into projects

Alerting

Get notified on test failures

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Configuration

Troubleshooting

Migration Guides

Resources

Choose Your Approach

Your Own Runner

Stably Cloud Runner

Option 1: Your Own CI Runner

Option 2: Stably Cloud Runner

Massive Parallelization

No Infrastructure Management

Consistent Environment

Built-in Retries

Troubleshooting

Next Steps

CLI Reference

GitHub Actions

Test Groups

Alerting

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Configuration

Troubleshooting

Migration Guides

Resources

​Choose Your Approach

Your Own Runner

Stably Cloud Runner

​Option 1: Your Own CI Runner

​Option 2: Stably Cloud Runner

Massive Parallelization

No Infrastructure Management

Consistent Environment

Built-in Retries

​Troubleshooting

​Next Steps

CLI Reference

GitHub Actions

Test Groups

Alerting

Choose Your Approach

Option 1: Your Own CI Runner

Option 2: Stably Cloud Runner

Troubleshooting

Next Steps