Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.stably.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

The Stably GitHub Action runs your Playwright tests on Stably’s cloud infrastructure directly from your GitHub Actions workflows. It supports two modes: Agents (v2) for project-based runs, and Classic (v1) for test-suite-based runs. The action auto-detects which version to use based on your inputs.

​
Quick Start

# .github/workflows/stably.yml
name: Stably Tests

on:
  push:
    branches: [main]
  pull_request:

permissions:
  pull-requests: write
  contents: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Run Stably Tests
        uses: stablyai/stably-runner-action@v4
        with:
          api-key: ${{ secrets.STABLY_API_KEY }}
          project-id: your-project-id

​
Inputs (v2 — Agents)

InputRequiredDefaultDescription
api-keyYes—Your Stably API key
project-idYes—Your Stably project ID
playwright-project-nameNo—Playwright project name(s) to run (maps to --project CLI flag). If omitted, all projects run.
environment-nameNo—Environment to use (e.g. “Staging”, “Production”). Falls back to the project’s default.
env-overridesNo—YAML string or JSON object of environment variable overrides
github-commentNotruePost test results as a PR/commit comment
github-tokenNo${{ github.token }}Token for posting comments. Override with a PAT if needed.
asyncNofalseLaunch tests without waiting for results. Comments won’t work in async mode.

​
Inputs (v1 — Classic)

InputRequiredDefaultDescription
api-keyYes—Your Stably API key
test-suite-idYes—The test suite ID to execute
environmentNoPRODUCTIONEnvironment to inherit variables from
variable-overridesNo{}JSON object of variable overrides
noteNo—Optional note to help identify the run
github-commentNotruePost test results as a PR/commit comment
github-tokenNo${{ github.token }}Token for posting comments
asyncNofalseLaunch tests without waiting for results

​
Outputs

OutputDescription
successBoolean indicating if the run was successful
runIdThe resulting run ID (v2 only)
testSuiteRunIdThe resulting test suite run ID (v1 only)

​
Examples

​
Run Specific Playwright Projects

- name: Run Smoke Tests
  uses: stablyai/stably-runner-action@v4
  with:
    api-key: ${{ secrets.STABLY_API_KEY }}
    project-id: your-project-id
    playwright-project-name: smoke-tests

​
Override Environment Variables

- name: Run Tests Against Staging
  uses: stablyai/stably-runner-action@v4
  with:
    api-key: ${{ secrets.STABLY_API_KEY }}
    project-id: your-project-id
    environment-name: Staging
    env-overrides: |
      BASE_URL: https://staging.example.com
      API_KEY: abc123

​
Use a Dynamic Preview URL

If your deployment URL is generated by a previous step (e.g. Vercel, Netlify): 
steps:
  - name: Deploy Preview
    id: deploy
    run: |
      PREVIEW_URL="https://my-app-${{ github.sha }}.vercel.app"
      echo "preview_url=$PREVIEW_URL" >> $GITHUB_OUTPUT

  - name: Run Stably Tests
    uses: stablyai/stably-runner-action@v4
    with:
      api-key: ${{ secrets.STABLY_API_KEY }}
      project-id: your-project-id
      env-overrides: |
        BASE_URL: ${{ steps.deploy.outputs.preview_url }}

​
Classic (v1) Example

- name: Run Tests (v1)
  uses: stablyai/stably-runner-action@v4
  with:
    api-key: ${{ secrets.STABLY_API_KEY }}
    test-suite-id: YOUR_TEST_SUITE_ID
    variable-overrides: |
      {
        "APP_URL": "https://example.com"
      }

​
Check Outputs

- name: Run Tests
  id: stably
  uses: stablyai/stably-runner-action@v4
  with:
    api-key: ${{ secrets.STABLY_API_KEY }}
    project-id: your-project-id

- name: Check Results
  run: echo "Success: ${{ steps.stably.outputs.success }}"

​
Permissions

If using the github-comment feature (enabled by default), add these permissions at the top of your workflow: 
permissions:
  pull-requests: write
  contents: write
Alternatively, enable read/write permissions for all workflows at Settings > Actions > General > Workflow permissions.
For organizations, you must first allow these permissions at the organization level before they can be set per-repository.

​
Using the CLI Instead

You can also run Stably tests directly via the CLI in GitHub Actions: 
steps:
  - uses: actions/checkout@v4

  - uses: actions/setup-node@v4
    with:
      node-version: '20'

  - run: npm ci

  - name: Run Stably Tests
    env:
      STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
      STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
    run: npx stably test

​
Next Steps

GitLab CI

Integrate with GitLab CI/CD

Alerting

Configure notifications for test results