Using Stably CLI in CI/CD - Automated Testing

The Stably CLI can run in any CI/CD environment, not just GitHub Actions. This guide covers how to authenticate and run tests using environment variables, making it easy to integrate with GitHub Actions, GitLab CI, CircleCI, Jenkins, Bitbucket Pipelines, and other CI systems.

Authentication for CI

In CI environments, authentication is handled through environment variables instead of interactive browser login. You must set two environment variables:

STABLY_API_KEY

string

required

Your Stably API key. Get it from the API Keys settings page.

STABLY_PROJECT_ID

string

required

Your Stably project ID. Find it in your Stably dashboard project settings or URL.

Never hardcode your API key in configuration files or commit it to version control. Always use your CI provider’s secret management features.

Setup Instructions

Get Your API Key

Go to the API Keys settings page and create or copy your API key.

Find Your Project ID

Navigate to your project in the Stably dashboard. The project ID is visible in the project settings or in the URL.

Configure CI Secrets

Add STABLY_API_KEY and STABLY_PROJECT_ID as secrets in your CI provider’s settings.

Run CLI Commands

Use the Stably CLI in your CI pipeline. The CLI automatically uses the environment variables for authentication.

Basic Usage

Once the environment variables are set, you can run any Stably CLI command without interactive authentication. Stably is fully Playwright compatible — all Playwright CLI options are supported:

# Run all tests
npx stably test

# Run with Playwright options
npx stably test --headed --project=chromium

# Run specific test file
npx stably test tests/login.spec.ts

# More Playwright options
npx stably test --workers=4 --retries=2 --grep="login"

Loading Test Variables in CI

Use the --env flag to load variables from a named Environment stored in Stably. This avoids managing .env files or duplicating secrets across CI providers.

Instead of adding each secret to your CI provider individually, upload your .env file to a Stably Environment via Bulk Import. The only CI secret you need to configure is STABLY_API_KEY — when credentials rotate, update them once in Stably and every pipeline picks up the change automatically. See Bulk Import for details.

--env is a global option — it must go before the subcommand, with a space (not =):

# ✅ Correct
npx stably --env Production test

# ❌ Wrong — causes "unknown option" error
npx stably test --env=Production

You can also load from a local .env file committed to your repo (for non-sensitive values):

npx stably test --env-file .env.ci

For all available CLI options and configuration, see the Stably CLI documentation.

CI Platform Examples

GitHub Actions

.github/workflows/stably.yml

name: Stably Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install browsers
        run: npx stably install --with-deps

      - name: Run Stably tests
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
        run: npx stably test

If you’re using GitHub Actions and want a simpler setup with cloud execution, consider using the Stably GitHub Action instead.

GitLab CI

.gitlab-ci.yml

stages:
  - test

stably_tests:
  stage: test
  image: node:20
  
  variables:
    STABLY_API_KEY: $STABLY_API_KEY
    STABLY_PROJECT_ID: $STABLY_PROJECT_ID
  
  before_script:
    - npm ci
    - npx stably install
  
  script:
    - npx stably test

In GitLab, add STABLY_API_KEY and STABLY_PROJECT_ID as CI/CD variables in Settings > CI/CD > Variables. Mark them as Protected and Masked for security.

CircleCI

.circleci/config.yml

version: 2.1

orbs:
  node: circleci/node@5

jobs:
  test:
    docker:
      - image: cimg/node:20
    steps:
      - checkout
      - node/install-packages:
          pkg-manager: npm
      - run:
          name: Install browsers
          command: npx stably install
      - run:
          name: Run Stably tests
          command: npx stably test

workflows:
  test-workflow:
    jobs:
      - test

Add STABLY_API_KEY and STABLY_PROJECT_ID in CircleCI under Project Settings > Environment Variables. Project-level environment variables are automatically available to all jobs — no need to redeclare them in the config file.

Jenkins

Jenkinsfile

pipeline {
    agent {
        docker {
            image 'node:20'
        }
    }
    
    environment {
        STABLY_API_KEY = credentials('stably-api-key')
        STABLY_PROJECT_ID = credentials('stably-project-id')
    }
    
    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
                sh 'npx stably install'
            }
        }
        
        stage('Test') {
            steps {
                sh 'npx stably test'
            }
        }
    }
}

Store credentials in Jenkins using Manage Jenkins > Credentials.

Bitbucket Pipelines

bitbucket-pipelines.yml

image: node:20

pipelines:
  default:
    - step:
        name: Run Stably Tests
        caches:
          - node
        script:
          - npm ci
          - npx stably install
          - npx stably test

Add STABLY_API_KEY and STABLY_PROJECT_ID as Repository variables in Repository settings > Pipelines > Repository variables.

Azure DevOps

azure-pipelines.yml

trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'

  - script: npm ci
    displayName: 'Install dependencies'

  - script: npx stably install
    displayName: 'Install browsers'

  - script: npx stably test
    displayName: 'Run Stably tests'
    env:
      STABLY_API_KEY: $(STABLY_API_KEY)
      STABLY_PROJECT_ID: $(STABLY_PROJECT_ID)

Add the variables in Azure DevOps under Pipelines > Library or as pipeline variables.

Troubleshooting

Authentication Failed

Verify that both STABLY_API_KEY and STABLY_PROJECT_ID are correctly set:

# Check if variables are set (don't print the actual values)
echo "API Key set: ${STABLY_API_KEY:+yes}"
echo "Project ID set: ${STABLY_PROJECT_ID:+yes}"

Ensure the API key is valid and hasn’t expired.

Browser Not Found

Make sure browsers are installed in your CI environment:

npx stably install

Tests Timeout in CI

CI environments may be slower than local machines. Increase timeouts:

npx stably test --timeout=60000

Or configure timeouts in playwright.config.ts.

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Configuration

Troubleshooting

Migration Guides

Resources

Using CLI in CI

Authentication for CI

Setup Instructions

Basic Usage

Loading Test Variables in CI

CI Platform Examples

GitHub Actions

GitLab CI

CircleCI

Jenkins

Bitbucket Pipelines

Azure DevOps

Troubleshooting

​Authentication for CI

​Setup Instructions

​Basic Usage

​Loading Test Variables in CI

​CI Platform Examples

​GitHub Actions

​GitLab CI

​CircleCI

​Jenkins

​Bitbucket Pipelines

​Azure DevOps

​Troubleshooting

Authentication for CI

Setup Instructions

Basic Usage

Loading Test Variables in CI

CI Platform Examples

GitHub Actions

GitLab CI

CircleCI

Jenkins

Bitbucket Pipelines

Azure DevOps

Troubleshooting