Skip to main content
Stably integrates seamlessly with your existing GitLab CI/CD pipelines. Use Stably’s Playwright SDK to unlock AI-powered testing capabilities including auto-heal, AI assertions, and centralized reporting—all while maintaining 100% compatibility with your existing Playwright tests.

​
Prerequisites

  • A GitLab repository with an existing .gitlab-ci.yml file (or the ability to create one)
  • Node.js 18+ configured in your pipeline
  • Playwright tests in your repository

​
Quick Setup

​
1. Install Stably Playwright SDK

Update your project dependencies to use Stably’s Playwright SDK alongside Playwright: 
npm install --save-dev @playwright/test @stablyai/playwright-test
Commit the updated package.json and lock file to your repository.

​
2. Update Test Imports

Replace all imports from @playwright/test with @stablyai/playwright-test in your test files: 
// Before
import { test, expect } from "@playwright/test";

// After
import { test, expect } from "@stablyai/playwright-test";

​
3. Configure GitLab CI/CD Variables

Add your Stably API key as a GitLab CI/CD variable:
  1. Get your Team API Key from Stably Settings
  2. Go to your repository at Settings > CI/CD > Variables
  3. Add a variable named STABLY_API_KEY with your API key
  4. Mark it as “Masked” to protect the secret

​
4. Update Your GitLab CI/CD Configuration

Your pipeline remains the same—just ensure the STABLY_API_KEY environment variable is set:
.gitlab-ci.yml
test:
  script:
    - npm ci
    - npx playwright test
  variables:
    STABLY_API_KEY: $STABLY_API_KEY
The Stably SDK is a drop-in extension for Playwright, so you continue using npx playwright test instead of a different command. All your existing Playwright CLI flags and options work the same way.

​
Running Tests in Parallel

Stably works with GitLab’s parallel keyword to run your tests across multiple parallel jobs, reducing overall execution time and providing faster feedback.

​
Example: Parallel Execution

Here’s a complete pipeline that runs tests across 4 parallel jobs:
.gitlab-ci.yml
stages:
  - test

test:
  stage: test
  image: mcr.microsoft.com/playwright:v1.48.0-jammy
  parallel: 4
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  variables:
    STABLY_API_KEY: $STABLY_API_KEY
  timeout: 1h

​
How Parallel Execution Works

By using GitLab’s parallel execution:
  • Multiple jobs are created automatically (4 in the example above)
  • Each job receives a unique shard of tests to execute using $CI_NODE_INDEX and $CI_NODE_TOTAL
  • Tests run simultaneously across all jobs, reducing total execution time
  • Results are collected and aggregated in the Stably dashboard
You can adjust the number of parallel jobs by changing the parallel value. For example, to run 8 parallel jobs: 
test:
  parallel: 8
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

​
Add Stably Reporter (Optional, but highly reccomended)

To integrate with Stably’s cloud platform and view test results in the dashboard, configure the reporter in your playwright.config.ts:
playwright.config.ts
import { defineConfig } from "@playwright/test";
import { stablyReporter } from "@stablyai/playwright-test";

export default defineConfig({
  reporter: [
    ["list"],
    stablyReporter({ apiKey: process.env.STABLY_API_KEY, projectId: "your_project_id_here" }),
  ],
});
In your GitLab CI/CD configuration, add the environment variables: 
test:
  script:
    - npm ci
    - npx playwright test
  variables:
    STABLY_API_KEY: $STABLY_API_KEY
    STABLY_PROJECT_ID: $STABLY_PROJECT_ID
  • Find your Project ID in your project settings on the Stably Dashboard and add it to your GitLab CI/CD variables at Settings > CI/CD > Variables.
Once configured, view comprehensive results in the Stably dashboard:
  • Test execution timeline showing parallel run progress
  • Screenshots and videos for failed tests
  • AI-generated failure analysis and suggested fixes
  • Historical trends and flakiness detection
  • Detailed logs from each parallel job
All test results from parallel jobs are automatically aggregated into a single unified report, making it easy to review the complete test run regardless of how many shards executed.

​
Complete Pipeline Examples

  • Basic Setup
  • Parallel with Artifacts
  • Multiple Browsers
  • Matrix Jobs
  • With Stably Reporter
Minimal configuration for single-threaded execution:
.gitlab-ci.yml
stages:
  - test

test:
  stage: test
  image: mcr.microsoft.com/playwright:v1.48.0-jammy
  script:
    - npm ci
    - npx playwright test
  variables:
    STABLY_API_KEY: $STABLY_API_KEY

​
Pipeline-Specific Configuration

​
Cache Dependencies

Speed up your pipeline by caching node_modules: 
test:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - node_modules/
      - ~/.cache/ms-playwright/
  script:
    - npm ci
    - npx playwright test

​
Run Only on Specific Branches

Limit test execution to specific branches or merge requests: 
test:
  script:
    - npx playwright test
  only:
    - main
    - merge_requests
  variables:
    STABLY_API_KEY: $STABLY_API_KEY

​
Custom Docker Image

Use a custom Docker image with specific Playwright version: 
test:
  image: mcr.microsoft.com/playwright:v1.48.0-jammy
  parallel: 4
  before_script:
    - node --version
    - npm --version
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  variables:
    STABLY_API_KEY: $STABLY_API_KEY

​
Troubleshooting

​
Missing Playwright Browsers

If you see errors about missing browsers, ensure you’re using the Playwright Docker image or install browsers explicitly: 
script:
  - npm ci
  - npx playwright install --with-deps
  - npx playwright test

​
Timeout Issues

For long-running test suites, increase the job timeout: 
test:
  timeout: 2h
  script:
    - npx playwright test

​
Permission Errors

If you encounter permission errors with the cache or artifacts, add: 
test:
  before_script:
    - chmod -R 777 node_modules/

​
Next Steps