Prerequisites
- A GitLab repository with an existing
.gitlab-ci.yml
file (or the ability to create one) - Node.js 18+ configured in your pipeline
- Playwright tests in your repository
Quick Setup
1. Install Stably Playwright SDK
Update your project dependencies to use Stably’s Playwright SDK instead of the standard Playwright package:package.json
and lock file to your repository.
2. Update Test Imports
Replace all imports from@playwright/test
with @stablyai/playwright-test
in your test files:
3. Configure GitLab CI/CD Variables
Add your Stably API key as a GitLab CI/CD variable:- Get your Team API Key from Stably Settings
- Go to your repository at
Settings > CI/CD > Variables
- Add a variable named
STABLY_API_KEY
with your API key - Mark it as “Masked” to protect the secret
4. Update Your GitLab CI/CD Configuration
Your pipeline remains the same—just ensure theSTABLY_API_KEY
environment variable is set:
.gitlab-ci.yml
The Stably SDK is a drop-in replacement for Playwright, so you continue using
npx playwright test
instead of a different command. All your existing Playwright CLI flags and options work the same way.Running Tests in Parallel
Stably works with GitLab’s parallel keyword to run your tests across multiple parallel jobs, reducing overall execution time and providing faster feedback.Example: Parallel Execution
Here’s a complete pipeline that runs tests across 4 parallel jobs:.gitlab-ci.yml
How Parallel Execution Works
By using GitLab’s parallel execution:- Multiple jobs are created automatically (4 in the example above)
- Each job receives a unique shard of tests to execute using
$CI_NODE_INDEX
and$CI_NODE_TOTAL
- Tests run simultaneously across all jobs, reducing total execution time
- Results are collected and aggregated in the Stably dashboard
parallel
value. For example, to run 8 parallel jobs:
Add Stably Reporter (Optional)
To integrate with Stably’s cloud platform and view test results in the dashboard, add the Stably reporter:1. Install the Reporter
2. Configure in playwright.config.ts
playwright.config.ts
3. Add Environment Variables
In your GitLab CI/CD configuration, add theSTABLY_PROJECT_ID
variable:
Find your Project ID in your project settings on the Stably Dashboard and add it to your GitLab CI/CD variables at
Settings > CI/CD > Variables
.- Test execution timeline showing parallel run progress
- Screenshots and videos for failed tests
- AI-generated failure analysis and suggested fixes
- Historical trends and flakiness detection
- Detailed logs from each parallel job
Complete Pipeline Examples
- Basic Setup
- Parallel with Artifacts
- Multiple Browsers
- Matrix Jobs
- With Stably Reporter
Minimal configuration for single-threaded execution:
.gitlab-ci.yml
Pipeline-Specific Configuration
Cache Dependencies
Speed up your pipeline by caching node_modules:Run Only on Specific Branches
Limit test execution to specific branches or merge requests:Custom Docker Image
Use a custom Docker image with specific Playwright version:Troubleshooting
Missing Playwright Browsers
If you see errors about missing browsers, ensure you’re using the Playwright Docker image or install browsers explicitly:Timeout Issues
For long-running test suites, increase the job timeout:Permission Errors
If you encounter permission errors with the cache or artifacts, add:Next Steps
- Learn about AI-powered auto-healing to reduce flaky test failures
- Explore AI assertions for intelligent test validation
- Set up Slack notifications for test results
- Configure GitLab integration for merge request comments and pipeline status