Skip to main content

​
Overview

Stably Cloud provides scalable infrastructure for running your Playwright tests with high parallelism, AI-powered features, and comprehensive reporting. Your tests run in the cloud while using your existing playwright.config.ts configuration.

​
Prerequisites

1

Deploy Your Tests

Before running tests in the cloud, ensure your latest test code is deployed:
npx stably deploy
If you’re using the web editor, publish your changes instead. The deployment happens automatically.
2

Get API Key

Obtain your API key from the settings page for API and CI integrations.

​
Running Tests in the Cloud

After deploying your tests, you can trigger cloud execution using several methods:
  • Stably CLI
  • REST API
  • GitHub Action
  • Scheduled Runs

​
Using the Command Line

The simplest way to run tests in the cloud:
# Run all tests
npx stably cloud test

# Run specific projects
npx stably cloud test --project=smoke --project=critical

# Combine with grep patterns
npx stably cloud test --project=release --grep @p0

# Run with environment variables
npx stably cloud test --env=STAGING
  • --project: Specify which Playwright projects to run (can be used multiple times)
  • --grep: Filter tests by pattern
  • --env: Set environment for variable overrides
  • --workers: Control parallelism (cloud supports up to 100 workers)
  • --retries: Set retry attempts for failed tests

​
Running Specific Test Groups

If you’ve organized your tests using Playwright projects, you can run specific groups in the cloud: 
# Run smoke tests only
npx stably cloud test --project=smoke

# Run multiple test groups
npx stably cloud test --project=smoke --project=release

# Run tests with specific tags
npx stably cloud test --project=critical --grep @p0

# Run all P0 tests across projects
npx stably cloud test --grep @p0

​
Cloud Execution Features

High Parallelism

Run up to 100 workers in parallel for faster test execution. Contact us if you need more.

AI Auto-Heal

Tests automatically adapt to minor UI changes, reducing maintenance burden.

Smart Reporting

Get detailed insights with screenshots, traces, and AI-powered failure analysis.

Environment Management

Easily switch between environments and manage test data variables.

​
Monitoring Test Runs

After triggering a test run, you can monitor progress:
1

View Real-Time Progress

Navigate to your Stably dashboard to see:
  • Real-time test execution status
  • Pass/fail rates
  • Test duration and performance metrics
2

Review Results

Once tests complete, access:
  • Detailed test reports with screenshots
  • Trace files for debugging failures
  • AI-generated failure analysis
  • Historical trends and analytics
3

Receive Notifications

Get alerts via:
  • Email notifications
  • Slack messages
  • GitHub PR comments
Configure notifications in your project settings.

​
Best Practices

Always run npx stably deploy after making test changes to ensure the cloud runs the latest version.
Organize tests into logical projects (smoke, regression, critical) for flexible execution strategies.
Take advantage of cloud workers by ensuring your tests are independent and can run in parallel.
Set appropriate retry counts for different test types. Critical tests might need fewer retries, while flaky tests might need more.
Track your test execution time and optimize slow tests to reduce costs and improve feedback time.

​
Troubleshooting

Problem: Cloud tests are running outdated test codeSolution: Run npx stably deploy to deploy the latest changes
Problem: Getting 401 or 403 errors when using the APISolution: Verify your API key is correct and has proper permissions. Get a new key from the settings page.
Problem: Error saying specified project doesn’t existSolution: Ensure the project name matches exactly what’s defined in your playwright.config.ts
Problem: Tests are taking longer than expectedSolution:
  • Increase worker count for better parallelism
  • Check if tests have dependencies preventing parallel execution
  • Review individual test timeouts

​
Next Steps

Define Test Groups

Learn how to organize tests using Playwright projects

Schedule Tests

Configure automatic test runs with cron schedules

CI/CD Integration

Integrate cloud testing into your CI/CD pipeline

Configure Notifications

Set up alerts for test results