Run Tests on Stably Cloud - Cloud Execution

Overview

Stably Cloud provides scalable infrastructure for running your Playwright tests with high parallelism, AI-powered features, and comprehensive reporting. Your tests run in the cloud while using your existing playwright.config.ts configuration.

Prerequisites

Publish Your Tests

Before running tests in the cloud, ensure your latest tests are published from the UI.

Get API Key

Obtain your API key from the settings page for API and CI integrations.

Running Tests in the Cloud

You can trigger cloud execution using several methods:

Web Editor
Stably CLI
REST API
GitHub Action
Scheduled Runs

Using the Stably Web Editor

The easiest way to run tests in the cloud is directly from the Stably dashboard:

Navigate to your project in the Stably dashboard
Select the tests you want to run
Click the Run button
Choose your run configuration (projects, grep patterns, environment)
Monitor results in real-time from the dashboard

Web Editor Guide

Learn more about the Stably Web Editor

Using the Command Line

Run tests from your terminal:

# Run all tests
npx stably cloud test

# Run specific projects
npx stably cloud test --project=smoke --project=critical

# Combine with grep patterns
npx stably cloud test --project=release --grep @p0

# Run with environment variables
npx stably cloud test --env=STAGING

CLI Options

--project: Specify which Playwright projects to run (can be used multiple times)
--grep: Filter tests by pattern
--env: Set environment for variable overrides
--workers: Control parallelism (cloud supports up to 100 workers)
--retries: Set retry attempts for failed tests

Using the API

Trigger test runs programmatically:

curl -X 'POST' \
  'https://api.stably.ai/v1/projects/{projectId}/runs' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "projects": ["smoke", "critical"],
    "grep": "@p0"
  }'

API Reference

View complete API documentation with all available endpoints and parameters

Using GitHub Actions

Integrate cloud testing into your GitHub workflows:

.github/workflows/stably.yaml

name: Run Stably Tests

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Run Cloud Tests
        uses: stablyai/stably-runner-action@v4
        with:
          api-key: ${{ secrets.STABLY_API_KEY }}
          project-id: your-project-id
          projects: "smoke,critical"

GitHub Integration Guide

Learn more about GitHub Actions integration

Using Schedules

Configure automatic test runs using stably.yaml:

stably.yaml

schedules:
  nightly-smoke:
    cron: "0 2 * * *"
    stablyArgs: "--project smoke --project mobile-smoke"
  
  hourly-critical:
    cron: "0 * * * *"
    stablyArgs: "--project critical --grep @p0"

Scheduling Guide

Learn how to configure scheduled test runs

Cloud Execution Features

High Parallelism

Run up to 100 workers in parallel for faster test execution. Contact us if you need more.

AI Auto-Heal

Tests automatically adapt to minor UI changes, reducing maintenance burden.

Smart Reporting

Get detailed insights with screenshots, traces, and AI-powered failure analysis.

Environment Management

Easily switch between environments and manage test data variables.

Monitoring Test Runs

After triggering a test run, you can monitor progress:

View Real-Time Progress

Navigate to your Stably dashboard to see:

Real-time test execution status
Pass/fail rates
Test duration and performance metrics

Review Results

Once tests complete, access:

Detailed test reports with screenshots
Trace files for debugging failures
AI-generated failure analysis
Historical trends and analytics

Receive Notifications

Get alerts via:

Email notifications
Slack messages
GitHub PR comments

Configure notifications in your project settings.

Best Practices

Use Project Organization

Organize tests into logical projects (smoke, regression, critical) for flexible execution strategies.

Leverage High Parallelism

Take advantage of cloud workers by ensuring your tests are independent and can run in parallel.

Configure Retries

Set appropriate retry counts for different test types. Critical tests might need fewer retries, while flaky tests might need more.

Monitor Resource Usage

Track your test execution time and optimize slow tests to reduce costs and improve feedback time.

Troubleshooting

API Authentication Errors

Problem: Getting 401 or 403 errors when using the APISolution: Verify your API key is correct and has proper permissions. Get a new key from the settings page.

Project Not Found

Problem: Error saying specified project doesn’t existSolution: Ensure the project name matches exactly what’s defined in your playwright.config.ts

High Execution Time

Problem: Tests are taking longer than expectedSolution:

Increase worker count for better parallelism
Check if tests have dependencies preventing parallel execution
Review individual test timeouts

Next Steps

Define Test Groups

Learn how to organize tests using Playwright projects

Schedule Tests

Configure automatic test runs with cron schedules

CI/CD Integration

Integrate cloud testing into your CI/CD pipeline

Configure Notifications

Set up alerts for test results

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Stably Classic

Troubleshooting

Run Tests

Overview

Prerequisites

Running Tests in the Cloud

Using the Stably Web Editor

Web Editor Guide

Using the Command Line

Using the API

API Reference

Using GitHub Actions

GitHub Integration Guide

Using Schedules

Scheduling Guide

Cloud Execution Features

High Parallelism

AI Auto-Heal

Smart Reporting

Environment Management

Monitoring Test Runs

Best Practices

Troubleshooting

Next Steps

Define Test Groups

Schedule Tests

CI/CD Integration

Configure Notifications

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Stably Classic

Troubleshooting

​Overview

​Prerequisites

​Running Tests in the Cloud

​Using the Stably Web Editor

Web Editor Guide

​Using the Command Line

​Using the API

API Reference

​Using GitHub Actions

GitHub Integration Guide

​Using Schedules

Scheduling Guide

​Cloud Execution Features

High Parallelism

AI Auto-Heal

Smart Reporting

Environment Management

​Monitoring Test Runs

​Best Practices

​Troubleshooting

​Next Steps

Define Test Groups

Schedule Tests

CI/CD Integration

Configure Notifications

Overview

Prerequisites

Running Tests in the Cloud

Using the Stably Web Editor

Using the Command Line

Using the API

Using GitHub Actions

Using Schedules

Cloud Execution Features

Monitoring Test Runs

Best Practices

Troubleshooting

Next Steps