Scheduled Test Runs

Overview

Schedule automatic test runs using cron expressions to continuously monitor your application. Stably runs your configured Playwright projects on the schedule you define.

Defining Schedules

Create a stably.yaml file in the root of your repository to configure scheduled test runs.

Basic Configuration

stably.yaml

schedules:
  nightly-smoke:
    cron: "0 2 * * *"
    stablyArgs: "--project smoke --project mobile-smoke"
  
  hourly-critical:
    cron: "0 * * * *"
    stablyArgs: "--project critical --grep @p0"

Each schedule consists of:

Schedule name: A descriptive identifier (e.g., nightly-smoke)
cron: A cron expression defining when to run
stablyArgs: Arguments passed to the Stably CLI (same as npx stably test)

Cron Expression Format

Cron expressions use the standard UNIX format:

┌───────────── minute (0 - 59)
│ ┌───────────── hour (0 - 23)
│ │ ┌───────────── day of month (1 - 31)
│ │ │ ┌───────────── month (1 - 12)
│ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday)
│ │ │ │ │
│ │ │ │ │
* * * * *

Common Cron Patterns

Daily Schedules

# Every day at 2:00 AM UTC
cron: "0 2 * * *"

# Every day at 9:00 AM and 5:00 PM UTC
cron: "0 9,17 * * *"

# Every day at midnight UTC
cron: "0 0 * * *"

Hourly Schedules

# Every hour on the hour
cron: "0 * * * *"

# Every 6 hours (00:00, 06:00, 12:00, 18:00)
cron: "0 */6 * * *"

# Every 15 minutes
cron: "*/15 * * * *"

Weekly Schedules

# Every Monday at 9:00 AM UTC
cron: "0 9 * * 1"

# Every weekday (Mon-Fri) at 8:00 AM UTC
cron: "0 8 * * 1-5"

# Every Sunday at midnight UTC
cron: "0 0 * * 0"

Monthly Schedules

# First day of every month at 2:00 AM UTC
cron: "0 2 1 * *"

# Last day of every month (use 28-31)
cron: "0 2 28-31 * *"

# 15th of every month at noon UTC
cron: "0 12 15 * *"

Use crontab.guru to easily validate and understand your cron expressions.

Cron Expression Limitations

Stably crons do not support alternative expressions like MON, SUN, JAN, or DEC
You cannot configure both day of month and day of week simultaneously
The timezone is always UTC

Schedule Arguments

The stablyArgs field accepts the same arguments as the npx stably test command:

Project Selection

Run specific Playwright projects:

stablyArgs: "--project smoke"

# Multiple projects
stablyArgs: "--project smoke --project critical"

Test Filtering with Grep

Filter tests using patterns:

# Run only P0 tests
stablyArgs: "--grep @p0"

# Combine project and grep
stablyArgs: "--project critical --grep @p0"

Worker Configuration

Control parallelism:

# Use 50 parallel workers
stablyArgs: "--project regression --workers 50"

Environment Variables

Set environment for test data:

stablyArgs: "--project smoke --env PRODUCTION"

Complete Examples

Basic Monitoring
Weekday Testing
Multi-Environment
Priority-Based

stably.yaml

schedules:
  # Run smoke tests every hour
  hourly-health-check:
    cron: "0 * * * *"
    stablyArgs: "--project smoke"
  
  # Full regression nightly
  nightly-regression:
    cron: "0 2 * * *"
    stablyArgs: "--project regression --project e2e"

How Schedules Work

Configuration

Define schedules in stably.yaml in your repository root

Deployment

Deploy your tests and configuration:

npx stably deploy

Automatic Execution

Stably runs your tests at the scheduled times in UTC timezone

Results & Notifications

View results in your dashboard and receive notifications based on your notification settings

Best Practices

Match Testing Frequency to Risk

Run critical tests more frequently (hourly) and comprehensive suites less often (daily/weekly)

Consider Time Zones

All schedules run in UTC. Plan around your team’s working hours and off-peak times

Use Descriptive Names

Name schedules clearly to indicate what and when (e.g., weekday-morning-smoke)

Balance Coverage and Cost

More frequent runs provide faster feedback but consume more resources

Optimize for Fast Feedback

Run smoke tests frequently (every 15-60 minutes)
Run comprehensive regression suites less often (daily/weekly)
Use --grep to run only priority tests more frequently

Leverage Off-Peak Hours

Schedule resource-intensive tests during off-peak hours
Run nightly regressions when usage is low
Consider weekend runs for comprehensive testing

Use Multiple Schedules

Create different schedules for different test types
Separate production monitoring from staging validation
Balance frequency with test importance

Monitor Schedule Performance

Review execution times and adjust schedules accordingly
Ensure tests complete before the next scheduled run
Optimize slow tests that delay feedback

Viewing Scheduled Runs

Access Dashboard

Navigate to your project in the Stably dashboard

View Scheduled Runs

See all scheduled test runs with:

Schedule name and configuration
Last execution time and results
Next scheduled execution time
Execution history and trends

Review Results

Click on any run to see:

Detailed test results with pass/fail status
Screenshots and traces for failures
AI-powered failure analysis
Historical trends and flakiness detection

Troubleshooting

Schedules Not Running

Problem: Scheduled tests aren’t executingSolution:

Verify stably.yaml is in repository root
Ensure you’ve run npx stably deploy after creating/updating schedules
Check cron expression syntax using crontab.guru
Verify project names match your playwright.config.ts

Incorrect Run Times

Problem: Tests run at unexpected timesSolution:

Remember all schedules run in UTC timezone
Convert your local time to UTC for the cron expression
Use a timezone converter to verify the schedule

Tests Taking Too Long

Problem: Scheduled runs don’t complete before the next run startsSolution:

Increase worker count in stablyArgs (e.g., --workers 50)
Reduce schedule frequency to allow tests to complete
Optimize slow tests or remove non-critical tests
Split into multiple schedules with different test sets

Invalid Cron Expression

Problem: Schedule not accepted or parsing errorsSolution:

Use standard 5-field cron format only
Don’t use named days/months (MON, JAN, etc.)
Don’t combine day-of-month and day-of-week
Validate using crontab.guru

Complete Configuration Reference

Schedule Entry Structure

schedules:
  <schedule-name>:
    cron: "<cron-expression>"
    stablyArgs: "<cli-arguments>"

Field Definitions

Field	Type	Required	Description
`schedules`	object	Yes	Root object containing all schedule definitions
`<schedule-name>`	string	Yes	Unique identifier for the schedule
`cron`	string	Yes	Standard UNIX cron expression (5 fields)
`stablyArgs`	string	No	Arguments passed to `npx stably test`

Valid stablyArgs

Argument	Example	Description
`--project`	`--project smoke`	Run specific Playwright project
`--grep`	`--grep @p0`	Filter tests by pattern
`--workers`	`--workers 50`	Set parallel worker count
`--env`	`--env STAGING`	Set environment for variables
`--retries`	`--retries 2`	Configure test retries

Next Steps

Define Test Groups

Organize tests using Playwright projects

Configure Notifications

Set up alerts for scheduled test results

Run on Cloud

Learn about cloud test execution

Testing Guides

Best practices for scheduled monitoring

Getting started

Stably 2.0

Stably Classic

Troubleshooting

Overview

Defining Schedules

Basic Configuration

Cron Expression Format

Common Cron Patterns

Cron Expression Limitations

Schedule Arguments

Complete Examples

How Schedules Work

Best Practices

Match Testing Frequency to Risk

Consider Time Zones

Use Descriptive Names

Balance Coverage and Cost

Viewing Scheduled Runs

Troubleshooting

Complete Configuration Reference

Schedule Entry Structure

Field Definitions

Valid stablyArgs

Next Steps

Define Test Groups

Configure Notifications

Run on Cloud

Testing Guides

Getting started

Stably 2.0

Stably Classic

Troubleshooting

​Overview

​Defining Schedules

​Basic Configuration

​Cron Expression Format

​Common Cron Patterns

​Cron Expression Limitations

​Schedule Arguments

​Complete Examples

​How Schedules Work

​Best Practices

Match Testing Frequency to Risk

Consider Time Zones

Use Descriptive Names

Balance Coverage and Cost

​Viewing Scheduled Runs

​Troubleshooting

​Complete Configuration Reference

​Schedule Entry Structure

​Field Definitions

​Valid stablyArgs

​Next Steps

Define Test Groups

Configure Notifications

Run on Cloud

Testing Guides

Overview

Defining Schedules

Basic Configuration

Cron Expression Format

Common Cron Patterns

Cron Expression Limitations

Schedule Arguments

Complete Examples

How Schedules Work

Best Practices

Viewing Scheduled Runs

Troubleshooting

Complete Configuration Reference

Schedule Entry Structure

Field Definitions

Valid stablyArgs

Next Steps