Scheduled Test Runs

Overview

Schedule automatic test runs using cron expressions to continuously monitor your application. Stably runs your configured Playwright projects on the schedule you define.

Defining Schedules

With AI
With Visual Editor
Manually

It couldn’t be simpler. Just use the AI chat to add the desired test schedule to your stably.yaml config.

Create a stably.yaml file in the root of your repository to configure scheduled test runs.

Schedule Configuration

Most users should be able to simply define schedules using AI or the UI editor

Basic Configuration

stably.yaml

schedules:
  nightly-smoke:
    cron: "0 2 * * *"
    stablyTestArgs: "--project smoke --project mobile-smoke"
    timezone: "America/New_York"

  hourly-critical:
    cron: "0 * * * *"
    stablyTestArgs: "--project critical --grep @p0"

Each schedule consists of:

Schedule name: A descriptive identifier (e.g., nightly-smoke)
cron: A cron expression defining when to run
stablyTestArgs: Arguments passed to the Stably CLI (same as npx stably test) (optional)
timezone: IANA timezone string (e.g., America/New_York) (optional, defaults to UTC)
autofix: Automatically fix failing tests after each run (optional, see Autofix)

Cron Expression Format

Cron expressions use the standard UNIX format:

┌───────────── minute (0 - 59)
│ ┌───────────── hour (0 - 23)
│ │ ┌───────────── day of month (1 - 31)
│ │ │ ┌───────────── month (1 - 12)
│ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday)
│ │ │ │ │
│ │ │ │ │
* * * * *

Common Cron Patterns

Daily Schedules

# Every day at 2:00 AM UTC
cron: "0 2 * * *"

# Every day at 9:00 AM and 5:00 PM UTC
cron: "0 9,17 * * *"

# Every day at midnight UTC
cron: "0 0 * * *"

Hourly Schedules

# Every hour on the hour
cron: "0 * * * *"

# Every 6 hours (00:00, 06:00, 12:00, 18:00)
cron: "0 */6 * * *"

# Every 15 minutes
cron: "*/15 * * * *"

Weekly Schedules

# Every Monday at 9:00 AM UTC
cron: "0 9 * * 1"

# Every weekday (Mon-Fri) at 8:00 AM UTC
cron: "0 8 * * 1-5"

# Every Sunday at midnight UTC
cron: "0 0 * * 0"

Monthly Schedules

# First day of every month at 2:00 AM UTC
cron: "0 2 1 * *"

# Last day of every month (use 28-31)
cron: "0 2 28-31 * *"

# 15th of every month at noon UTC
cron: "0 12 15 * *"

Use crontab.guru to easily validate and understand your cron expressions.

Cron Expression Limitations

Stably crons do not support alternative expressions like MON, SUN, JAN, or DEC
You cannot configure both day of month and day of week simultaneously
The default timezone is UTC. You can override it per schedule with the timezone field

Schedule Arguments

The stablyTestArgs field accepts the same arguments as the npx stably test command:

Project Selection

Run specific Playwright projects:

stablyTestArgs: "--project smoke"

# Multiple projects
stablyTestArgs: "--project smoke --project critical"

Test Filtering with Grep

Filter tests using patterns:

# Run only P0 tests
stablyTestArgs: "--grep @p0"

# Combine project and grep
stablyTestArgs: "--project critical --grep @p0"

Parallelism Configuration

Control parallelism:

# Use 50 parallel tests
stablyTestArgs: "--project regression --workers 50"

Environment Variables

Set environment for test data:

stablyTestArgs: "--project smoke --env PRODUCTION"

Complete Examples

Basic Monitoring
Weekday Testing
Multi-Environment
Priority-Based

stably.yaml

schedules:
  # Run smoke tests every hour
  hourly-health-check:
    cron: "0 * * * *"
    stablyTestArgs: "--project smoke"

  # Full regression nightly with autofix
  nightly-regression:
    cron: "0 2 * * *"
    stablyTestArgs: "--project regression --project e2e"
    autofix: true

stably.yaml

schedules:
  # Critical tests every weekday morning
  weekday-morning:
    cron: "0 8 * * 1-5"
    stablyTestArgs: "--project critical --grep @p0"
  
  # Full suite Friday nights
  weekend-regression:
    cron: "0 22 * * 5"
    stablyTestArgs: "--project regression --workers 100"

stably.yaml

schedules:
  # Production monitoring
  production-hourly:
    cron: "0 * * * *"
    stablyTestArgs: "--project smoke --env PRODUCTION"
  
  # Staging validation
  staging-every-4-hours:
    cron: "0 */4 * * *"
    stablyTestArgs: "--project smoke --project critical --env STAGING"

stably.yaml

schedules:
  # P0 tests every 30 minutes
  p0-frequent:
    cron: "*/30 * * * *"
    stablyTestArgs: "--grep @p0"
  
  # P1 tests every 4 hours
  p1-regular:
    cron: "0 */4 * * *"
    stablyTestArgs: "--grep @p1"
  
  # P2 tests nightly
  p2-nightly:
    cron: "0 3 * * *"
    stablyTestArgs: "--grep @p2"

How Schedules Work

Configuration

Define schedules in stably.yaml in your repository root

Deployment

Publish your changes

Automatic Execution

Stably runs your tests at the scheduled times in the configured timezone (UTC by default)

Results & Notifications

View results in your dashboard and receive notifications based on your notification settings

Best Practices

Match Testing Frequency to Risk

Run critical tests more frequently (hourly) and comprehensive suites less often (daily/weekly)

Use Descriptive Names

Name schedules clearly to indicate what and when (e.g., weekday-morning-smoke)

Optimize for Fast Feedback

Run smoke tests frequently (every 15-60 minutes)
Run comprehensive regression suites less often (daily/weekly)
Use --grep to run only priority tests more frequently

Leverage Off-Peak Hours

Schedule resource-intensive tests during off-peak hours
Run nightly regressions when usage is low
Consider weekend runs for comprehensive testing

Use Multiple Schedules

Create different schedules for different test types
Separate production monitoring from staging validation
Balance frequency with test importance

Monitor Schedule Performance

Review execution times and adjust schedules accordingly
Ensure tests complete before the next scheduled run
Optimize slow tests that delay feedback

Viewing Scheduled Runs

Access Dashboard

Navigate to your project in the Stably dashboard

View Scheduled Runs

See all scheduled test runs with:

Schedule name and configuration
Last execution time and results
Next scheduled execution time
Execution history and trends

Review Results

Click on any run to see:

Detailed test results with pass/fail status
Screenshots and traces for failures
AI-powered failure analysis
Historical trends and flakiness detection

Troubleshooting

Schedules Not Running

Problem: Scheduled tests aren’t executingSolution:

Verify stably.yaml is in repository root
Check cron expression syntax using crontab.guru
Verify project names match your playwright.config.ts

Incorrect Run Times

Problem: Tests run at unexpected timesSolution:

Schedules default to UTC unless a timezone is specified
Use the timezone field to run in your local timezone (e.g., timezone: "America/New_York")
Use a timezone converter to verify the schedule

Invalid Cron Expression

Problem: Schedule not accepted or parsing errorsSolution:

Use standard 5-field cron format only
Don’t use named days/months (MON, JAN, etc.)
Don’t combine day-of-month and day-of-week
Validate using crontab.guru

Complete Configuration Reference

Schedule Entry Structure

schedules:
  <schedule-name>:
    cron: "<cron-expression>"
    stablyTestArgs: "<cli-arguments>"
    timezone: "<iana-timezone>"

Field Definitions

Field	Type	Required	Description
`schedules`	object	Yes	Root object containing all schedule definitions
`<schedule-name>`	string	Yes	Unique identifier for the schedule
`cron`	string	Yes	Standard UNIX cron expression (5 fields)
`stablyTestArgs`	string	No	Arguments passed to `npx stably test`
`timezone`	string	No	IANA timezone (e.g., `America/New_York`). Defaults to UTC
`autofix`	boolean	No	Automatically run `stably fix` on failing tests after the run completes

The stably.yaml file also supports agent configuration. See CLI Agent Configuration for details.

Valid stablyTestArgs

Argument	Example	Description
`--project`	`--project smoke`	Run specific Playwright project
`--grep`	`--grep @p0`	Filter tests by pattern
`--workers`	`--workers 50`	Set parallel test count
`--env`	`--env STAGING`	Set environment for variables
`--retries`	`--retries 2`	Configure test retries

Next Steps

Autofix

Automatically fix failing tests after scheduled runs

Define Test Groups

Organize tests using Playwright projects

Configure Notifications

Set up alerts for scheduled test results

Run on Cloud

Learn about cloud test execution

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Configuration

Troubleshooting

Migration Guides

Resources

Overview

Defining Schedules

Schedule Configuration

Cron Expression Format

Common Cron Patterns

Cron Expression Limitations

Schedule Arguments

Complete Examples

How Schedules Work

Best Practices

Match Testing Frequency to Risk

Use Descriptive Names

Viewing Scheduled Runs

Troubleshooting

Complete Configuration Reference

Schedule Entry Structure

Field Definitions

Valid stablyTestArgs

Next Steps

Autofix

Define Test Groups

Configure Notifications

Run on Cloud

​Overview

​Defining Schedules

​Schedule Configuration

​Cron Expression Format

​Common Cron Patterns

​Cron Expression Limitations

​Schedule Arguments

​Complete Examples

​How Schedules Work

​Best Practices

Match Testing Frequency to Risk

Use Descriptive Names

​Viewing Scheduled Runs

​Troubleshooting

​Complete Configuration Reference

​Schedule Entry Structure

​Field Definitions

​Valid stablyTestArgs

​Next Steps

Autofix

Define Test Groups

Configure Notifications

Run on Cloud

Overview

Defining Schedules

Schedule Configuration

Cron Expression Format

Common Cron Patterns

Cron Expression Limitations

Schedule Arguments

Complete Examples

How Schedules Work

Best Practices

Viewing Scheduled Runs

Troubleshooting

Complete Configuration Reference

Schedule Entry Structure

Field Definitions

Valid stablyTestArgs

Next Steps