Skip to main content

Overview

The Stably CLI is a command-line tool for developers who prefer working in the terminal. It includes an interactive AI agent for test creation and the powerful stably fix command for automatic test maintenance.

Authentication

# Get your API key from https://app.stably.ai/settings
export STABLY_API_KEY=stably_...

# Or authenticate interactively
npx stably auth login

Connect Your Project

cd /path/to/your/project
npx stably connect

# This creates a stably.config.yaml and links to your Stably project

Creating Tests

# Start the interactive session
npx stably

> "Create a test for the checkout flow"
 Test created: tests/checkout.spec.ts

> "Add assertions for the order confirmation page"
 Updated test with new assertions

> "Run it"
 Running tests...

One-Shot Commands

# Generate a test from description
npx stably generate "test user registration with email validation"

# Generate from URL (records your actions)
npx stably record https://app.example.com/signup

# Import existing Playwright tests
npx stably import tests/*.spec.ts

Running Tests

Local Runner

Run tests on your machine or in your CI:
# Run all tests locally
npx stably test

# Run specific group
npx stably test smoke

# Run with Playwright options
npx stably test --headed --project=chromium

Cloud Runner

Run tests on Stably’s managed infrastructure:
# Run on Stably cloud
npx stably test smoke --cloud

# Or use the explicit command
npx stably cloud test smoke

# Parallel execution (sharding)
npx stably cloud test regression --shards 5

The npx stably fix Command

The most powerful CLI feature—runs tests, detects failures, and applies AI fixes automatically:
# Run tests and auto-fix failures
npx stably fix

# Fix specific group
npx stably fix smoke

# Dry run (show proposed fixes without applying)
npx stably fix --dry-run

# Interactive mode (review each fix)
npx stably fix --interactive

How It Works

  1. Runs your test suite
  2. For each failure:
    • Captures full context (screenshots, logs, DOM)
    • AI analyzes and generates a fix
    • Applies fix locally (or creates PR)
    • Reruns test to verify
  3. Reports results and success rate

Example Output

npx stably fix

# ✓ 45 tests passed
# ✗ 3 tests failed
# 
# Analyzing failures...
# 
# Test: checkout.spec.ts > complete purchase
# Issue: Selector '.checkout-button' not found
# Fix: Updated to '[data-testid="checkout-btn"]'
# Status: ✓ Fixed and verified
# 
# Test: login.spec.ts > invalid credentials
# Issue: Error message assertion failed
# Fix: Updated expected text to match new design
# Status: ✓ Fixed and verified
# 
# Test: search.spec.ts > filter results
# Issue: Timeout waiting for results
# Fix: Added explicit wait for API response
# Status: ⚠ Manual review needed (confidence: 0.72)
# 
# Summary: 2 auto-fixed, 1 needs review

Command Reference

Core Commands

stably                          # Start interactive AI agent
stably generate DESCRIPTION     # Generate test from description
stably record [url]             # Record test by clicking through app
stably test [group]             # Run tests locally
stably cloud test GROUP         # Run tests on Stably cloud
stably fix [group]              # Run tests and auto-fix failures
stably deploy                   # Deploy tests to cloud
stably open [file]              # Open in web platform

Configuration Commands

stably connect                  # Link project to Stably cloud
stably config init              # Create stably.config.yaml
stably config validate          # Validate configuration
stably auth login               # Authenticate with Stably
stably auth logout              # Remove authentication

Test Management

stably list                     # List all test groups
stably status                   # Show recent test runs
stably logs [run-id]            # View test run logs
stably results [run-id]         # View detailed test results

Advanced Options

# Run with custom reporter
npx stably test --reporter=html,json

# Run with environment variables
npx stably test --env=staging

# Run specific files
npx stably test --grep "checkout"

# Debug mode
npx stably test --debug --headed

# Dry run (validate without executing)
npx stably test --dry-run

Jump Between CLI and Web

Seamlessly switch between terminal and browser:
# Open current test in web editor
npx stably open tests/checkout.spec.ts

# Open test results in dashboard
npx stably open --results

# Start recording in browser
npx stably record --web

Troubleshooting

Authentication Issues

# Clear and re-authenticate
npx stably auth logout
npx stably auth login

# Or use API key directly
export STABLY_API_KEY=stably_...

Tests Work Locally but Fail on Cloud

Check environment variables and base URL:
# stably.config.yaml
cloudRunner:
  env:
    BASE_URL: https://staging.example.com

Next Steps