CLI commands

$ stably

🤖 Stably Agent
   Type your request or question. Press Ctrl+C to exit.

> Create a test for the checkout flow on our e-commerce site

Analyzing your application...
I'll create a test that covers:
  • Adding items to cart
  • Proceeding to checkout
  • Completing payment

✓ Created tests/checkout.spec.ts

> The login test is failing with a timeout error

Looking at the failure context...
The selector '.login-btn' no longer exists. I found a matching
element with '[data-testid="sign-in"]'.

Apply fix? (y/n): y
✓ Updated tests/auth.spec.ts

> What's our test coverage for the dashboard?

You have 12 tests covering the dashboard:
  • 4 tests for user settings
  • 3 tests for analytics widgets
  • 5 tests for navigation

Missing coverage: notification preferences, export functionality

# Capture output paths
stably create "login" | grep "^- " | cut -c3-

$ stably create "checkout flow for guest users"

Analyzing application...
Generating tests for: checkout flow for guest users

Created files:
- /absolute/path/to/tests/checkout-guest-add-to-cart.spec.ts
- /absolute/path/to/tests/checkout-guest-payment.spec.ts

# .github/workflows/auto-tests.yml
name: Auto-generate Tests

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  generate-tests:
    # Skip PRs created by stably-bot to prevent infinite loops
    if: github.event.pull_request.user.login != 'stably-bot'
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.head_ref }}

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install browsers
        run: npx stably install

      - name: Generate tests
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
          # App credentials for login during test generation
          TEST_USERNAME: ${{ secrets.TEST_USERNAME }}
          TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
        run: npx stably create  # Automatically analyzes PR changes

      - name: Check for new tests
        id: check
        run: |
          # Checks for changes in common test directories
          if [[ -n $(git status --porcelain tests/ e2e/ __tests__ 2>/dev/null) ]]; then
            echo "has_changes=true" >> $GITHUB_OUTPUT
          fi

      - name: Create PR with generated tests
        if: steps.check.outputs.has_changes == 'true'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          BRANCH="auto-tests/${{ github.head_ref }}-${{ github.run_number }}"
          git config user.name "stably-bot"
          git config user.email "bot@stably.ai"
          git checkout -b "$BRANCH"
          git add tests/ e2e/ __tests__/
          git commit -m "test: auto-generate tests for PR #${{ github.event.pull_request.number }}"
          git push -u origin "$BRANCH"
          gh pr create \
            --title "Generated tests for #${{ github.event.pull_request.number }}" \
            --body "Auto-generated tests for the changes in #${{ github.event.pull_request.number }}" \
            --base "${{ github.head_ref }}"

# .github/workflows/staging-tests.yml
name: Generate Tests from Staging

on:
  deployment_status:
    # Triggers when any deployment status changes.
    # Example: if you have a "staging" deployment environment in GitHub,
    # this fires automatically when that deployment succeeds.
    # See: https://docs.github.com/en/actions/deployment/about-deployments
  workflow_dispatch:
    # Allow manual triggers for ad-hoc test generation

jobs:
  generate-staging-tests:
    runs-on: ubuntu-latest
    # Only run on successful staging deployments (skip production, preview, etc.)
    # Skip PRs created by stably-bot to prevent infinite loops
    if: >
      (github.event_name == 'workflow_dispatch' ||
       (github.event.deployment_status.state == 'success' &&
        github.event.deployment.environment == 'staging')) &&
      github.actor != 'stably-bot'
    permissions:
      contents: write
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
        with:
          # Check out the exact commit that was deployed
          ref: ${{ github.event.deployment.sha || github.sha }}

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install browsers
        run: npx stably install

      - name: Generate tests for staging
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
          # App credentials for login during test generation
          TEST_USERNAME: ${{ secrets.TEST_USERNAME }}
          TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
        run: |
          npx stably create "Go to ${{ vars.STAGING_URL }} and create tests for any new features between this and the last staging deployment. Plan it out first."

      - name: Check for new tests
        id: check
        run: |
          # Checks for changes in common test directories
          if [[ -n $(git status --porcelain tests/ e2e/ __tests__ 2>/dev/null) ]]; then
            echo "has_changes=true" >> $GITHUB_OUTPUT
          fi

      - name: Create PR with generated tests
        if: steps.check.outputs.has_changes == 'true'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          BRANCH="staging-tests/$(date +%Y%m%d-%H%M%S)"
          git config user.name "stably-bot"
          git config user.email "bot@stably.ai"
          git checkout -b "$BRANCH"
          git add tests/ e2e/ __tests__/
          git commit -m "test: auto-generate tests from staging deployment"
          git push -u origin "$BRANCH"
          gh pr create \
            --title "Generated tests from staging deployment" \
            --body "Auto-generated tests based on new features detected on staging." \
            --base main

# Called by AI coding agents (Cursor, Copilot, etc.)
# The agent can invoke this command to generate tests autonomously

stably create "PaymentService class with edge cases"

# Chain with test execution
stably create "user login and logout flow" && stably test

$ stably fix run_abc123

Analyzing 3 failures...

checkout.spec.ts > complete purchase
  Issue: Selector '.checkout-button' not found
  Fix: Updated to '[data-testid="checkout-btn"]'
  ✓ Fixed

login.spec.ts > invalid credentials
  Issue: Error message assertion failed
  Fix: Updated expected text to match new design
  ✓ Fixed

Summary: 2 auto-fixed, 1 requires manual review

# .github/workflows/self-healing-tests.yml
name: Self-Healing Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test-and-fix:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci && npx stably install

      - name: Run tests
        id: test
        continue-on-error: true
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
        run: npx stably test

      - name: Auto-fix failures
        if: steps.test.outcome == 'failure'
        env:
          STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
          STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
        run: npx stably fix

      - name: Commit fixes
        if: steps.test.outcome == 'failure'
        run: |
          git config user.name "stably-bot"
          git config user.email "bot@stably.ai"
          git add tests/
          git commit -m "fix: auto-repair failing tests" || exit 0
          git push

# Called by AI coding agents for autonomous test maintenance
# Chain test execution with automatic repair

stably test || stably fix

# Full pipeline: test → fix → verify
stably test || (stably fix && stably test)