Autofix for Tests

Documentation Index

Fetch the complete documentation index at: https://docs.stably.ai/llms.txt

Use this file to discover all available pages before exploring further.

• Accurate triage — Compares past and current test run replays side-by-side to pinpoint exactly what changed. Digs into network requests, console logs, and DOM state to find the real root cause — especially effective at eliminating flaky tests.
• Scales to real workloads — Whether you have 5 failures or 500, Autofix spins up on-demand cloud browsers to validate every fix in parallel. Built for production, not demos.
• Every fix comes with proof — No vibe-test fixes. Every code change is re-run in a real browser, producing a full replay report with screenshots, traces, and DOM snapshots before anything reaches your codebase.

​

Quick Setup

stably test || stably fix

schedules:
  my-nightly-run:
    cron: "0 0 * * *"
    autofix: true

​

Ways to Use Autofix

​

Automatic (at trigger time)

​

On-demand (after any failed run)

• Autofix on Cloud — Click “Fix with Agent” to start a cloud agent session. The agent diagnoses failures and generates fixes on Stably infrastructure; at the end you can create a PR (if your repo is connected).
• Auto-heal on your device with CLI — Copy the ready-to-run npx stably fix <runId> command and run it locally. Fixes are applied to your working tree. Review with git diff, commit when ready.

​

In CI

​

Comparison

​

What Autofix Can Fix

await page.getByRole('button', { name: 'Continue to Payment' }).click();
await page.getByLabel('Card number').fill('4242424242424242');

await page.getByRole('button', { name: 'Continue to Shipping' }).click();
await page.getByLabel('Standard Shipping').check();
await page.getByRole('button', { name: 'Continue to Payment' }).click();
await page.getByLabel('Card number').fill('4242424242424242');

// ❌ Discount applied to subtotal instead of pre-tax total
const discount = subtotal * discountRate;

// ✅ Discount applied to pre-tax total as expected
const discount = preTaxTotal * discountRate;

await page.goto('/dashboard');
await page.getByRole('button', { name: 'Export' }).click();

await page.goto('/dashboard');
await page.waitForResponse('**/api/dashboard/stats');
await page.getByRole('button', { name: 'Export' }).click();

​

How Autofix handles failing locators

await page.getByRole('button', { name: 'Submit' }).click();

await page.getByRole('button', { name: 'Place Order' }).click();

await page.locator('.pricing-toggle > div:nth-child(2) > button.active').click();

const [annualToggle] = await page.getLocatorsByAI('annual billing toggle button');
await annualToggle.click();

​

How It Works

1

Triage

Autofix analyzes all failing tests and groups them by root cause — is it an outdated test, a real bug, or a flaky test? Repeated failures with the same root cause are automatically skipped to save cost.

2

Fix

Each issue gets a targeted fix. Autofix opens a real browser to inspect your live application, understand what changed, and apply the right code changes.

3

Validate in a real browser

Every fix is re-run in a real browser to confirm it actually works. You get a full replay report with screenshots, traces, and DOM snapshots — proof that the fix is correct before any code is merged.

4

Deliver fixes

A summary report is generated with root causes and code diffs. How fixes reach your codebase depends on the trigger method:

• Cloud runs with connected repo: A PR/MR is created automatically. Review and merge when ready — Stably does not push to your repo until you merge.
• Cloud runs without connected repo: View diagnosis and code diffs in the dashboard. Apply changes to your codebase manually, or connect your repo to enable automatic PRs.
• CLI runs: Fixes are applied to local files. Results are always uploaded to the dashboard — review diffs and create a PR from there (repo connected). In CI, you can also add git commit/push/PR steps to your workflow if you prefer.

​

FAQ

​

Next Steps