Skip to main content

​
AI auto-heal

Stably AI keeps your Playwright tests resilient as your app evolves. Auto-heal operates at two layers:
  • Action-level auto-heal: Fixes broken locators and stabilizes visual assertions in-line during a run.
  • Full regeneration (Maintenance Agent): Rewrites parts of a test when the underlying flow changed. See Maintenance agents.

​
Action-level auto-heal

Action-level healing runs during test execution and attempts the smallest safe fix required for the step to pass.

​
Getting started

Run your existing Playwright tests with Stably to enable auto-heal: 
npx stably test
That’s it. We handle the rest—zero code changes required on your end. Stably automatically detects failures and applies intelligent healing where possible.

​
Locator auto-heal with describe()

We leverage Playwright’s new locator annotation API to add intent directly to locators. Locator auto-heal only works with locators that use describe()—the human-readable description improves traces and enables healing. 
// Example: annotate locators with intent using describe()
await page.getByTestId('get-started-button').describe('CTA button').click();
How it works:
  • When a locator with describe() fails, Stably’s AI uses the description and page context to find the updated element.
  • Successful alternatives are persisted in the test reporter, which can then be used by Maintenance Agents to auto-generate a file change to fix the source test.
  • Locators without describe() will not be auto-healed.
What if I don’t have any describe() annotations?You can quickly add them to your existing tests using Cursor or Claude. Use this prompt:
Add .describe() annotations to all Playwright locators in this file. 
Use nearby comments, variable names, and context to create clear, 
human-readable descriptions. For example:
// Click the CTA button
- await page.getByRole('button', { name: 'Get Started' }) 
  becomes: await page.getByRole('button', { name: 'Get Started' }).describe('CTA button').click();
For future AI-generated tests, ensure your tests automatically include describe() annotations. See our Cursor/Claude Code Agent templates for best practices.

​
Auto-heal for toHaveScreenshot()

toHaveScreenshot() is the most common source of flakiness in tests. Stably distinguishes layout/content changes from benign render variance and heals accordingly. How it works:
  • When Playwright detects a visual diff and the assertion fails, Stably uses AI to analyze whether the diff is due to benign render variance (font hinting, subpixel movement, antialiasing) or an actual UI change.
  • If the diff is determined to be render-only, Stably does not report it as a failure—keeping your test runs clean of false positives.
// Your existing assertion
await expect(page).toHaveScreenshot('dashboard.png');
This approach dramatically reduces screenshot flakiness without requiring manual tolerance tuning.

​
Full test regeneration

When the UI or flow meaningfully changes (e.g., a new intermediate step or renamed route), Stably’s Maintenance Agent can regenerate steps using the test’s intent, prior run history, and your product knowledge base. See Maintenance agents for the end‑to‑end flow and review UX.

​
Configure models and behavior

You control the model and category-level toggles in stably.config.ts.
Model support: Stably currently supports only claude-sonnet-4-5 for auto-heal functionality. This model has demonstrated the best accuracy and reliability for both locator and screenshot healing. We plan to expand support for additional models in the future.
import { defineStablyConfig } from '@stablyai/playwright-test';

// stably.config.ts
export default defineStablyConfig({
  use: {
    autoHeal: {
      // Global defaults
      enabled: true,
      model: 'claude-sonnet-4-5',

      // Locator-specific config
      locators: {
        // inherits global settings, can override
        enabled: true,
        timeout: 30000,
      },

      // Screenshot-specific config
      screenshots: {
        enabled: true,
      },
    },
  },
});
Configuration notes:
  • Set autoHeal.enabled: false to turn auto-heal off globally.
  • Override locators and screenshots to tune each category independently.
  • The timeout option controls how long to wait for auto-heal attempts (default: 30000ms).

​
Best practices

  • Always use describe() on locators you want to auto-heal. Without describe(), locator healing will not activate.
  • Keep toHaveScreenshot() targets deterministic by waiting for network/animation idle states before asserting.
  • Use Maintenance Agents for substantive flow changes; rely on action-level healing for incremental, non-breaking UI shifts.

​
Troubleshooting

  • Locator keeps flapping: Annotate with describe() and add nearby context (e.g., “submit in checkout form”). Consider scoping with getByRole or getByTestId for more stability.
  • Visual diff only on CI: Ensure consistent fonts and GPU settings in your CI environment. Stably analyzes render-only variance, but if diffs persist, you may need to update your baseline screenshot or add a masked region in Playwright.
  • Too much healing: Update your source test to have a good baseline. You can find file update suggestions in our test reporter to apply the healed locators back to your code.

​
References

  • Playwright locator descriptions (describe()): see the latest Playwright release notes and examples.
  • Stably Maintenance Agents: Auto-maintainance agents.
⌘I