Skip to main content

​
AI auto-heal

Stably AI keeps your Playwright tests resilient as your app evolves. Auto-heal operates at two layers:
  • Action-level auto-heal: Fixes broken locators and stabilizes visual assertions in-line during a run.
  • Maintenance agents: Turn successful heals into code changes (e.g., PRs) so your source tests stay up to date.

​
Action-level auto-heal

Action-level healing runs during test execution and attempts the smallest safe fix required for the step to pass.

​
Getting started

Run your existing Playwright tests with Stably to enable auto-heal: 
npx stably test
That’s it. Stably automatically detects failures and applies intelligent healing where possible.

​
Locator auto-heal with describe()

We leverage Playwright’s new locator annotation API to add intent directly to locators. Locator auto-heal only works with locators that use describe()—the human-readable description improves traces and enables healing. 
// Example: annotate locators with intent using describe()
await page.getByTestId('get-started-button').describe('CTA button').click();
How it works:
  • When a locator with describe() fails, Stably’s AI uses the description and page context to find the updated element.
  • Successful alternatives are persisted in the test reporter, which can then be used by Maintenance Agents to auto-generate a file change to fix the source test.
  • Locators without describe() will not be auto-healed.
What if I don’t have any describe() annotations?You can quickly add them to your existing tests using Cursor or Claude. Use this prompt:
Add .describe() annotations to all Playwright locators in this file. 
Use nearby comments, variable names, and context to create clear, 
human-readable descriptions. For example:
// Click the CTA button
- await page.getByRole('button', { name: 'Get Started' }) 
  becomes: await page.getByRole('button', { name: 'Get Started' }).describe('CTA button').click();
For future AI-generated tests, ensure your tests automatically include describe() annotations. See our Cursor/Claude Code Agent templates for best practices.

​
Auto-heal for toHaveScreenshot()

toHaveScreenshot() is the most common source of flakiness in tests. Stably distinguishes layout/content changes from benign render variance and heals accordingly. How it works:
  • When Playwright detects a visual diff and the assertion fails, Stably uses AI to analyze whether the diff is due to benign render variance (font hinting, subpixel movement, antialiasing) or an actual UI change.
  • If the diff is determined to be render-only, Stably does not report it as a failure—keeping your test runs clean of false positives.
// Your existing assertion
await expect(page).toHaveScreenshot('dashboard.png');
This approach dramatically reduces screenshot flakiness without requiring manual tolerance tuning.

​
Configure models and behavior

You control the model and category-level toggles in stably.config.ts.
Model support: Stably currently supports only claude-sonnet-4-5 for auto-heal functionality. This model has demonstrated the best accuracy and reliability for both locator and screenshot healing. We plan to expand support for additional models in the future.
import { defineStablyConfig } from '@stablyai/playwright-test';

// stably.config.ts
export default defineStablyConfig({
  use: {
    autoHeal: {
      // Global defaults
      enabled: true,
      model: 'claude-sonnet-4-5',

      // Locator-specific config
      locators: {
        // inherits global settings, can override
        enabled: true,
        timeout: 30000,
      },

      // Screenshot-specific config
      screenshots: {
        enabled: true,
      },
    },
  },
});
Configuration notes:
  • Set autoHeal.enabled: false to turn auto-heal off globally.
  • Override locators and screenshots to tune each category independently.
  • The timeout option controls how long to wait for auto-heal attempts (default: 30000ms).

​
Best practices

  • Always use describe() on locators you want to auto-heal. Without describe(), locator healing will not activate.
  • Keep toHaveScreenshot() targets deterministic by waiting for network/animation idle states before asserting.
  • Use Maintenance Agents for substantive flow changes; rely on action-level healing for incremental, non-breaking UI shifts.

​
Troubleshooting

  • Locator keeps flapping: Annotate with describe() and add nearby context (e.g., “submit in checkout form”). Consider scoping with getByRole or getByTestId for more stability.
  • Visual diff only on CI: Ensure consistent fonts and GPU settings in your CI environment. Stably analyzes render-only variance, but if diffs persist, you may need to update your baseline screenshot or add a masked region in Playwright.
  • Too much healing: Update your source test to have a good baseline. You can find file update suggestions in our test reporter to apply the healed locators back to your code.

​
References

  • Playwright locator descriptions (describe()): see the latest Playwright release notes and examples.