Stably - AI-native Testing Platform

Visual assertions are the flakiest part of Playwright test suites. Stably augments the Playwright runner with AI-powered screenshot prompts that understand intent, tolerate dynamic content, and explain failures—no golden baseline required.

Why not vanilla Playwright? expect(page).toHaveScreenshot() excels for static pages with reliable baselines. For highly dynamic UIs, Stably’s prompt assertions cover intent-based validation that Playwright’s pixel diffing cannot reliably express.

Installation

Install the Stably Playwright test integration and import it in place of the Playwright test runner:

import { test, expect } from '@stablyai/playwright-test';

The API surface stays the same as Playwright, but you gain access to AI assertions.

aiAssert

Use prompt-based visual assertions to validate your UI without maintaining golden screenshots.

Basic Usage

import { test, expect } from '@stablyai/playwright-test';

test('personalized dashboard renders key metrics', async ({ page }) => {
  await page.goto('/dashboard');

  await expect(page).aiAssert(
    'Shows revenue trend chart for at least 6 months and spotlight card for this account',
    {
      fullPage: true, // optional, defaults to false
    }
  );
});

How It Works

aiAssert performs intelligent visual validation in three steps:

Stabilization: Waits for the page to stabilize and captures the screenshot.
AI Analysis: Evaluates the screenshot against your prompt using Stably’s vision model.
Assertion Outcome: If the prompt is satisfied, the assertion resolves. Otherwise it throws with a natural-language explanation describing what the AI found missing or inconsistent.

Each assertion performs one AI call that completes in a few seconds, and failures surface the AI reasoning in the thrown Playwright assertion error. aiAssert includes built-in caching—if the captured screenshot hasn’t changed since the last run, no AI call is made, saving time and cost.

Method Signature

await expect(page).aiAssert(
  prompt: string,
  options?: ScreenshotPromptOptions
): Promise<void>

Parameters:

prompt - Human-readable description of what should appear on screen
options - Configuration options
- model - AI model to use (see Model Selection)
- Plus all standard Playwright screenshot options (fullPage, animations, etc.)

Types:

type AIModel =
  | "openai/o4-mini"
  | "google/gemini-3.1-pro-preview"
  | "google/gemini-3-flash-preview";

// ScreenshotPromptOptions extends Playwright's screenshot options
interface ScreenshotPromptOptions {
  model?: AIModel;
  fullPage?: boolean;
  animations?: 'disabled' | 'allow';
  clip?: { x: number; y: number; width: number; height: number };
  timeout?: number;
  // ... plus other Playwright screenshot options
}

The assertion resolves when the UI matches the intent described in the prompt. On failure it throws a Playwright assertion error with the AI-generated reasoning.

Common Use Cases

Validate Personalization

await expect(page).aiAssert(
  'Hero advertises "Welcome back, Taylor" with a check-in button below'
);

Check Conditional UI States

await expect(page.locator('.toast')).aiAssert(
  'Displays success toast with green icon and copy "Changes saved"'
);

Verify Data Visualizations

await expect(page.locator('#chart')).aiAssert(
  'Line chart trends upward with three milestone markers labeled M1, M2, M3'
);

These assertions stay resilient even as layout, colors, or supporting content evolve, as long as the core intent aligns with the prompt.

Model Selection

You can specify which AI model to use. If not specified, the backend default is used.

await expect(page).aiAssert(
  'Dashboard shows revenue chart and user metrics',
  { model: 'google/gemini-3-flash-preview' }
);

Model	Provider	Characteristics
`google/gemini-3-flash-preview`	Google	Fast, efficient
`google/gemini-3.1-pro-preview`	Google	Most capable
`openai/o4-mini`	OpenAI	Efficient reasoning

Advanced Options

All standard Playwright screenshot options are supported:

await expect(page).aiAssert(
  'Left nav shows an orange "Live" badge and the promo banner highlights AI upgrades',
  {
    model: 'google/gemini-3-pro-preview',
    animations: 'disabled',
    fullPage: true,
    clip: { x: 0, y: 0, width: 1280, height: 720 },
    timeout: 30000,
  }
);

Common options:

model - AI model to use (see Model Selection)
fullPage - Capture the full scrollable page
animations - Set to 'disabled' to disable CSS animations
clip - Capture a specific rectangular area
timeout - Maximum time to wait for stabilization

Scoping with Locators

Combine with Playwright locators to scope assertions to specific components:

// Assert on a specific element
await expect(page.locator('.header')).aiAssert(
  'Navigation bar with user profile and notifications icon'
);

// Assert on the entire page
await expect(page).aiAssert(
  'Dashboard with sidebar, header, and main content area'
);

When to Use

Use aiAssert when:
- The page can change materially between runs while maintaining semantic intent
- You need to validate invariants (e.g., “shows revenue chart”) even when the UI layout or styling changes
- You have dynamic content (personalized data, A/B tests, real-time updates)
- Layout and styling evolve frequently and describing intent is easier than maintaining golden snapshots
Use expect(page).toHaveScreenshot() when:
- You have a stable, non-dynamic screen
- Minor rendering differences are acceptable (Stably’s auto-heal handles font rendering, anti-aliasing, and subtle visual variations)
- Pixel-perfect matching is important and the page structure rarely changes

Stably offers auto-heal for toHaveScreenshot() that handles minor rendering differences like font anti-aliasing and subtle layout shifts. However, for pages where major structural changes are acceptable as long as semantic invariants remain valid, use aiAssert instead. Prompt-driven assertions tolerate variance, provide contextual reasoning, and continue to work even when UI components are re-arranged or re-styled.

toMatchAriaPrompt (Coming Soon)

Stably is expanding prompt assertions beyond screenshots to support semantic validation directly on accessibility trees.

Method Signature

await expect(page).toMatchAriaPrompt(
  prompt: string,
  options?: {
    cache?: {
      name: string;
    } | true;
  }
): Promise<{ success: boolean; reason?: string }>

This method will validate UI semantics and structure without requiring visual screenshots, enabling faster and more accessible test assertions.

Best Practices

Write intent-focused prompts: Mention critical UI elements (primary CTA, key metrics, legal copy) rather than pixel-perfect descriptions.
Stabilize before capture: Pair assertions with network and animation stabilization waits to reduce UI churn.
Scope assertions: Use Playwright locators to focus on components that matter.
Be specific: Provide enough detail in prompts for the AI to differentiate success states.

Troubleshooting

Prompt is too vague Refine the description with salient visual cues. The AI needs enough detail to differentiate success states.

// ❌ Too vague
await expect(page).aiAssert('Shows dashboard');

// ✅ Specific
await expect(page).aiAssert(
  'Dashboard with revenue chart trending up, user count of 1,234, and green success banner'
);

Assertion feels slow Keep assertions scoped using locators, and watch for upcoming caching updates to accelerate reruns.

// Faster: scope to specific component
await expect(page.locator('.metric-card')).aiAssert(
  'Revenue: $12,345 with +15% indicator'
);

Unexpected failure reasoning Review the assertion error message—it often reveals missing UI elements or structural changes worth addressing.

try {
  await expect(page).aiAssert('Dashboard shows revenue card and usage chart');
} catch (error) {
  console.error('AI assertion failed:', error.message);
  // Example: "Expected spotlight card with user quota, but found subscription upsell banner instead."
  throw error;
}

References

Stably Playwright SDK: @stablyai/playwright-test
Playwright screenshot options: Playwright PageAssertions

Getting started

Stably Web Editor

Stably CLI

Stably Cloud Runner

Stably SDK

Stably Test Reporter

Configuration

Troubleshooting

Migration Guides

Resources

AI Assertions

Installation

aiAssert

Basic Usage

How It Works

Method Signature

Common Use Cases

Model Selection

Advanced Options

Scoping with Locators

When to Use

toMatchAriaPrompt (Coming Soon)

Method Signature

Best Practices

Troubleshooting

References

​Installation

​aiAssert

​Basic Usage

​How It Works

​Method Signature

​Common Use Cases

​Model Selection

​Advanced Options

​Scoping with Locators

​When to Use

​toMatchAriaPrompt (Coming Soon)

​Method Signature

​Best Practices

​Troubleshooting

​References

Installation

aiAssert

Basic Usage

How It Works

Method Signature

Common Use Cases

Model Selection

Advanced Options

Scoping with Locators

When to Use

toMatchAriaPrompt (Coming Soon)

Method Signature

Best Practices

Troubleshooting

References