Playwright AI Tests

# Stably SDK — AI Rules (Playwright‑compatible)

**Assumption:** Full Playwright parity unless noted below.

## Import

```ts
import { test, expect } from "@stablyai/playwright-test";
```

## Install & Setup

```bash
npm install @playwright/test @stablyai/playwright-test
export STABLY_API_KEY=YOUR_KEY
```

```ts
import { setApiKey } from "@stablyai/playwright-test";
setApiKey("YOUR_KEY");
```


## AI Assertions (intent‑based visuals)

```ts
await expect(page).toMatchScreenshotPrompt(
  "Shows revenue trend chart and spotlight card",
  { timeout: 30_000 }
);
await expect(page.locator(".header"))
  .toMatchScreenshotPrompt("Nav with avatar and bell icon");
```

**Signature:** `expect(page|locator).toMatchScreenshotPrompt(prompt: string, options?: ScreenshotOptions)`

* Use for **dynamic** UIs; keep prompts specific; scope with locators when possible.
* **Consider whether you need `fullPage: true`**: Ask yourself if the assertion requires content beyond the visible viewport (e.g., long scrollable lists, full page layout checks). If only viewport content matters, omit `fullPage: true` — it's faster and cheaper. Use it only when you genuinely need to capture content outside the browser window's visible area. 

## AI Extraction (visual → data)

```ts
const txt = await page.extract("List revenue, active users, and churn rate");
```

Typed with Zod:

```ts
import { z } from "zod";
const Metrics = z.object({ revenue: z.string(), activeUsers: z.number(), churnRate: z.number() });
const m = await page.extract("Return revenue (currency), active users, churn %", { schema: Metrics });
```

**Signatures:**

* `page.extract(prompt: string): Promise<string>`
* `page.extract<T extends z.AnyZodObject>(prompt, { schema: T }): Promise<z.output<T>>`

## CI Reporter / Cloud

```bash
npm install @stablyai/playwright-test
```

```ts
// playwright.config.ts
import { defineConfig } from "@playwright/test";
import { stablyReporter } from "@stablyai/playwright-test";

export default defineConfig({
  reporter: [
    ["list"],
    stablyReporter({ apiKey: process.env.STABLY_API_KEY, projectId: "YOUR_PROJECT_ID" }),
  ],
});
```


## Commands

```bash
npx playwright test   # preferred to enable full auto‑heal path
# All Playwright CLI flags still work (headed, ui, project, file filters…)

# When running tests for debugging/getting stacktraces:
npx playwright test --reporter=list  # disable HTML reporter, shows terminal output directly
```

## Development Workflow

**Iterative Test Development:**
* You can **dynamically simulate test execution in the browser**, change tests accordingly, until the test passes
* When debugging failures, run `npx playwright test` with `--reporter=list` flag to disable the HTML reporter and get immediate stacktrace output in the terminal

## AI Agent (autonomous workflows)

Use the `agent` fixture to execute complex, multi-step workflows:

```ts
test("complex workflow", async ({ agent, page }) => {
  await page.goto("/orders");
  await agent.act("Find the first pending order and mark it as shipped", { page });
});

// Or create manually
const agent = context.newAgent();
await agent.act("Your task here", { page, maxCycles: 30 });
```

**Signature:** `agent.act(prompt: string, options: { page: Page, maxCycles?: number, model?: string }): Promise<{ success: boolean }>`

* Default maxCycles: 30
* Supported models: `anthropic/claude-sonnet-4-5-20250929` (default), `google/gemini-2.5-computer-use-preview-10-2025`

## Best Practices

* Add `.describe()` to important locators; combine with `getByRole`/`getByTestId`.
* Stabilize before visuals (network/animation idle).
* Scope visual checks with locators; keep prompts specific with labels/units.
* Use `toHaveScreenshot` for stable pixel‑perfect UIs; `toMatchScreenshotPrompt` for dynamic UIs.
* **Be deliberate with `fullPage: true`**: Default to viewport-only screenshots. Only use `fullPage: true` when your assertion genuinely requires content beyond the visible viewport (e.g., verifying footer content on a long page, checking full scrollable lists). Viewport captures are faster and more cost-effective. 

## Troubleshooting

* **Slow assertions** → scope visuals; reduce viewport.
* **Agent stops early** → increase `maxCycles` or break task into smaller steps.

## Minimal Template

```ts
import { test, expect } from "@stablyai/playwright-test";

test("AI‑enhanced dashboard", async ({ page, agent }) => {
  await page.goto("/dashboard");
  
  // Use agent for complex workflows
  await agent.act("Navigate to settings and enable notifications", { page });
  
  // Use AI assertions for dynamic content
  await expect(page).toMatchScreenshotPrompt(
    "Dashboard shows revenue chart (>= 6 months) and account spotlight card"
  );
});
```