Writing reliable locators is often painful—users must inspect HTML, guess selectors, and tweak through trial and error. This becomes especially hard when dealing with dynamic lists or repeating elements.
page.getLocatorsByAI(prompt) lets you describe what you want in natural language (e.g., “all product cards in the carousel”) and automatically returns stable locators.
Why not vanilla Playwright? Low-level page.locator() requires precise selectors that break when the DOM changes. On the other extreme, fully agentic agent.act() is powerful but heavyweight for simple element selection. getLocatorsByAI() bridges the gap—giving you AI-assisted, self-healing locators that feel native to Playwright. Built-in Autofix: The Key Advantage
The biggest pain point with traditional Playwright locators is maintenance. When developers refactor the UI, selectors break and tests fail—even though the functionality still works. With getLocatorsByAI, your tests automatically adapt to DOM changes because they target semantic intent, not brittle selectors.
Traditional Playwright
With getLocatorsByAI
// ❌ Fragile: breaks when class names or structure change
test('add item to cart', async ({ page }) => {
await page.goto('/products');
// These selectors WILL break when:
// - Designer renames .product-card to .item-card
// - Dev wraps cards in a new container div
// - Class becomes .ProductCard (React component)
await page.locator('.product-grid .product-card').first().click();
await page.locator('button.add-to-cart-btn').click();
// Test fails → Manual fix required → Repeat forever
});
// ✅ Resilient: adapts automatically to DOM changes
test('add item to cart', async ({ page }) => {
await page.goto('/products');
// Describes WHAT you want, not WHERE it is
const { locator: products } = await page.getLocatorsByAI(
'product cards in the product listing'
);
await products.first().click();
const { locator: addBtn } = await page.getLocatorsByAI(
'the Add to Cart button'
);
await addBtn.click();
// Class renamed? Structure changed? Still works.
});
Why does this work? getLocatorsByAI uses the page’s accessibility tree (aria snapshot) to understand elements semantically. As long as your UI remains accessible, the AI can locate elements by their meaning rather than their implementation details.Because aria snapshots are non-visual, prompts referencing colors, sizes, or positions (e.g., “the blue button” or “the large header”) won’t work. Describe elements by their role, label, or text content instead.
Installation
Install the Stably Playwright test integration and import it in place of the Playwright test runner:
import { test, expect } from '@stablyai/playwright-test';
getLocatorsByAI method is automatically available on all page objects.
You can also import types for TypeScript:
import type {
AIModel,
GetLocatorsByAIOptions,
GetLocatorsByAIResult
} from '@stablyai/playwright-test';
getLocatorsByAI
Use prompt-based locator generation to find elements without writing fragile selectors.
Basic Usage
import { test, expect } from '@stablyai/playwright-test';
test('interact with product cards', async ({ page }) => {
await page.goto('/products');
const { locator, count, reason } = await page.getLocatorsByAI(
'all product cards in the featured section'
);
// Use like any Playwright locator
await expect(locator).toHaveCount(count);
// Get all matching elements
const cards = await locator.all();
await cards[0].click();
});
How It Works
getLocatorsByAI performs intelligent element location in three steps:
- Aria Snapshot: Captures the page’s accessibility tree to understand the semantic structure of the DOM.
- AI Analysis: Evaluates your prompt against the aria snapshot to identify matching elements.
- Locator Generation: Returns Playwright-native locators that can be used like any standard locator.
Method Signature
await page.getLocatorsByAI(
prompt: string,
options?: GetLocatorsByAIOptions
): Promise<GetLocatorsByAIResult>
prompt - Natural language description of the element(s) to locateoptions - Optional configuration
Types:
interface GetLocatorsByAIOptions {
model?: AIModel;
}
type AIModel =
| "openai/o4-mini"
| "google/gemini-3-pro-preview"
| "google/gemini-3-flash-preview";
interface GetLocatorsByAIResult {
locator: Locator; // Playwright locator for matched elements
count: number; // Number of elements found
reason: string; // AI's explanation of how it found the elements
}
locator behaves exactly like Playwright’s native Locator object. Call .all() to get an array of individual locators for each matched element.
Common Use Cases
Locate Dynamic Lists
const { locator } = await page.getLocatorsByAI('all items in the shopping cart');
const items = await locator.all();
const { locator } = await page.getLocatorsByAI('the submit button in the checkout form');
await locator.click();
const { locator } = await page.getLocatorsByAI('the third row in the users table');
await expect(locator).toBeVisible();
Model Selection
You can specify which AI model to use. If not specified, the backend default is used.
// Use Google's fast model
const { locator } = await page.getLocatorsByAI('the submit button', {
model: 'google/gemini-3-flash-preview'
});
// Use OpenAI's reasoning model
const { locator } = await page.getLocatorsByAI('the third product card', {
model: 'openai/o4-mini'
});
| Model | Provider | Characteristics |
|---|
google/gemini-3-flash-preview | Google | Fast, efficient |
google/gemini-3-pro-preview | Google | Most capable |
openai/o4-mini | OpenAI | Efficient reasoning |
Best Practices
One Locator Type Per Prompt
While you can request multiple unrelated locators in a single prompt (e.g., “get all the circle buttons and the first two headers”), this is discouraged. Playwright’s strictness model recommends locators that uniquely identify target elements. Mixing unrelated elements in one locator can lead to unexpected behavior when the page changes. // ❌ Discouraged: multiple unrelated element types
const { locator } = await page.getLocatorsByAI('all the add buttons and the page title');
// ✅ Recommended: separate calls for different element types
const { locator: addButtons } = await page.getLocatorsByAI('all Add to Cart buttons');
const { locator: title } = await page.getLocatorsByAI('the page title');
Use the reason Field
The reason field explains how the AI interpreted your prompt. Use it for debugging when locators don’t match as expected. It’s also attached to the test report.
const { locator, count, reason } = await page.getLocatorsByAI('all product cards');
console.log(`Found ${count} elements`);
console.log(`AI reason: ${reason}`);
Troubleshooting
No elements found
The AI couldn’t locate elements matching your description. Refine your prompt to use semantic identifiers.
// ❌ Too vague or visual
const { count } = await page.getLocatorsByAI('the buttons');
// count: 0 (or unexpected results)
// ✅ More specific and semantic
const { count } = await page.getLocatorsByAI('buttons with role "button" in the form');
// ❌ Matches all links on the page
const { locator } = await page.getLocatorsByAI('links');
// ✅ Scoped to specific section
const { locator } = await page.getLocatorsByAI('links in the footer navigation');
_snapshotForAI not being a function, you need to upgrade Playwright:
npm install @playwright/test@latest
getLocatorsByAI requires Playwright v1.54.1 or higher.
When to Use
-
Use
getLocatorsByAI when:
- You need to locate elements in dynamic lists or repeated structures
- Writing precise selectors is difficult due to complex or changing DOM
- You want self-healing locators that adapt to minor DOM changes
- You’re prototyping tests and want to describe elements naturally
-
Use standard Playwright locators when:
- You have stable, unique selectors (IDs, data-testid attributes)
- Performance is critical and AI overhead is not acceptable
- The element structure is simple and unlikely to change
References
