The Stably Reporter streams your Playwright test results, traces, and screenshots to Stably for debugging and AI-powered maintenance with stably fix.
With the reporter enabled, you get a rich test results dashboard with detailed run history, failure analysis, and auto-heal reports that show exactly what changed and how Stably fixed your tests.
Installation
npm install -D @stablyai/playwright-test@latest
Requires Playwright 1.52.0+
Configuration
Using stably test? The reporter is automatically configured—no setup needed. Skip to Verify Setup. .env file:
STABLY_API_KEY=your_api_key_here # Get from https://auth.stably.ai/org/api_keys/
STABLY_PROJECT_ID=your_project_id_here # Find in your project URL on app.stably.ai
playwright.config.ts:
import { defineConfig } from "@playwright/test";
import { stablyReporter } from "@stablyai/playwright-test";
import dotenv from "dotenv";
import path from "path";
dotenv.config({ path: path.resolve(__dirname, ".env") });
export default defineConfig({
reporter: [
["list"],
stablyReporter({
apiKey: process.env.STABLY_API_KEY,
projectId: process.env.STABLY_PROJECT_ID,
}),
],
use: {
trace: "on", // Required for debugging in Stably dashboard
},
});
Set trace: "on" instead of the default "on-first-retry" to capture traces for all runs.
Verify Setup
Run a test to confirm the reporter is working:
You should see a dashboard URL printed after tests complete. View results at app.stably.ai.
Using with stably fix
The reporter enables AI-powered test maintenance:
npx stably test # Run tests (results stream to Stably)
stably fix # AI analyzes failures and generates fixes (auto-detects run ID)
Sensitive Data Scrubbing
You can prevent sensitive values from appearing in traces by passing a sensitiveValues string array to the reporter. Each string will be redacted wherever it appears exactly in the trace.
stablyReporter({
apiKey: process.env.STABLY_API_KEY,
projectId: process.env.STABLY_PROJECT_ID,
sensitiveValues: [
process.env.TEST_USER_PASSWORD,
process.env.API_SECRET,
// Add any other sensitive strings to scrub
],
});
Sensitive values are scrubbed from trace data, but screenshots and videos may still display secrets if they appear on screen during test execution. Avoid entering sensitive data in visible form fields when possible.Only exact string matches are scrubbed—partial matches within objects or arrays are not detected.To ensure the integrity of Playwright artifacts, the following patterns are never scrubbed—even when included in sensitiveValues:
- The characters
/ and * (when alone)
- Strings containing
/artifact/, https://, http://, or *********
- Pure numeric values (e.g.,
123, 456789)
- Values of internal keys:
sha1, _sha1, pageref, downloadsPath, tracesDir, pageId, and any key ending in sha1
Troubleshooting
| Issue | Solution |
|---|
| Missing traces | Set trace: "on" in config, or use stably test |
| Auth errors | Verify STABLY_API_KEY and STABLY_PROJECT_ID |
Next Steps
