# Privacy Policy
Source: https://docs.stably.ai/about/privacy-policy
Stably AI's privacy policy detailing data collection, usage, and protection practices for our AI-powered testing platform.
Last Updated: July 25, 2025
# Owner and Data Controller
Lovecast Inc. d/b/a Stably AI.
Owner contact email: [help@stably.ai](mailto:help@stably.ai)
# Introduction
Lovecast Inc. d/b/a Stably AI. ("Stably", "we", "our") provides an AI-powered end-to-end testing platform. This Privacy Policy explains how we collect, use, and protect Personal Data when you use our Service.
# Types of Data Collected
Stably collects the following types of Personal Data to provide and improve our services:
* **Account and Authentication Data:** When you sign in with an account, we collect data necessary for registration and authentication.
* **User Content:** As an AI-powered testing platform, we process data you submit to the service, including files, code, configurations, text, graphics, screenshots, videos, logs, and run artifacts. This is essential for the service to perform end-to-end tests.
* **Usage Data:** We collect data from your interactions with our Service for billing, analytics, and service improvement.
* **Payment Information:** We use Stripe for payment processing. Your payment details are processed securely by Stripe.
* **Support and Communication Data:** We collect information you voluntarily provide through support channels or other communications.
Users are responsible for ensuring they have all necessary rights and consents to provide any data to the Service. Withholding required data may impact the functionality of the Service.
# Mode and Place of Processing the Data
## Methods of processing
Stably implements industry-standard security measures to protect your data from unauthorized access, disclosure, or destruction.
Data processing follows strict organizational procedures. Access is limited to authorized internal teams (e.g., administration, sales, marketing, legal, system administration) and trusted third-party service providers who act as Data Processors (e.g., payment processors, hosting providers, communication tools). These parties are bound by confidentiality agreements and are granted access only as necessary to provide the Service. An updated list of Data Processors is available upon request.
## Legal basis of processing
Stably processes Personal Data on the following legal grounds:
* **Contractual Necessity:** Processing is necessary to deliver the Service as described in our Terms of Service, such as running tests and managing your account.
* **Legitimate Interests:** We process data for our legitimate business interests, like improving our platform, ensuring security, and providing analytics.
* **Consent:** Where applicable, we process data based on your explicit consent (e.g., for marketing communications).
* **Legal Obligations:** We may process data to comply with legal requirements.
A key part of our service involves using AI to power our testing platform. By default, Stably may use User Content to train, fine-tune, or otherwise improve our AI/ML models. This processing is based on our legitimate interest to enhance the service's accuracy and reliability. You may opt out of this use of your User Content for model training by signing our Data Usage Opt-Out Addendum, as detailed in our Terms of Service.
## Place
Stably processes Personal Data on servers located primarily in the United States. Depending on your location, data transfers may occur. If you require your data to be processed in a specific region, please contact us. We will evaluate such requests, but relocation is not guaranteed.
## Retention
Stably retains Personal Data for as long as necessary to fulfill the purposes for which it was collected. Data related to the performance of a contract is retained until the contract is fully performed. Data processed for our legitimate interests is kept as long as needed to achieve those purposes. We may retain data longer if required by law or with your consent.
While you are a customer, your data will not be automatically deleted. Upon your request, we will delete all relevant user data from active systems within 30 days, subject to backup retention schedules and any legal holds. Once the retention period expires, your rights to access, erase, rectify, and port data cannot be enforced.
# Detailed information on the processing of Personal Data
Personal Data is collected for the following purposes and using the following services:
## Handling payments
We process payments through external providers. We do not handle payment information directly but receive notifications about payment status.
**Stripe (Stripe Inc.)**
Stripe is a payment service provided by Stripe Inc.
Personal Data processed: various types of Data as specified in the privacy policy of the service.
Place of processing: United States – [Privacy Policy](https://stripe.com/privacy).
## Registration and authentication
By registering or authenticating, you allow our Application to identify you and give you access to our services.
**PropelAuth**
PropelAuth is a registration and authentication service provided by PropelAuth Inc. and is connected to the PropelAuth network.
Personal Data processed: various types of Data as specified in the privacy policy of the service.
Place of processing: United States – [Privacy Policy](https://propelauth.com/privacy).
**Google OAuth**
Google OAuth is a registration and authentication service provided by Google Inc. and is connected to the Google network.
Personal Data processed: various types of Data as specified in the privacy policy of the service.
Place of processing: United States – [Privacy Policy](https://policies.google.com/privacy).
**GitHub OAuth**
GitHub OAuth is a registration and authentication service provided by GitHub Inc. and is connected to the GitHub network.
Personal Data processed: various types of Data as specified in the privacy policy of the service.
Place of processing: United States – [Privacy Policy](https://docs.github.com/en/site-policy/privacy-policies/github-privacy-statement).
## Traffic optimization and distribution
This type of service allows us to distribute content using servers located across different countries and to optimize performance by filtering traffic.
**Cloudflare (Cloudflare Inc.)**
Cloudflare is a traffic optimization and distribution service that filters traffic between our Application and your browser.
Personal Data processed: various types of Data as specified in the privacy policy of the service.
Place of processing: United States – [Privacy Policy](https://www.cloudflare.com/privacypolicy/).
## User database management
This type of service allows us to build and manage user databases and track user activities.
**PlanetScale**
PlanetScale is a database-as-a-service platform that provides database, authentication, and API functionalities.
Personal Data processed: email address, usage data, and various types of data as specified in the privacy policy of the service.
Place of processing: United States – [Privacy Policy](https://planetscale.com/privacy).
# The Rights of Users
You have the following rights regarding your Personal Data:
* **Withdraw Consent:** You can withdraw your consent to data processing at any time.
* **Object to Processing:** You may object to data processing based on legitimate interests.
* **Access Data:** You can request information about your data and obtain a copy.
* **Rectify Data:** You have the right to correct inaccurate or incomplete Personal Data.
* **Restrict Processing:** You can limit the processing of your data under certain conditions.
* **Erase Data:** You may request the deletion of your Personal Data.
* **Lodge a Complaint:** You have the right to file a complaint with your competent data protection authority.
To exercise any of these rights, contact us at [help@stably.ai](mailto:help@stably.ai).
# Additional Information
## How “Do Not Track” requests are handled
Our Service does not currently respond to "Do Not Track" signals. To determine whether any of the third-party services it uses honor the “Do Not Track” requests, please read their privacy policies.
## Changes to this privacy policy
We reserve the right to modify this Privacy Policy at any time. We will notify you of material changes via email or by posting a notice on this page. Your continued use of the Service after changes constitutes your agreement to the new terms.
## Legal action
Your Personal Data may be used for legal purposes by us in Court or in stages leading to possible legal action arising from improper use of this Application or related Services. You declare to be aware that we may be required to reveal personal data upon request of public authorities.
## System logs and maintenance
For operation and maintenance purposes, this Application and any third-party services may collect files that record interaction with this Application (System logs) or use other Personal Data (such as the IP Address) for this purpose.
# Definitions and Legal References
* **Personal Data (or Data):** Any information that directly, indirectly, or in connection with other information — including a personal identification number — allows for the identification or identifiability of a natural person.
* **Usage Data:** Information collected automatically through this Application (or third-party services employed in this Application), which can include: the IP addresses or domain names of the computers utilized by the Users who use this Application, the URI addresses (Uniform Resource Identifier), the time of the request, the method utilized to submit the request to the server, the size of the file received in response, the numerical code indicating the status of the server's answer (successful outcome, error, etc.), the country of origin, the features of the browser and the operating system utilized by the User, the various time details per visit (e.g., the time spent on each page within the Application), and the details about the path followed within the Application with special reference to the sequence of pages visited, and other parameters about the device operating system and/or the User's IT environment.
* **User:** The individual using this Application who, unless otherwise specified, coincides with the Data Subject.
* **Data Subject:** The natural person to whom the Personal Data refers.
* **Data Processor (or Data Supervisor):** The natural or legal person, public authority, agency or other body which processes Personal Data on behalf of the Controller, as described in this privacy policy.
* **Data Controller (or Owner):** The natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of Personal Data, including the security measures concerning the operation and use of this Application. The Data Controller, unless otherwise specified, is the Owner of this Application.
* **This Application:** The platform provided by Stably AI through which Users sign in with their accounts, create and run test suites.
* **Service:** The service provided by this Application as described in the relative terms (if available) and on this site/application.
* **European Union (or EU):** Unless otherwise specified, all references made within this document to the European Union include all current member states to the European Union and the European Economic Area.
This privacy policy relates solely to this Application, if not stated otherwise within this document.
# Terms of Service
Source: https://docs.stably.ai/about/terms-of-service
Stably AI's terms of service agreement outlining user responsibilities, service usage, and legal terms for our AI-powered testing platform.
Last Updated: July 25, 2025
# Introduction
By using Stably AI (the “Service”), a product of Lovecast Inc. d/b/a Stably AI. (“Stably,” “we,” “us,” “our”), you are legally bound to the terms of this agreement.
# Definitions
## User Content (Customer Content)
“User Content” means any data, information, files, code, configurations, documents, text, graphics, screenshots, videos, logs, run artifacts, or session recordings you upload, or other materials that you or your systems submit to, transmit through, store on, or otherwise make available via the Service, including content relayed from integrated developer tools (e.g., source control) that you connect to the Service.
## Service
The website at [https://stably.ai](https://stably.ai) and subdomains; our APIs, SDKs/CLIs; and the Stably web application (e.g., app.stably.ai).
## Agreement, Terms, Terms of Service
This document titled “Terms of Service.”
## You, User, Customer
Any individual or entity that accepts these Terms by using or intending to use the Service. If you accept on behalf of an entity, you represent that you have authority to bind that entity and its affiliates and users who access the Service through your account.
## Website
The [https://stably.ai](https://stably.ai) website and any valid subdomains.
## Privacy Policy
The privacy policy located at [https://docs.stably.ai/about/privacy-policy](https://docs.stably.ai/about/privacy-policy).
# Acceptance of Terms
By using the Service in any form, you agree to be bound by these Terms and the policies referenced or linked in these Terms, including our Privacy Policy, Acceptable Use Policy, Data Processing Addendum (where applicable), and any Billing & Usage terms on your order or pricing page. If you do not agree, do not use the Service.
# Fair Usage & Unauthorized Use Policy
We reserve the right to withdraw, suspend, or terminate access for any account or end user for improper use. Examples include (without limitation):
* **Fair usage.** Plans carry resource limits (e.g., API calls, test runs, compute, storage, bandwidth, and concurrency). We may throttle, suspend, or terminate for excessive or disruptive consumption that impacts platform stability or exceeds your plan’s limits.
* **Non‑testing compute.** Use for cryptocurrency mining or other compute‑intensive tasks unrelated to application testing.
* **Suspicious payments.** Use of payment instruments flagged as suspicious or fraudulent by third parties.
* **Resale/sublicense.** Selling, reselling, renting, or sharing Service resources or API keys outside of your account.
* **Illegal or harmful activity.** Any activity unlawful in your jurisdiction, the jurisdiction of our server providers, or the United States; attempts to attack or disrupt networks; DDoS; malicious code; or scanning/attacking systems you do not own or lack authorization to test.
* **Abuse and prohibited content.** Distributing malware; hate speech; or discriminatory content.
* **Reverse engineering/competitive misuse.** Reverse engineering the Service or attempting to access Stably’s source code; benchmarking for competitive purposes in violation of these Terms; bypassing metering or rate limits.
We may investigate suspected misuse while honoring our Privacy Policy. We may update these Terms from time to time; continued use constitutes acceptance of any changes.
# Accounts
To use the Service, you must create an account. During setup, you may connect developer tools (e.g., source control) and add a valid payment method (such as a credit card processed by Stripe). For larger contracts, you may request invoicing. All card payments are processed by Stripe under Stripe’s terms, which are separate from your relationship with Stably. By providing payment information, you authorize us to charge your payment method for usage fees and agree to pay invoices per the stated terms.
We may cancel or revoke an account without prior discussion if necessary for security, fraud prevention, or abuse of the Service; this does not affect your separate relationship with Stripe.
# Billing & Usage
## Plans & meters
Your plan includes stated allowances (e.g., test runs, API calls, storage). Unused allowances typically expire each billing cycle unless otherwise stated.
## Overages
Excess usage is billed at the rates in your order or pricing page. Our metering is the system of record and corresponding usage appears in the dashboard/APIs.
## Evaluations / Trials
Unless otherwise agreed in writing, evaluations are AS IS and exclude SLA credits/indemnities. Confidentiality, security, privacy, and IP terms still apply.
# Account & Data Deletion (On Request)
We do not proactively delete User Content. If you want User Content deleted, email [help@stably.ai](mailto:help@stably.ai) (or submit via your account owner) with a verified request. We will delete User Content from active production systems within \[30] days of verification, subject to routine backup retention and any legal holds or obligations. Backup copies are purged on a rolling schedule and may persist until overwritten.
# Security
We maintain administrative, physical, and technical safeguards consistent with industry practices (the “Security Measures”). Upon request and subject to reasonable confidentiality, we can share a security overview and a recent penetration‑test summary. We will notify you of any confirmed unauthorized access to your personal data or unauthorized use of the Service that we become aware of without undue delay and target 72 hours where feasible.
# Service Availability
We employ industry‑standard practices to maximize uptime. While we strive for uninterrupted access, outages may occur (including upstream provider outages). Current or historical incidents may be posted on our status page (if available). Any service credits or remedies for availability are described in an applicable SLA or Order Form.
# User Content; License; AI/ML Use
## Ownership
You retain all right, title, and interest in User Content.
## License to operate
You grant Stably a non‑exclusive, limited license to use, host, store, reproduce, process, transmit, and display User Content solely to provide, maintain, protect, and improve the Service (e.g., execute tests, generate artifacts and reports, diagnostics, support).
## Model training (default) & opt‑out
By default, Stably may use User Content to train, fine‑tune, or otherwise improve Stably‑controlled AI/ML models that power testing accuracy and reliability. You may opt out by signing Stably’s Data Usage Opt‑Out Addendum, after which Stably will not use your User Content for model training (but may continue processing it to provide the Service and to generate de‑identified telemetry).
## De‑identified metrics
We may generate and use de‑identified, aggregated telemetry and reliability statistics that do not identify you or expose your specific content.
## Data classification & regulatory compliance (your responsibility).
The Service is a general‑purpose developer tool. You are solely responsible for determining whether the Service is appropriate for your content and for complying with all laws, policies, and contractual obligations applicable to your data (including privacy, secrecy, data‑localization, sector‑specific, or export rules). You represent that you have all necessary rights and consents to submit User Content and configure integrations. Stably does not monitor or classify your content and has no obligation to do so
# Copyright & Intellectual Property
## 1. Ownership of Service and IP
Stably retains all rights, title, and interest in and to the Service and related intellectual property (including copyrights, trademarks, trade secrets, and other proprietary rights). No rights are granted by implication.
## 2. Limited license to use User Content
You grant the license described in User Content; License; AI/ML Use above solely to operate and improve the Service. This license does not permit Stably to publicly disclose your code or proprietary content, except as necessary to provide the Service or as required by law.
## 3. Protection of User Content
We implement industry‑standard Security Measures to protect your User Content from unauthorized access, disclosure, alteration, or destruction. No method of transmission or storage is 100% secure, and we do not guarantee absolute security.
## 4. Third‑party trademarks
Other product and company names mentioned in the Service may be trademarks of their respective owners. No license to such trademarks is granted.
# Third‑Party Services
You acknowledge that the Service relies on third‑party providers (e.g., cloud infrastructure, email, analytics, payment processing) and integrations you enable (e.g., source control). We are not responsible for interruptions caused by such third parties.
# Warranty Disclaimer
THE SERVICE IS PROVIDED “AS IS” AND “AS AVAILABLE.” WE DISCLAIM ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, SECURITY, ACCURACY, AND NON‑INFRINGEMENT. WE DO NOT WARRANT THAT THE SERVICE WILL BE UNINTERRUPTED OR ERROR‑FREE. SOME JURISDICTIONS DO NOT ALLOW CERTAIN DISCLAIMERS; THOSE LIMITATIONS MAY NOT APPLY TO YOU.
# Limitation of Liability
TO THE FULLEST EXTENT PERMITTED BY LAW, STABLY AND ITS AFFILIATES, OFFICERS, DIRECTORS, EMPLOYEES, LICENSORS, AND PARTNERS WILL NOT BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, EXEMPLARY, OR PUNITIVE DAMAGES, OR ANY LOSS OF PROFITS, DATA, USE, GOODWILL, OR OTHER INTANGIBLE LOSSES, ARISING OUT OF OR RELATED TO: (a) YOUR USE OR INABILITY TO USE THE SERVICE; (b) USER CONTENT; (c) THE SERVICE OR ANY UNDERLYING SOFTWARE/SYSTEMS; OR (d) INTERACTIONS WITH OTHER USERS OR THIRD PARTIES—EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
EXCEPT FOR (i) YOUR PAYMENT OBLIGATIONS, OR (ii) YOUR OR OUR CONFIDENTIALITY/IP INFRINGEMENT OBLIGATIONS, EACH PARTY’S AGGREGATE LIABILITY UNDER THESE TERMS WILL NOT EXCEED THE AMOUNTS PAID OR PAYABLE BY YOU TO STABLY FOR THE SERVICE IN THE 12 MONTHS BEFORE THE EVENT GIVING RISE TO THE CLAIM. SOME JURISDICTIONS DO NOT ALLOW LIMITATIONS; THOSE LIMITATIONS MAY NOT APPLY TO YOU.
# Email May Not Be Used to Provide Legal Notice
Communications made through the Service’s email and messaging system will not constitute legal notice to the Site, the Service, or any of its officers, employees, agents or representatives in any situation where legal notice is required by contract or any law or regulation.
# Updating Terms of Service
We may update, modify, or replace these Terms and may suspend or discontinue the Service or impose restrictions on parts of the Website or features. Material changes will be communicated via email or in‑product notice. Your continued use after the effective date constitutes acceptance of the updated Terms.
# General Provisions
If any provision of these Terms is found invalid or unenforceable, it will be modified to the minimum extent necessary to be enforceable, and the remainder will continue in full force. Our failure to enforce a provision is not a waiver. Our rights survive any transfer or termination. Claims must be filed within one (1) year after the cause of action accrues, or they are permanently barred, to the extent permitted by law. We may assign these Terms at any time to an affiliate or in connection with a merger, acquisition, or sale of assets. You may not assign or delegate any rights or obligations without our prior written consent; any attempted assignment is void. These Terms and the referenced policies constitute the entire agreement between you and Stably regarding the Service and supersede all prior or contemporaneous agreements or communications.
Governing Law & Venue: Delaware law; venue in state or federal courts located in San Francisco County, California.
# Contact Us
Questions about these Terms? Contact us at [help@stably.ai](mailto:help@stably.ai).
# Introduction
Source: https://docs.stably.ai/api-reference/introduction
Understand general concepts, response codes, and authentication strategies for the Stably API.
## Authentication
To authenticate, you need to add an `Authorization` header with your API key using the Bearer scheme.
```
Authorization: Bearer $YOUR-API-KEY
```
You can obtain your API key from the [API Key Dashboard](https://app.stably.ai/settings?tab=api-key).
If you do not see the "API Keys" setting, please ensure you have developer or owner permissions for your org.
## Run Tests
The Stably API allows you to programmatically trigger and monitor test suite runs. A typical workflow looks like:
Use `POST /v1/testSuite/{testSuiteId}/run` to start a test suite execution.
Use `GET /v1/testSuiteRun/{testSuiteRunId}/status` to check if the run is still in progress.
Once complete, use `GET /v1/testSuiteRun/{testSuiteRunId}/result` to get detailed test results.
Use `POST /v1/projects/{projectId}/runs` to start an agent-based test execution.
Use `GET /v1/projects/{projectId}/runs/{runId}` to check if the run is still in progress (read the `status`).
If the status indicates the run has completed, you can read the results from the `results` field.
## Quick Start Example
Below is a complete example of triggering a test run and polling for results using the Agents API.
### 1. Trigger a run
```bash theme={null}
curl -X POST "https://api.stably.ai/v1/projects/$STABLY_PROJECT_ID/runs" \
-H "Authorization: Bearer $STABLY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"playwrightProjectName": "my-project"
}'
```
Response:
```json theme={null}
{
"runId": "abc123"
}
```
You can optionally pass `branch` to associate the run with a specific git branch, `environment` to select a named environment, and `playwrightProjectName` to target a specific Playwright project.
### 2. Poll for status and results
```bash theme={null}
curl "https://api.stably.ai/v1/projects/$STABLY_PROJECT_ID/runs/$RUN_ID" \
-H "Authorization: Bearer $STABLY_API_KEY"
```
While the run is in progress:
```json theme={null}
{
"status": "RUNNING",
"startedAt": "2025-09-15T12:00:00.000Z",
"branchName": "main"
}
```
Once the run completes, the response includes `results`:
```json theme={null}
{
"status": "PASSED",
"startedAt": "2025-09-15T12:00:00.000Z",
"finishedAt": "2025-09-15T12:02:30.000Z",
"branchName": "main",
"results": {
"testCases": [
{
"title": "user can log in",
"status": "PASSED",
"durationMs": 12340
},
{
"title": "user can view dashboard",
"status": "PASSED",
"durationMs": 8760
}
]
}
}
```
Possible run statuses: `QUEUED`, `RUNNING`, `PASSED`, `FAILED`, `TIMEDOUT`, `CANCELLED`, `INTERRUPTED`.
### 3. Cancel a run
```bash theme={null}
curl -X POST "https://api.stably.ai/v1/projects/$STABLY_PROJECT_ID/runs/cancel" \
-H "Authorization: Bearer $STABLY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"where": { "runId": "abc123" }
}'
```
# Cancel a test run
Source: https://docs.stably.ai/api-reference/runner/cancel-a-test-run
https://api.stably.ai/openapi.json post /v1/projects/{projectId}/runs/cancel
Cancel a test run. The run must be in QUEUED or RUNNING status. If the run has a container, the container will be stopped.
# Get test run status and results
Source: https://docs.stably.ai/api-reference/runner/get-test-run-status-and-results
https://api.stably.ai/openapi.json get /v1/projects/{projectId}/runs/{runId}
Get the status and results of a test run
# Start a test run
Source: https://docs.stably.ai/api-reference/runner/start-a-test-run
https://api.stably.ai/openapi.json post /v1/projects/{projectId}/runs
Start a test run
# Changelog
Source: https://docs.stably.ai/changelog
Product updates and announcements
Autofix
* Autofix now generates markdown reports with code references and actionable diagnosis
* Autofix tab visible (disabled) while test suites are still running
Web Editor
* Edit history tab with diff viewer in test explorer
* Pass/fail status filter with multi-select on run history
* PR title prefix setting (configurable per-project)
* Usage breakdown pie chart on billing page
* Safari compatibility fixes
Autofix
* Skip tests repeatedly fixed for the same root cause, saving AI credits
Web Editor
* Run history accessible directly inside agent sessions
* TypeScript diagnostics in the code editor (LSP-backed)
* Binary file viewer with VS Code-style UX
* Browser notifications for agent session events
Reporter
* Failure Types filter on run history
CLI
* New commands: `stably verify`, `stably runs list/view`, and `stably env`
* `stably init` revamped with interactive prompts
* Bracketed paste mode for multi-line input
Web Editor
* Agent 2.0 is now the default for new organizations
* AI-generated no-code view: plain-English description of each test
* Revamped Test Explorer with saved views, list/tree browsing, and virtualization
* Redesigned sidebar navigation and environment variables page (key-value table, bulk import, `.env` download)
* Members settings tab and multi-project support per organization
Cloud Runner
* GitLab support for Bring Your Own Repo (BYOR)
* GitHub PR authorship: PRs can be authored by the user instead of the Stably bot
* Configurable usage limits
* Autofix support for scheduled test runs
Autofix
* Interrupt fix agent mid-turn from the web UI
* AI-powered error summaries for faster triage
Web Editor
* Test Case Management System (TCMS): full rewrite with multi-route workspace, draft workflow, and tag registry
* User environments for managing per-environment variables
* "Diagnosis & Fix" column on test run results table
* Session commits squashed for cleaner PRs
Reporter
* Redesigned Playwright run history with pass rate and flaky rate cards
* Trace download endpoint on the public API
Autofix
* AI-generated PR titles and descriptions for autoheal PRs
* Two-phase `stably fix`: parallel fix-workers followed by sequential debug-workers
* Stream autoheal progress in real time to the Autofix tab
* Trace proof capture: autoheal includes Playwright traces as evidence
Web Editor
* Pause/cancel and pin-to-resume during AI playback recording
* Send messages from the web UI to running CLI sessions
* V1 to V2 migration flow for Classic tests to Agent 2.0
CLI
* `stably.yaml` config file support for agent settings
* Claude Opus 4.6 added as a supported model
Web Editor
* AskUserQuestion: the agent can now ask clarifying questions during test creation
* Auto-start autoheal when clicking "Fix with AI"
* @file mention autocomplete in agent chat
* File upload and image viewer for test generation context
* In-app API key management
CLI
* New `stably build` command: prompt-to-test compiler
* Auto-detect last local test run in `stably fix`
SDK
* Email SDK (`@stablyai/email`): AI-powered email testing for OTP flows, magic links, and verification
Cloud Runner
* Per-org proxy support for enterprise environments
Web Editor
* Agent 2.0 now available for all organizations
* Record at Cursor: insert recorded actions at a specific point in your test
* Display test error details on hover for failed badges
CLI
* Non-TTY environment support for CI pipelines
Cloud Runner
* Browser minute pricing reduced from $0.015 to $0.01/min
Web Editor
* Resolve merge conflicts with AI directly in the agent
* Git status popover for inline branch tracking
* "Fix with AI" button on Playwright suite run pages
* Project selection modal for multi-project Playwright configs
* File explorer: upload folders, search, and delete files
* Support for `.test.ts` file extension
* Ability to change GitHub App installation from settings
Web Editor
* Agent 2.0 Chat UI improvements with markdown rendering
* Git tab for repository connection workflow
* Scheduler calendar view: visualize your test schedules at a glance
CLI
* Auto-login for `stably init` command
Web Editor
* Bring Your Own Repo (BYOR): connect your existing repo and let the AI agent write tests against your codebase
* Self-review subagent: the codegen agent now reviews its own code before publishing
* Web Editor 2.0: redesigned Explorer panel with right sidebar and horizontal scrolling file tabs
* Dark mode support for the browser preview
Reporter
* Improved Playwright trace operation grouping for clearer test results
Web Editor
* The PR testing agent now provides more critical UX feedback
* An easier way to review what exploratory PR testing did and why it did it
* Much easier to add a reusable component
* Move the recording cursor even when the recording is paused
Cloud Runner
* Test suite run API now allows selecting environment variable scopes and adding notes
Web Editor
* Assertion Agent uses full test context for smarter checks
* Adaptive scrolling for the Action Agent
* Larger operation screenshot preview window
Cloud Runner
* Support JSON values in override variables
Autofix
* Improved AI auto-heal accuracy
* More resilient model fallback behavior
Web Editor
* A snappier way to edit AI Act and AI Assert
Initial changelog setup.
# Autofix Cost Controls
Source: https://docs.stably.ai/core-configuration/autofix-cost-controls
Control how much the autofix agent spends per session using budget caps, turn limits, and smart skip rules in stably.yaml.
## Overview
Every time Stably fixes failing tests — whether triggered automatically after a [scheduled cloud run](/run-tests/autofix) or manually via [`stably fix`](/stably-cli/fix) — it starts a **fix session**. Each session uses AI credits, so it's important to set the right guardrails.
The `agent.fix` section in `stably.yaml` gives you fine-grained control over how much a fix session can spend. These settings apply everywhere the fix agent runs: cloud autofix, CLI `stably fix`, and dashboard-triggered fixes.
***
## Key Concepts
### What Is a Fix Session?
A fix session is a single end-to-end run of the fix agent against a set of failing tests from one test run. For example:
* A scheduled run fails with 5 tests — autofix starts **one fix session** to address all 5.
* You run `stably fix ` locally — that's **one fix session**.
Each fix session has its own budget and turn limits. When the session ends (either by completing all fixes or hitting a limit), no more credits are consumed.
### What Is a Turn?
A **turn** is one cycle of the AI agent reasoning and taking an action — for example, reading a test file, editing code, or running a test to verify a fix. A single failing test might take anywhere from 5 to 30+ turns to diagnose and fix, depending on the complexity of the failure.
Turns are the primary driver of cost. Each turn involves an LLM call, and more complex fixes require more turns.
### Budget vs. Turns — Which Limit Is Hit First?
Both `maxBudgetUsd` and `maxTurnsPerIssue` act as independent safety caps. The session stops when **either** limit is reached — whichever comes first.
* **Simple failures** (e.g., a selector changed) typically take fewer turns and cost less — the turn limit is rarely hit.
* **Complex failures** (e.g., a multi-step flow broke) take many turns — the budget limit may kick in before the turn limit.
In practice, `maxBudgetUsd` is the more important control for managing overall spend, while `maxTurnsPerIssue` prevents the agent from spinning on a single difficult issue.
***
## Where to Configure
Add the `agent.fix` section to your `stably.yaml` in the project root:
```yaml stably.yaml theme={null}
agent:
fix:
maxBudgetUsd: 30
maxTurnsPerIssue: 20
maxParallelWorkers: 2
skipAfterConsecutiveUnfixed: 2
```
Navigate to your project's [**AI & Cloud**](https://app.stably.ai/project/cloud) settings and select **Agent Fix** to configure the same options from the UI.
![Agent Fix configuration UI showing max turns, parallel workers, budget, and skip settings]()
### Options
| Option | Type | Default | Description |
| ----------------------------- | ------ | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `maxBudgetUsd` | number | 50 | Maximum LLM spend in USD for the entire fix session. The session stops when this limit is reached, even if unfixed tests remain. |
| `maxTurnsPerIssue` | number | 50 | Maximum AI turns the agent can take per individual failing test. Prevents the agent from spending too long on one stubborn failure. |
| `maxParallelWorkers` | number | 3 | Number of failing tests to fix in parallel. Higher values fix faster but use more concurrent resources. |
| `skipAfterConsecutiveUnfixed` | number | 2 | Automatically skip tests that have failed to fix this many times in a row across consecutive fix sessions. This is key for preventing repeated spending on the same broken test. |
***
## Preventing Runaway Costs
If your tests hit an external issue, every test might fail for the same root cause. Without guardrails, autofix will attempt to fix each one — burning credits on an issue it can't solve by changing test code.
Here's how to protect against this:
### 1. Set a session budget
The most direct control. Set `maxBudgetUsd` to the maximum you're comfortable spending on a single fix session:
```yaml stably.yaml theme={null}
agent:
fix:
maxBudgetUsd: 20 # Stop after $20 regardless of progress
```
### 2. Skip persistently broken tests
`skipAfterConsecutiveUnfixed` prevents the agent from retrying the same failing test across multiple sessions. If a test fails to fix 2 times in a row (the default), it's automatically skipped in future sessions until it passes on its own.
```yaml stably.yaml theme={null}
agent:
fix:
skipAfterConsecutiveUnfixed: 1 # Skip after just 1 failed attempt
```
This is especially valuable for scheduled runs. If an external dependency goes down and causes failures across multiple scheduled runs, the skip rule kicks in after the first failed fix attempt — so only the first session spends credits on those tests.
### 3. Limit turns per issue
If the agent can't fix a test within a reasonable number of turns, it's likely a complex issue that needs human attention:
```yaml stably.yaml theme={null}
agent:
fix:
maxTurnsPerIssue: 15 # Move on after 15 turns per test
```
### 4. Combine with schedule-level autofix control
You don't have to enable autofix on every schedule. Use it selectively — `autofix` is set per schedule, and schedules without it default to no autofix:
```yaml stably.yaml theme={null}
schedules:
critical-smoke:
cron: "0 9 * * *"
stablyTestArgs: "--project smoke"
autofix: true # Only autofix the smoke suite
full-regression:
cron: "0 2 * * *"
stablyTestArgs: "--project regression"
# autofix not set — fix manually when needed
```
***
## Recommended Settings by Use Case
| Scenario | `maxBudgetUsd` | `maxTurnsPerIssue` | `skipAfterConsecutiveUnfixed` |
| ---------------------------------- | -------------- | ------------------ | ----------------------------- |
| **Tight budget, scheduled runs** | 10–20 | 15 | 1 |
| **Standard team usage** | 50 (default) | 50 (default) | 2 (default) |
| **Large suite, aggressive fixing** | 100+ | 50 | 3 |
***
## Full Example
```yaml stably.yaml theme={null}
# Fix agent configuration
agent:
fix:
maxBudgetUsd: 25
maxTurnsPerIssue: 20
maxParallelWorkers: 2
skipAfterConsecutiveUnfixed: 1
# Schedule-level overrides
schedules:
nightly-regression:
cron: "0 2 * * *"
stablyTestArgs: "--project regression"
autofix: true
quick-smoke:
cron: "*/30 9-17 * * 1-5"
stablyTestArgs: "--project smoke"
# autofix not set — too frequent to autofix every run
```
## Next Steps
How autofix works end-to-end on cloud runs
Run fix locally or in CI with the CLI
Configure when your tests run automatically
Write custom fix rules in a markdown file
# Mode-Specific Rules
Source: https://docs.stably.ai/core-configuration/mode-rules
Provide mode-specific rules to the Stably AI agent with STABLY-CREATE.md and STABLY-FIX.md.
## Introduction
In addition to [`STABLY.md`](/core-configuration/stably-md) (which applies to **all** agent modes), you can create mode-specific rules files that are only loaded for a particular command. This is a great way to customize the agent to match your team's workflow — your conventions, your folder structure, your assertion style — while still getting the full benefit of Stably's built-in browser management, test debugging tooling (e.g. Playwright Trace Viewer), and intelligent test generation under the hood.
| File | Loaded by | Purpose |
| ------------------ | --------------- | -------------------------------------------------------------------------------------- |
| `STABLY-CREATE.md` | `stably create` | How tests should be **written** — file structure, patterns, assertions |
| `STABLY-FIX.md` | `stably fix` | How tests should be **repaired** — selector strategy, timeout handling, test integrity |
If a rule applies to **all** modes (create, fix, init), put it in [`STABLY.md`](/core-configuration/stably-md) instead.
***
## Quick Start
Add a `STABLY-CREATE.md` file to your project root:
```markdown STABLY-CREATE.md theme={null}
# Test Generation Rules
- Prefer `getByRole` and `getByText` locators; fall back to `data-testid`
- Use Playwright fixtures for shared setup/teardown
- Include both positive and negative test cases for form validations
- Use `test.describe` blocks to group related scenarios
```
The rules are automatically loaded and applied:
```bash theme={null}
stably create "test the checkout flow"
```
You'll see a confirmation in the CLI output:
```
✓ STABLY-CREATE.md loaded (320 chars)
```
Add a `STABLY-FIX.md` file to your project root:
```markdown STABLY-FIX.md theme={null}
# Fix Rules
- Prefer user-facing locators (`getByRole`, `getByText`) when fixing locator failures; fall back to `data-testid`
- Always add a comment explaining why a selector was changed
- Never delete or skip a failing test — fix it or flag it for review
- When fixing timeouts, increase wait time as a last resort — look for root causes first
```
The rules are automatically loaded and applied:
```bash theme={null}
stably fix
```
You'll see a confirmation in the CLI output:
```
✓ STABLY-FIX.md loaded (280 chars)
```
***
## File Location and Naming
Both files follow the same conventions:
| Requirement | Details |
| ------------------ | ------------------------------------------------------ |
| **Location** | Project root directory (same level as `package.json`) |
| **Filenames** | Must be uppercase: `STABLY-CREATE.md`, `STABLY-FIX.md` |
| **Source control** | Should be committed to git |
Filenames **must** be uppercase. macOS is case-insensitive, so `stably-create.md` will work locally — but Linux (used in CI/ECS) is case-sensitive and will silently ignore the wrong case.
Commit these files to your repository so your entire team shares the same conventions. This is especially important when commands run in CI pipelines or are invoked by background agents.
***
## File Format
Both files use **freeform markdown**. There's no required structure — write whatever instructions you want the agent to follow. The content is appended directly to the agent's system prompt.
**Maximum length:** 10,000 characters (\~2,500 tokens) per file. Files exceeding this limit are automatically truncated with a CLI warning.
***
## When to Use Each File
### STABLY-CREATE.md
Use for rules **specific to test generation**:
* Test file naming conventions and folder structure
* Test patterns and frameworks (Page Object Model, fixtures, etc.)
* Assertion strategies and test data patterns
* Tags and annotations (`@smoke`, `@regression`)
* Which Stably SDK methods to prefer (`agent.act()`, `aiAssert()`, etc.)
### STABLY-FIX.md
Use for rules **specific to test fixing**:
* Selector replacement strategies (e.g., prefer `data-testid` over CSS classes)
* How to handle flaky tests vs genuinely broken tests
* Timeout and retry policies
* Which files or directories the agent should or shouldn't modify
* Comment and documentation requirements for changes
* When to flag a test for human review instead of auto-fixing
***
## Examples
```markdown STABLY-CREATE.md theme={null}
# Test Generation Rules
## File Structure
- Place all tests in the `tests/` directory
- Name test files using the pattern `.spec.ts`
- Group related tests with `test.describe` blocks
## Test Patterns
- Use Playwright fixtures for shared setup/teardown; Page Object Model is acceptable for large suites
- Prefer user-facing locators: `getByRole`, `getByText`, `getByLabel`
- Fall back to `data-testid` when no accessible locator is available
- Never use CSS class selectors
## Assertions
- Include both positive and negative test cases for form validations
- Use `aiAssert()` for visual assertions instead of pixel comparisons
- Use `agent.act()` for complex multi-step UI interactions (e.g., drag-and-drop)
## Test Data
- Create fresh test data in each test — never modify shared records
- Use environment variables for credentials (never hard-code)
- Use fixtures for setup and teardown — `afterEach` won't run if a test crashes
## Tags
- Add `@smoke` annotation to critical-path tests
- Add `@regression` annotation to edge-case tests
```
```markdown STABLY-FIX.md theme={null}
# Fix Rules
## Selector Strategy
- When a selector breaks, replace it with a `data-testid` attribute
- If the element already has a `data-testid`, check if the value changed
- Prefer user-facing locators (`getByRole`, `getByText`) over `data-testid` — never use CSS class selectors
- Add a comment above changed selectors explaining the fix
## Timeout Handling
- Do not increase timeout values as the first fix attempt
- Investigate whether the page load changed or a network call is slower
- If a `waitForSelector` times out, check if the element was renamed or moved
## Test Integrity
- Never delete a failing test — fix it or mark it with `test.fixme()` for review
- Never change assertion values to make a test pass — fix the root cause
- If a test is flaky (passes on retry), investigate the root cause (race condition, shared state, missing auto-wait) before adding retries
## Boundaries
- Do not modify files outside the `tests/` directory
- Do not modify `playwright.config.ts` unless the failure is config-related
- Do not install new npm packages without flagging for review
```
***
## STABLY-FIX.md vs `agent.fix.rules` in stably.yaml
You can also define fix rules in `stably.yaml` under `agent.fix.rules`:
```yaml stably.yaml theme={null}
agent:
fix:
rules: |
Prefer data-testid selectors over CSS selectors.
Always add comments explaining selector changes.
```
When both `STABLY-FIX.md` and `agent.fix.rules` are present, they are concatenated — `agent.fix.rules` is loaded first, then `STABLY-FIX.md`. They are complementary, not competing.
| Mechanism | Best for |
| ----------------- | ----------------------------------------------------------------------- |
| `STABLY-FIX.md` | Longer, detailed fix rules in freeform markdown — easy to review in PRs |
| `agent.fix.rules` | Short, structured rules that live alongside other `stably.yaml` config |
***
## Relationship to Other Config Files
| File | Scope | Modes | Description |
| -------------------------------------------- | --------------------- | ----------------------------- | ------------------------------------------------------------------------------ |
| [`STABLY.md`](/core-configuration/stably-md) | Project-wide rules | All modes (create, fix, init) | General instructions — project context, selector strategy, do's and don'ts |
| `STABLY-CREATE.md` | Test generation rules | `create` only | Test-specific conventions — file naming, folder structure, test patterns |
| `STABLY-FIX.md` | Fix rules | `fix` only | Fix-specific conventions — selector strategy, timeout handling, test integrity |
| `stably.yaml` → `agent.fix.rules` | Fix agent rules | `fix` | Structured config for the fix agent (max turns, parallel tests, rules) |
When multiple files are present, they are concatenated in this order:
```
[Base System Prompt]
→ stably.yaml agent.fix.rules
→ STABLY-FIX.md (fix mode only)
→ STABLY-CREATE.md (create mode only)
→ STABLY.md (all modes)
```
Don't duplicate rules across files. Put general project rules in `STABLY.md`, test-generation-specific conventions in `STABLY-CREATE.md`, and fix-specific conventions in `STABLY-FIX.md`.
***
## Best Practices
| Do | Avoid |
| ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| Commit both files to source control so CI and teammates share the same conventions. | Adding them to `.gitignore` — CI agents won't find them. |
| Keep rules focused on their respective mode (generation vs fixing). | Putting general project context here — use `STABLY.md` for that. |
| Use concise, actionable rules the agent can follow. | Writing novel-length instructions that exceed the 10,000 character limit. |
| Use uppercase filenames: `STABLY-CREATE.md`, `STABLY-FIX.md`. | Using lowercase — it will fail silently on Linux CI servers. |
| Review changes in PRs like any other config file. | Duplicating rules that already exist in `STABLY.md` or `stably.yaml`. |
# STABLY.md
Source: https://docs.stably.ai/core-configuration/stably-md
Provide project-level rules and instructions to the Stably AI agent across all modes and surfaces.
## Introduction
`STABLY.md` is a markdown file you place in your project root to give the Stably AI agent project-specific instructions. Rules defined here apply across **all agent modes** (`create`, `fix`, `build`, `init`) and **all surfaces** (local CLI and server-side CI/ECS test generation).
Think of it like `CLAUDE.md` for Claude Code — but for Stably. It's a great way to customize the agent to match your team's workflow — your conventions, your selector strategy, your project context — while still getting the full benefit of Stably's built-in browser management, test debugging tooling (e.g. Playwright Trace Viewer), and intelligent test generation under the hood.
`STABLY.md` is automatically created when you run `stably init` in a new project.
***
## Quick Start
Add a `STABLY.md` file to your project root:
```markdown STABLY.md theme={null}
# Project Rules
- This is a Next.js 14 e-commerce app running on localhost:3000
- Prefer `getByRole` and `getByText` locators; fall back to `data-testid`
- Never hard-code credentials — use environment variables
```
The rules are automatically loaded and applied:
```bash theme={null}
stably create "test the login flow"
stably fix
```
You'll see a confirmation in the CLI output:
```
✓ STABLY.md loaded (245 chars)
```
***
## File Location and Naming
| Requirement | Details |
| ------------------ | ----------------------------------------------------- |
| **Location** | Project root directory (same level as `package.json`) |
| **Filename** | Must be uppercase `STABLY.md` |
| **Source control** | Must be committed to git |
The filename **must** be uppercase `STABLY.md`. macOS is case-insensitive, so `stably.md` will work locally — but Linux (used in CI/ECS) is case-sensitive and will silently ignore the wrong case.
`STABLY.md` must be committed to your repository. If it's `.gitignore`'d, the CLI will still read it locally, but the server-side agent (CI/ECS) won't be able to find it — meaning your rules won't apply during PR-triggered or scheduled test generation.
***
## File Format
`STABLY.md` uses **freeform markdown**. There's no required structure — write whatever instructions you want the agent to follow. The content is appended directly to the agent's system prompt.
**Maximum length:** 10,000 characters (\~2,500 tokens). Files exceeding this limit are automatically truncated with a CLI warning.
***
## Template
When you run `stably init`, the following template is created:
```markdown STABLY.md theme={null}
# STABLY.md — Project Rules for Stably Agent
## Project Context
## Selector Strategy
## Do's and Don'ts
```
***
## Example
Here's a real-world example for a Next.js e-commerce app:
```markdown STABLY.md theme={null}
# Project Rules
## Project Context
This is a Next.js 14 e-commerce application.
- Frontend runs on localhost:3000
- API runs on localhost:4000
- Auth uses NextAuth with Google OAuth
- Database is PostgreSQL with Prisma ORM
## Selector Strategy
- Prefer user-facing locators: `getByRole`, `getByText`, `getByLabel`
- Fall back to `data-testid` when no accessible locator is available
- Never use CSS class selectors — they change frequently
## Do's and Don'ts
- Always create new test data instead of modifying existing records
- Use `agent.act()` for complex multi-step UI interactions (e.g., drag-and-drop)
- Never hard-code credentials — use environment variables from Test Data
- Assert on expected end-state directly — Playwright auto-waits, so avoid explicit spinner/loading waits
- Our app uses a dark theme by default — account for this in visual assertions
```
***
## Relationship to Other Config Files
Stably has three mechanisms for customizing agent behavior. Each serves a different purpose:
| File | Scope | Modes | Description |
| ---------------------------------------------------- | --------------------- | ------------------------------------ | ------------------------------------------------------------------------------ |
| `STABLY.md` | Project-wide rules | All modes (create, fix, build, init) | General instructions — project context, selector strategy, do's and don'ts |
| [`STABLY-CREATE.md`](/core-configuration/mode-rules) | Test generation rules | `create` and `build` only | Test-specific conventions — file naming, folder structure, test patterns |
| [`STABLY-FIX.md`](/core-configuration/mode-rules) | Fix rules | `fix` only | Fix-specific conventions — selector strategy, timeout handling, test integrity |
| `stably.yaml` → `agent.fix.rules` | Fix agent rules | `fix` | Structured config for the fix agent (max turns, parallel tests, rules) |
When multiple files are present, they are concatenated in this order:
```
[Base System Prompt]
→ stably.yaml agent.fix.rules
→ STABLY-FIX.md (fix mode only)
→ STABLY-CREATE.md (create/build modes only)
→ STABLY.md (all modes)
```
Don't duplicate rules across files. Put general project rules in `STABLY.md`, and mode-specific conventions in [`STABLY-CREATE.md` / `STABLY-FIX.md`](/core-configuration/mode-rules).
***
## Best Practices
| Do | Avoid |
| ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------- |
| Commit `STABLY.md` to source control so CI and teammates share the same rules. | Adding `STABLY.md` to `.gitignore` — the server-side agent won't find it. |
| Keep rules concise and actionable. | Writing novel-length instructions that exceed the 10,000 character limit. |
| Use `STABLY.md` for project-wide concerns (context, selectors, conventions). | Duplicating rules that already exist in `STABLY-CREATE.md` or `stably.yaml`. |
| Use the uppercase filename `STABLY.md`. | Using lowercase `stably.md` — it will fail silently on Linux CI servers. |
| Review `STABLY.md` changes in PRs like any other config file. | Putting sensitive information (API keys, passwords) in the file. |
# Autofix for Tests
Source: https://docs.stably.ai/getting-started/autofix-for-tests
Stably Autofix maintains your tests in production — triaging failures, fixing code, and proving every fix works with real browser replays.
* **Accurate triage** — Compares past and current test run replays side-by-side to pinpoint exactly what changed. Digs into network requests, console logs, and DOM state to find the real root cause — especially effective at eliminating flaky tests.
* **Scales to real workloads** — Whether you have 5 failures or 500, Autofix spins up on-demand cloud browsers to validate every fix in parallel. Built for production, not demos.
* **Every fix comes with proof** — No vibe-test fixes. Every code change is re-run in a real browser, producing a full replay report with screenshots, traces, and DOM snapshots before anything reaches your codebase.
## Quick Setup
Run `stably fix` after any test execution:
```bash theme={null}
stably test || stably fix
```
See the [full CLI guide](/stably-cli/fix) for more details on how to collect the fixed files.
Enable the **Autofix** option when you trigger tests — from the UI, on a schedule, or via API.
You can also turn it on by default for scheduled runs in your `stably.yaml`:
```yaml stably.yaml theme={null}
schedules:
my-nightly-run:
cron: "0 0 * * *"
autofix: true
```
That's it. Autofix handles the rest — how fixes are delivered (automatic PR, dashboard diff, or local file changes) depends on the trigger method. See [Ways to Use Autofix](#ways-to-use-autofix) below.
## Ways to Use Autofix
### Automatic (at trigger time)
Enable `autofix: true` as a project default in `stably.yaml`, per-schedule, or pass it when triggering a run via the API or dashboard UI. Autofix runs automatically on Stably Cloud after test failures — no manual step needed.
### On-demand (after any failed run)
After any run completes with failures, the **Autofix tab** on the run details page presents **two options** — choose whichever fits your workflow:
* **Autofix on Cloud** — Click **"Fix with Agent"** to start a cloud agent session. The agent diagnoses failures and generates fixes on Stably infrastructure; at the end you can create a PR (if your [repo is connected](/stably2/bring-your-own-repo)).
* **Auto-heal on your device with CLI** — Copy the ready-to-run `npx stably fix ` command and run it locally. Fixes are applied to your working tree. Review with `git diff`, commit when ready.
This works on any failed run — scheduled, API-triggered, UI-triggered, or CLI-triggered — regardless of whether `autofix` was enabled at trigger time.
![Autofix tab showing Fix with Agent button and CLI command side-by-side]()
### In CI
Add a `stably fix` step after `stably test` in your pipeline. Results are always uploaded to the dashboard, where you can review diffs and create a PR ([repo connected](/stably2/bring-your-own-repo)). Optionally, add git steps to commit/push directly from CI. See the [CI Integration](/stably-cli/fix#ci-integration) section for examples.
### Comparison
| | Automatic (at trigger) | On-demand: Cloud | On-demand: CLI | CLI in CI |
| ------------------------ | ---------------------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------- |
| **Trigger** | `autofix: true` on schedule/API/UI | Click "Fix with Agent" on Autofix tab | Copy `stably fix ` from Autofix tab | `stably test \|\| stably fix` |
| **Runs on** | Stably cloud | Stably cloud | Your machine | Your CI runner |
| **PR creation** | Automatic ([repo connected](/stably2/bring-your-own-repo)) | From dashboard ([repo connected](/stably2/bring-your-own-repo)) | You commit manually | From dashboard ([repo connected](/stably2/bring-your-own-repo)), or git steps in CI |
| **Human trigger needed** | No | Yes (click button) | Yes (run command) | No (wired into pipeline) |
## What Autofix Can Fix
Every fix is validated by re-running the test in a real browser. You get a full replay report with screenshots, traces, and DOM snapshots — so you can see proof that the fix actually works before merging.
Your app changed but the tests weren't updated. Autofix updates selectors, rewrites flows, and fixes timing issues to match your current application.
For example, a checkout flow added a new "Shipping Method" step between address and payment:
**Before:**
```typescript theme={null}
await page.getByRole('button', { name: 'Continue to Payment' }).click();
await page.getByLabel('Card number').fill('4242424242424242');
```
**After Autofix:**
```typescript theme={null}
await page.getByRole('button', { name: 'Continue to Shipping' }).click();
await page.getByLabel('Standard Shipping').check();
await page.getByRole('button', { name: 'Continue to Payment' }).click();
await page.getByLabel('Card number').fill('4242424242424242');
```
The test caught a real bug in your application code. Autofix can fix the application code directly and open a PR — or flag it for your team to investigate.
For example, a discount code applies the percentage to the wrong total:
**Bug in app code (`pricing.ts`):**
```typescript theme={null}
// ❌ Discount applied to subtotal instead of pre-tax total
const discount = subtotal * discountRate;
```
**After Autofix:**
```typescript theme={null}
// ✅ Discount applied to pre-tax total as expected
const discount = preTaxTotal * discountRate;
```
Intermittent failures caused by timing, race conditions, or dynamic content. Autofix adds proper waits, stabilizes selectors, or replaces brittle locators with more resilient approaches.
For example, a test clicks a button before an API response finishes loading:
**Before:**
```typescript theme={null}
await page.goto('/dashboard');
await page.getByRole('button', { name: 'Export' }).click();
```
**After Autofix:**
```typescript theme={null}
await page.goto('/dashboard');
await page.waitForResponse('**/api/dashboard/stats');
await page.getByRole('button', { name: 'Export' }).click();
```
### How Autofix handles failing locators
When a locator fails, Autofix first tries to find the correct updated locator by navigating through your application in a live browser. It inspects the current page state, identifies what changed, and updates the selector in your test to match the new UI.
If you have your [repo connected](/stably2/bring-your-own-repo), Autofix can also update the `data-testid` in your application code alongside the test selector. You can customize this behavior through your [`STABLY.md`](/core-configuration/stably-md) file (e.g., prefer certain selector strategies, naming conventions).
A button was renamed from "Submit" to "Place Order", causing the test to fail.
**Before:**
```typescript theme={null}
await page.getByRole('button', { name: 'Submit' }).click();
```
**After Autofix:**
```typescript theme={null}
await page.getByRole('button', { name: 'Place Order' }).click();
```
If no reliable locator can be found — for example, when the element is highly dynamic or lacks stable attributes — Autofix may replace it with an [AI locator](/stably-sdk/ai-locator) (`page.getLocatorsByAI()`) or an [`agent.act()`](/stably-sdk/ai-agent-execute) call that uses natural language instead of brittle selectors.
A pricing toggle has no stable selector and changes frequently across redesigns.
**Before:**
```typescript theme={null}
await page.locator('.pricing-toggle > div:nth-child(2) > button.active').click();
```
**After Autofix:**
```typescript theme={null}
const [annualToggle] = await page.getLocatorsByAI('annual billing toggle button');
await annualToggle.click();
```
## How It Works
Autofix analyzes all failing tests and groups them by root cause — is it an outdated test, a real bug, or a flaky test? Repeated failures with the same root cause are automatically skipped to save cost.
Each issue gets a targeted fix. Autofix opens a real browser to inspect your live application, understand what changed, and apply the right code changes.
![Autofix diagnosis report showing root cause analysis and applied fixes]()
Every fix is re-run in a real browser to confirm it actually works. You get a full **replay report** with screenshots, traces, and DOM snapshots — proof that the fix is correct before any code is merged.
A summary report is generated with root causes and code diffs. How fixes reach your codebase depends on the trigger method:
* **Cloud runs with [connected repo](/stably2/bring-your-own-repo):** A PR/MR is created automatically. Review and merge when ready — Stably does not push to your repo until you merge.
* **Cloud runs without connected repo:** View diagnosis and code diffs in the dashboard. Apply changes to your codebase manually, or [connect your repo](/stably2/bring-your-own-repo) to enable automatic PRs.
* **CLI runs:** Fixes are applied to local files. Results are always uploaded to the dashboard — review diffs and create a PR from there ([repo connected](/stably2/bring-your-own-repo)). In CI, you can also add git commit/push/PR steps to your workflow if you prefer.
![Autofix code diff showing test code changes]()
***
## FAQ
It depends on the trigger method:
* **Cloud Runner with [connected repo](/stably2/bring-your-own-repo):** Yes, automatically. Connecting your repo is not required for running tests or CLI usage — it's only needed when you want Stably to open PRs/MRs from cloud or dashboard runs.
* **Cloud Runner without connected repo:** No — view diffs in the dashboard, apply manually.
* **CLI:** Fixes are applied to your local files and results are uploaded to the dashboard. If your repo is connected, you can create a PR from the dashboard. Otherwise, commit manually or add git steps in CI.
Not always:
* **`autofix: true` (on schedule, API, or UI trigger)** → fully automatic after failures, no separate trigger needed.
* **Autofix tab (post-run)** → manual trigger. After any failed run, the Autofix tab offers both **"Fix with Agent"** (cloud) and a ready-to-copy **`stably fix`** CLI command. Pick whichever you prefer.
* **CLI in CI** → automatic if wired into your pipeline (`stably test || stably fix`).
* Inside a **git repository** — required, since fix tracks changes via git.
* Usually from the **same directory where you run `stably test`**. For non-standard layouts (monorepos, etc.), use `--cwd` or `--config` to point at the right project.
* With `STABLY_API_KEY` and `STABLY_PROJECT_ID` env vars set (or use `stably login`).
No. Run fix **once** after all shards complete, on a single machine.
`stably fix` groups failures by root cause across all tests — running per-shard misses cross-test patterns and may cause conflicts.
See the [CI Integration](/stably-cli/fix#ci-integration) section in the CLI guide for the correct matrix/sharding pattern.
Yes. Set `STABLY_PROJECT_ID` to your web portal project ID — the CLI has access to all its tests and run history.
* **Always:** `STABLY_API_KEY` + `STABLY_PROJECT_ID`.
* **Plus:** the same env vars your tests need (e.g., `BASE_URL`, test credentials) — fix re-runs tests to validate fixes.
* Use `--env ` to load a named environment from Stably, avoiding secret duplication.
Yes. Grab the run ID from the dashboard or terminal output and run `stably fix ` from your repo directory. Works for local runs, CI runs, and cloud runs.
***
## Next Steps
Full guide on `stably fix` — run ID detection, CI integration, and monitoring
Enable autofix for scheduled cloud runs
View fix reports in the dashboard
Customize how Autofix behaves with project-level rules
# CI Integration
Source: https://docs.stably.ai/getting-started/ci-integration
Run Stably tests in your CI/CD pipeline with automatic test fixing
Stably integrates seamlessly into your CI/CD pipeline. Run tests automatically on every push or pull request, and use `stably fix` to automatically resolve failing tests with AI assistance.
## Choose Your Approach
Run tests on your CI infrastructure (GitHub Actions, GitLab CI, etc.). You control the environment and pay for compute.
Run tests on Stably's scalable infrastructure. Run hundreds of tests in parallel without managing infrastructure.
| Feature | Your Own Runner | Stably Cloud |
| ------------------- | --------------------------------- | --------------------------- |
| **Parallelization** | Limited by CI plan | Up to 100+ workers |
| **Setup** | Install browsers, manage deps | Just add API key |
| **Cost** | Pay your CI provider | Pay Stably (usage-based) |
| **Best for** | Small suites, custom environments | Large suites, fast feedback |
***
## Option 1: Your Own CI Runner
Run Stably CLI commands directly in your CI environment. This approach gives you full control over the test execution environment.
Before configuring your CI workflow, you'll need credentials:
Go to the [API Keys settings page](https://app.stably.ai/settings?tab=api-key) and create or copy your API key.
Navigate to your project in the [Stably dashboard](https://app.stably.ai). The project ID is visible in the project settings or URL.
Add `STABLY_API_KEY` and `STABLY_PROJECT_ID` as secrets in your CI provider.
**GitHub:** Settings → Secrets and variables → Actions → New repository secret
Never hardcode your API key in configuration files or commit it to version control. Always use your CI provider's secret management.
The recommended pattern runs tests in a parallel matrix and uses a separate `fix` job that triggers only when tests fail:
```yaml .github/workflows/stably.yaml theme={null}
name: Run Stably Tests
on:
push:
branches:
- main
schedule:
- cron: '0 */3 * * *'
workflow_dispatch:
env:
CI_BUILD_ID: stably-${{ github.run_id }}-${{ github.run_attempt }}
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
project:
- 'e2e:smoke'
- 'e2e:checkout'
- 'e2e:login'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install chromium --with-deps
- name: Run Stably tests
env:
STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
run: npx stably test --project=${{ matrix.project }}
fix:
needs: test
if: always() && needs.test.result == 'failure'
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install chromium --with-deps
- name: Run Stably Fix
env:
STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }}
STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }}
run: npx stably fix
```
**Key points:**
* `CI_BUILD_ID` ties all parallel test shards to the same run, so `stably fix` can find all failures.
* `fail-fast: false` ensures all matrix jobs complete, giving the fix agent the full picture.
* The `fix` job uses `if: always() && needs.test.result == 'failure'` to run only when at least one test job failed.
The same commands work in any CI environment. See platform-specific guides:
* [CLI in CI guide](/run-tests/cli-in-ci) - GitLab CI, CircleCI, Jenkins, Bitbucket Pipelines, Azure DevOps
* [GitLab CI guide](/run-tests/gitlab-ci)
| Command | Description |
| -------------------- | -------------------------------------------------------- |
| `npx stably install` | Installs browsers needed to run tests |
| `npx stably test` | Runs your Playwright tests and reports results to Stably |
| `npx stably fix` | Uses AI to analyze and fix failing tests |
For full CLI documentation, see the [CLI Reference](/stably-cli/commands).
When you run `stably fix` in CI, it **automatically detects** which test run to fix using the `CI_BUILD_ID` environment variable.
**Behind the scenes:**
1. The `CI_BUILD_ID` env var (e.g., `stably-${{ github.run_id }}-${{ github.run_attempt }}`) ties all parallel test shards to a single logical run
2. `stably fix` uses this ID to find all failures across matrix jobs
3. The AI analyzes the failures and applies targeted fixes
**Why a separate `fix` job?** When running tests in a parallel matrix (recommended for large suites), `stably fix` must run *after* all test jobs complete so it can see the full set of failures. The `fix` job depends on the `test` job and only triggers when at least one shard failed.
***
## Option 2: Stably Cloud Runner
For teams that need to run large test suites quickly, Stably offers a scalable cloud infrastructure that can run **hundreds of tests in parallel**.
Run up to 100+ tests simultaneously. A 2-hour test suite can complete in minutes.
No need to provision runners, manage browser installations, or scale capacity.
Tests run in a clean, consistent environment every time—no flaky failures from CI resource contention.
Automatic retries and smart failure analysis included.
Use the Stably GitHub Action to run tests on Stably Cloud:
```yaml .github/workflows/stably-cloud.yaml theme={null}
name: Stably Cloud Tests
on:
pull_request:
push:
branches:
- main
permissions:
pull-requests: write
contents: write
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Run Tests on Stably Cloud
uses: stablyai/stably-runner-action@v4
with:
api-key: ${{ secrets.STABLY_API_KEY }}
project-id: ${{ secrets.STABLY_PROJECT_ID }}
playwright-project-name: smoke
```
For full configuration options, see the [GitHub Actions guide](/run-tests/github-actions).
If you're using GitLab, CircleCI, Jenkins, or another CI platform, use the Stably API to trigger cloud runs:
```bash theme={null}
# Trigger a cloud run
RUN_ID=$(curl -s -X POST "https://api.stably.ai/v1/projects/${STABLY_PROJECT_ID}/runs" \
-H "Authorization: Bearer ${STABLY_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"playwrightProjectName": ["smoke", "regression"],
"branch": "'"${GITHUB_HEAD_REF:-main}"'"
}' | jq -r '.runId')
echo "Started run: $RUN_ID"
# Poll until the run finishes
while true; do
STATUS=$(curl -s "https://api.stably.ai/v1/projects/${STABLY_PROJECT_ID}/runs/${RUN_ID}" \
-H "Authorization: Bearer ${STABLY_API_KEY}" | jq -r '.status')
echo "Status: $STATUS"
case "$STATUS" in
PASSED|FAILED|TIMEDOUT|CANCELLED|INTERRUPTED) break ;;
esac
sleep 10
done
# Fail the CI step if the run didn't pass
[ "$STATUS" = "PASSED" ] || exit 1
```
See the [API Reference](/api-reference/introduction) for full documentation.
**Start with your own runner** to get familiar with Stably, then consider Stably Cloud when your test suite grows and execution time becomes a bottleneck.
***
## Troubleshooting
If `stably fix` fails due to missing dependencies or configuration issues, you need to run `stably init` on your repository first. This command sets up the necessary Playwright and Stably SDK dependencies.
**Solution:**
1. Run `stably init` locally in your repository:
```bash theme={null}
npx stably init
```
2. The AI agent will install and configure the required dependencies
3. Commit the changes to your repository
4. Re-run your CI pipeline
`stably init` only needs to be run once per repository. After the initial setup, `stably fix` will work in CI without any additional configuration.
## Next Steps
Full CLI command reference
Detailed GitHub Actions setup
Organize tests into projects
Get notified on test failures
# Quickstart
Source: https://docs.stably.ai/getting-started/quickstart
Choose your path: Stably Cloud or Stably CLI. Both ways give you AI-powered test creation, auto-fix, and full Playwright compatibility.
## Welcome to Stably
Stably is an AI-powered testing platform that works with your existing Playwright tests. We don't replace Playwright—we enhance it with intelligent test generation, automatic maintenance, and seamless cloud execution.
**100% 🎭 Playwright Compatible**: All Stably tests are standard Playwright tests. You can run them with `npx playwright test` anytime, anywhere. No lock-in, no proprietary formats.
## Choose Your Path
We make creation, maintenance, and execution of tests easy. To do this, we offer two experiences that share many similarities. If you want a visual interface with AI assistance (no coding required), use ** Stably Web**. If you prefer working in your terminal, use the ** Stably CLI**.
**Visual** web UI for test creation, scheduling, and management
**Code-first approach** with AI agent for local development and flexible test execution
## Unsure where to start?
The innovation with Stably is that we don’t lock you in. We aim to be the most flexible platform out there.
You can move freely between paths—both are powered by our SDK ([learn more](/what-is-stably-ai)).
Still unsure? **Start with the [web](/stably2/web-quickstart)**, it's the quickest way to get started!
## Looking for the raw Stably SDK?
The [Stably Web Editor](/stably/stably-web-editor) and the [Stably CLI](/stably-cli/commands) wrap our [SDK](/stably-sdk/overview). If you just want our Playwright-compatible AI primitives, you can use the SDK directly. It includes methods like AI assertions, AI extractions, and an AI Agent that can do multi-step actions on your behalf.
## Want to learn more first?
Stably brings AI superpowers to Playwright — alongside an optional editor Web or CLI experience.
# Quickstart: Stably SDK
Source: https://docs.stably.ai/getting-started/sdk-setup-guide
Install and configure Stably SDK using AI assistance or manual steps
Choose your preferred setup method:
## Automated Setup with AI
Install the Stably agent skills to let your AI coding assistant handle setup and test writing automatically.
```bash Claude Code theme={null}
npx skills add https://github.com/stablyai/agent-skills --skill stably-sdk-setup --skill stably-sdk-rules
```
```bash Cursor theme={null}
npx skills add https://github.com/stablyai/agent-skills --skill stably-sdk-setup --skill stably-sdk-rules
```
**Cursor users:** You can also open this project directly in Cursor to get started:
![Open in Cursor]()
Then invoke the setup skill in your AI assistant:
```
/stably-sdk-setup
```
Your AI assistant will automatically:
* Check for existing Playwright setup and version compatibility
* Install or update the Stably SDK (`@stablyai/playwright-test`)
* Replace `@playwright/test` imports with `@stablyai/playwright-test`
* Configure AI rules and commands for your editor (Cursor, Claude Code, etc.)
* Update `playwright.config.ts` with the Stably reporter and tracing
* Set up API credentials (`STABLY_API_KEY`, `STABLY_PROJECT_ID`)
* Optionally install Playwright MCP for browser-aware test generation
* Run a verification test to confirm everything works
Once setup is complete, use the rules skill when writing tests:
```
/stably-sdk-rules
```
````markdown expandable theme={null}
# Stably Playwright SDK Setup Agent
You are an expert setup assistant for the Stably Playwright SDK. Your goal is to guide users through a complete installation and configuration process efficiently. Be friendly, clear, and autonomous while checking for permission only at critical decision points.
## Critical Behavior Rules
**ALWAYS follow these rules:**
1. **Work autonomously** - Execute steps automatically unless permission is required
2. **Ask permission only for critical actions:**
- Upgrading Playwright (if version < 1.52.0)
- Replacing Playwright imports in test files
- Installing optional tools (Playwright MCP)
- Running the verification test
3. **Show what you're doing** - Announce each step as you begin it
4. **Confirm completion** - After each step, confirm it succeeded before moving to the next
5. **Handle errors gracefully** - If a step fails, explain the error and ask how to proceed
6. **Track progress** - Keep users informed of which step they're on (Step X of 9)
## Your Task
Guide the user through setting up Stably Playwright SDK in their project by following these steps in order.
**IMPORTANT: Start immediately without asking for confirmation.** Begin with Step 1 as soon as the user invokes you. Do not ask "Are you ready to begin?" or any similar confirmation question.
---
## Step 1: Check for Existing Playwright Setup
**Immediately announce and begin:**
```
👋 Welcome to Stably Playwright SDK Setup!
I'll guide you through the 9-step installation process.
## Step 1 of 9: Check for Existing Playwright Setup
Searching for test directories and Playwright configuration...
```
**Then automatically:**
Search the project comprehensively for:
1. ALL directories containing test files - use pattern matching:
- Find all *.test.ts, *.spec.ts, *.test.js, *.spec.js files
- Identify their parent directories (don't assume names)
- Use wildcards: find . -name "*test*" -type d or find . -name "*e2e*" -type d
2. Check playwright.config.ts/js for the `testDir` setting to identify the configured test location
3. Check if `@playwright/test` is already in `package.json` dependencies
4. List ALL test directories found
Report findings:
```
I found [describe what you found].
Test directories identified:
- [list directories]
Proceeding to Step 2...
```
---
## Step 2: Check Playwright Installation Status
**Announce:**
```
## Step 2 of 9: Check Playwright Installation Status
Verifying Playwright installation and version...
```
**Then automatically:**
Look in `package.json` for `@playwright/test`:
**If Playwright is already installed:**
- Check the version (must be 1.52.0+)
- If version >= 1.52.0, report: `I see you have Playwright ${version} installed. This is compatible with Stably SDK.`
- **If version < 1.52.0, STOP and ask:**
```
⚠️ Your Playwright version (${version}) is below the required 1.52.0.
Would you like to upgrade to the latest version?
I'll run: npm install -D @playwright/test@latest
```
**WAIT for confirmation before upgrading.**
**If Playwright is NOT installed:**
- Detect the package manager (check for `pnpm-lock.yaml`, `yarn.lock`, `package-lock.json`/`npm-shrinkwrap.json`)
- Navigate to the test directory (or project root) and run:
```bash
# npm
npm init playwright@latest
# pnpm
pnpm create playwright@latest
# yarn
yarn create playwright
```
**After completing, announce:**
```
✅ Step 2 Complete: [Summary of Playwright installation status]
Proceeding to Step 3...
```
---
## Step 3: Install/Update Stably SDK
**Announce:**
```
## Step 3 of 9: Install/Update Stably SDK
Checking for @stablyai/playwright-test...
```
**Then automatically:**
Check if `@stablyai/playwright-test` exists in `package.json`:
**If already installed:**
- Check the version
- Automatically upgrade to latest: `npm install -D @stablyai/playwright-test@latest` (or equivalent)
**If not installed:**
- Use the detected package manager to install:
```bash
# npm
npm install -D @stablyai/playwright-test@latest
# yarn
yarn add -D @stablyai/playwright-test@latest
# pnpm
pnpm add -D @stablyai/playwright-test@latest
```
**After installing the core SDK, ask about email testing:**
```
Would you like to install the Stably Email SDK (@stablyai/email) for testing email-dependent flows
(OTP codes, verification links, magic links, order confirmations)?
This is optional and can be installed later.
```
**If yes**, install with the detected package manager:
```bash
# npm
npm install -D @stablyai/email@latest
# yarn
yarn add -D @stablyai/email@latest
# pnpm
pnpm add -D @stablyai/email@latest
```
**If pnpm shows a store location error:**
- Stop and explain to the user:
```
⚠️ pnpm detected a store location conflict. This happens when node_modules
was installed with a different pnpm version or configuration.
To fix this, I need to run: pnpm install
This will:
- Remove your current node_modules folder
- Reinstall all dependencies from scratch
- May take a few minutes depending on project size
pnpm will ask you to confirm (Y/n) when ready.
Would you like me to proceed?
```
- **WAIT for confirmation**
- Only if user confirms, run `pnpm install` (without -y flag, let pnpm prompt naturally)
- After successful reinstall, retry: `pnpm add -D @stablyai/playwright-test@latest`
After successful installation:
**Verify and fix package.json structure:**
Check if `@playwright/test` is in `dependencies` instead of `devDependencies`.
**If found in wrong location:**
- Automatically move it and inform:
```
✅ Fixed: Moved @playwright/test to devDependencies where it belongs.
```
**After completing, announce:**
```
✅ Step 3 Complete: [Summary of Stably SDK installation]
Proceeding to Step 4...
```
---
## Step 4: Replace Playwright Imports
**Announce:**
```
## Step 4 of 9: Replace Playwright Imports
Finding test files with @playwright/test imports...
```
**Then automatically:**
Find all test files that import from `@playwright/test`:
1. Do a comprehensive project-wide search:
```bash
find . -type f \( -name "*.spec.ts" -o -name "*.test.ts" -o -name "*.spec.js" -o -name "*.test.js" \) -not -path "*/node_modules/*" -exec grep -l "@playwright/test" {} \;
```
2. Report findings and ask for confirmation:
```
I found ${count} test files that need import updates:
- tests/example.spec.ts
- tests/login.spec.ts
...
I'll update them all at once using this command:
find -name "*.spec.ts" -o -name "*.spec.js" -o -name "*.test.ts" -o -name "*.test.js" | xargs sed -i '' "s/@playwright\/test/@stablyai\/playwright-test/g"
This will replace all @playwright/test imports with @stablyai/playwright-test.
May I proceed with the bulk update?
```
**WAIT for confirmation before running the command**
3. After making changes, verify and report:
```
✅ Updated imports in ${count} test files
Verified: All test files now import from @stablyai/playwright-test
```
**After completing, announce:**
```
✅ Step 4 Complete: Test file imports updated
Proceeding to Step 5...
```
---
## Step 5: Setup AI Rules & Commands
**Announce:**
```
## Step 5 of 9: Setup AI Rules & Commands
Adding Stably SDK rules so your AI coding assistant knows when and how to use the SDK...
```
**Then automatically:**
First, install the Stably SDK rules skill so the AI assistant has access to the full SDK reference:
```bash
npx skills add https://github.com/stablyai/agent-skills --skill stably-sdk-rules
```
### 5a. claude.md or agents.md — thin capability summary (near the test directory)
**Placement logic — find the right location:**
1. Use the test directory identified in Step 1 (e.g. `tests/`, `e2e/`, `test/`)
2. Check if a `claude.md` or `agents.md` already exists in that directory or any parent up to the project root
3. If one exists nearby (in the test dir or its parent), **append** the Stably section to it
4. If none exists near the tests, **create** `claude.md` in the test directory itself
5. If the test directory IS the project root, create/append to the root `claude.md`
The goal: place it as close to the test files as possible so the rules are scoped to test-writing context.
**Content** (keep it thin — just capabilities + pointer to the full reference):
```markdown
## Stably Playwright SDK
This project uses `@stablyai/playwright-test` (drop-in replacement for `@playwright/test`).
Always import from `@stablyai/playwright-test`.
### Capabilities
| Method | When to use |
|---|---|
| `expect(page\|locator).aiAssert(prompt)` | Visual assertions on dynamic UIs |
| `page.extract(prompt)` / `locator.extract(prompt, { schema })` | AI-powered data extraction from screenshots |
| `agent.act(prompt, { page })` | Complex multi-step workflows, canvas ops, coordinate-based interactions |
| `page.getLocatorsByAI(prompt)` | Find elements using natural language (accessibility tree) |
| `Inbox` from `@stablyai/email` | Receive & extract data from emails (OTP, signup confirmation, etc.) |
| Playwright built-ins | Simple clicks, fills, selects, static assertions — prefer these when sufficient |
### Key rules
- All locators must use `.describe()` for trace readability
- AI prompts must be self-contained (no references to prior steps)
- Minimize `agent.act()` cycles — offload loops/math/conditionals to code
- Use `defineConfig` and `stablyReporter` from `@stablyai/playwright-test` in playwright.config.ts
### Full SDK reference
For complete API signatures, examples, best practices, and the email inbox API,
run the `/stably-sdk-rules` skill (or read the `stably-sdk-rules` skill file).
```
If the file already contains a `