Introduction
In addition to STABLY.md (which applies to all agent modes), you can create mode-specific rules files that are only loaded for a particular command. This is a great way to customize the agent to match your team’s workflow — your conventions, your folder structure, your assertion style — while still getting the full benefit of Stably’s built-in browser management, test debugging tooling (e.g. Playwright Trace Viewer), and intelligent test generation under the hood.
| File | Loaded by | Purpose |
|---|
STABLY-CREATE.md | stably create | How tests should be written — file structure, patterns, assertions |
STABLY-FIX.md | stably fix | How tests should be repaired — selector strategy, timeout handling, test integrity |
If a rule applies to all modes (create, fix, init), put it in STABLY.md instead.
Quick Start
STABLY-CREATE.md
STABLY-FIX.md
Create the file
Add a STABLY-CREATE.md file to your project root:# Test Generation Rules
- Prefer `getByRole` and `getByText` locators; fall back to `data-testid`
- Use Playwright fixtures for shared setup/teardown
- Include both positive and negative test cases for form validations
- Use `test.describe` blocks to group related scenarios
Run a test generation command
The rules are automatically loaded and applied:stably create "test the checkout flow"
✓ STABLY-CREATE.md loaded (320 chars)
Create the file
Add a STABLY-FIX.md file to your project root:# Fix Rules
- Prefer user-facing locators (`getByRole`, `getByText`) when fixing locator failures; fall back to `data-testid`
- Always add a comment explaining why a selector was changed
- Never delete or skip a failing test — fix it or flag it for review
- When fixing timeouts, increase wait time as a last resort — look for root causes first
Run a fix command
The rules are automatically loaded and applied:You’ll see a confirmation in the CLI output:✓ STABLY-FIX.md loaded (280 chars)
File Location and Naming
Both files follow the same conventions:
| Requirement | Details |
|---|
| Location | Project root directory (same level as package.json) |
| Filenames | Must be uppercase: STABLY-CREATE.md, STABLY-FIX.md |
| Source control | Should be committed to git |
Filenames must be uppercase. macOS is case-insensitive, so stably-create.md will work locally — but Linux (used in CI/ECS) is case-sensitive and will silently ignore the wrong case.
Commit these files to your repository so your entire team shares the same conventions. This is especially important when commands run in CI pipelines or are invoked by background agents.
Both files use freeform markdown. There’s no required structure — write whatever instructions you want the agent to follow. The content is appended directly to the agent’s system prompt.
Maximum length: 10,000 characters (~2,500 tokens) per file. Files exceeding this limit are automatically truncated with a CLI warning.
When to Use Each File
STABLY-CREATE.md
Use for rules specific to test generation:
- Test file naming conventions and folder structure
- Test patterns and frameworks (Page Object Model, fixtures, etc.)
- Assertion strategies and test data patterns
- Tags and annotations (
@smoke, @regression)
- Which Stably SDK methods to prefer (
agent.act(), aiAssert(), etc.)
STABLY-FIX.md
Use for rules specific to test fixing:
- Selector replacement strategies (e.g., prefer
data-testid over CSS classes)
- How to handle flaky tests vs genuinely broken tests
- Timeout and retry policies
- Which files or directories the agent should or shouldn’t modify
- Comment and documentation requirements for changes
- When to flag a test for human review instead of auto-fixing
Examples
STABLY-CREATE.md
STABLY-FIX.md
# Test Generation Rules
## File Structure
- Place all tests in the `tests/` directory
- Name test files using the pattern `<feature>.spec.ts`
- Group related tests with `test.describe` blocks
## Test Patterns
- Use Playwright fixtures for shared setup/teardown; Page Object Model is acceptable for large suites
- Prefer user-facing locators: `getByRole`, `getByText`, `getByLabel`
- Fall back to `data-testid` when no accessible locator is available
- Never use CSS class selectors
## Assertions
- Include both positive and negative test cases for form validations
- Use `aiAssert()` for visual assertions instead of pixel comparisons
- Use `agent.act()` for complex multi-step UI interactions (e.g., drag-and-drop)
## Test Data
- Create fresh test data in each test — never modify shared records
- Use environment variables for credentials (never hard-code)
- Use fixtures for setup and teardown — `afterEach` won't run if a test crashes
## Tags
- Add `@smoke` annotation to critical-path tests
- Add `@regression` annotation to edge-case tests
# Fix Rules
## Selector Strategy
- When a selector breaks, replace it with a `data-testid` attribute
- If the element already has a `data-testid`, check if the value changed
- Prefer user-facing locators (`getByRole`, `getByText`) over `data-testid` — never use CSS class selectors
- Add a comment above changed selectors explaining the fix
## Timeout Handling
- Do not increase timeout values as the first fix attempt
- Investigate whether the page load changed or a network call is slower
- If a `waitForSelector` times out, check if the element was renamed or moved
## Test Integrity
- Never delete a failing test — fix it or mark it with `test.fixme()` for review
- Never change assertion values to make a test pass — fix the root cause
- If a test is flaky (passes on retry), investigate the root cause (race condition, shared state, missing auto-wait) before adding retries
## Boundaries
- Do not modify files outside the `tests/` directory
- Do not modify `playwright.config.ts` unless the failure is config-related
- Do not install new npm packages without flagging for review
STABLY-FIX.md vs agent.fix.rules in stably.yaml
You can also define fix rules in stably.yaml under agent.fix.rules:
agent:
fix:
rules: |
Prefer data-testid selectors over CSS selectors.
Always add comments explaining selector changes.
STABLY-FIX.md and agent.fix.rules are present, they are concatenated — agent.fix.rules is loaded first, then STABLY-FIX.md. They are complementary, not competing.
| Mechanism | Best for |
|---|
STABLY-FIX.md | Longer, detailed fix rules in freeform markdown — easy to review in PRs |
agent.fix.rules | Short, structured rules that live alongside other stably.yaml config |
Relationship to Other Config Files
| File | Scope | Modes | Description |
|---|
STABLY.md | Project-wide rules | All modes (create, fix, init) | General instructions — project context, selector strategy, do’s and don’ts |
STABLY-CREATE.md | Test generation rules | create only | Test-specific conventions — file naming, folder structure, test patterns |
STABLY-FIX.md | Fix rules | fix only | Fix-specific conventions — selector strategy, timeout handling, test integrity |
stably.yaml → agent.fix.rules | Fix agent rules | fix | Structured config for the fix agent (max turns, parallel tests, rules) |
[Base System Prompt]
→ stably.yaml agent.fix.rules
→ STABLY-FIX.md (fix mode only)
→ STABLY-CREATE.md (create mode only)
→ STABLY.md (all modes)
Don’t duplicate rules across files. Put general project rules in STABLY.md, test-generation-specific conventions in STABLY-CREATE.md, and fix-specific conventions in STABLY-FIX.md.
Best Practices
| Do | Avoid |
|---|
| Commit both files to source control so CI and teammates share the same conventions. | Adding them to .gitignore — CI agents won’t find them. |
| Keep rules focused on their respective mode (generation vs fixing). | Putting general project context here — use STABLY.md for that. |
| Use concise, actionable rules the agent can follow. | Writing novel-length instructions that exceed the 10,000 character limit. |
Use uppercase filenames: STABLY-CREATE.md, STABLY-FIX.md. | Using lowercase — it will fail silently on Linux CI servers. |
| Review changes in PRs like any other config file. | Duplicating rules that already exist in STABLY.md or stably.yaml. |
