Skip to main content

​
The Problem with Code-Only PR Review Bots

Most PR review bots analyze your code statically—they read your code, check for patterns, suggest improvements, and catch potential issues based on syntax and logic analysis. But they never actually run your application. This means they miss an entire category of bugs:
  • UI bugs that only appear when the feature is actually used
  • Complex interaction bugs between multiple components
  • Edge cases that looks correct in code but fail in practice
  • Race conditions and timing issues
  • Subtle UX problems that affect real users

​
How Stably is Different: Exploratory Testing in Action

Stably doesn’t just analyze your code—it actually uses your application like a real QA engineer would.
Stably PR comment showing comprehensive test case execution with pass/fail results

​
It Creates a Testing Plan Like an Experienced Team Member

When your PR is submitted, Stably’s AI agent:
  1. Analyzes your code changes to understand what functionality was added or modified
  2. Reviews your PR description to understand the intent and expected behavior
  3. Creates a comprehensive testing plan covering:
    • Happy path scenarios
    • Edge cases specific to your changes
    • Potential failure modes
    • User interaction flows
This is exactly what an experienced QA engineer or product manager would do when manually reviewing a new feature.
The AI agent thinks like a stakeholder: “What could go wrong here? What edge cases should I test? How will users actually interact with this?”

​
It Catches Deep Bugs Through Actual Usage

By actually navigating through your application and interacting with it, Stably catches bugs that code analysis simply cannot detect.
Detailed bug report showing a critical issue caught during exploratory testing with screenshots
In this example, the agent:
  • Created a test suite without a schedule ✅
  • Verified it appeared correctly in the UI ✅
  • Attempted to delete it and confirmed the deletion modal appeared ✅
This type of bug would be impossible for a code-based PR review bot to catch because:
  • The code might look correct
  • All functions might be properly called
  • But the actual runtime behavior fails

​
Focus on New Features, Not Regression Testing

Important distinction: Stably’s PR exploratory testing is focused on testing the new functionality introduced in your PR, not running your existing test suites.
The exploratory testing agent:
  • ✅ Tests new features like a human teammate discovering the feature for the first time
  • ✅ Explores edge cases specific to the changes in your PR
  • ✅ Validates user experience and interaction flows
  • ✅ Catches unexpected behaviors through intelligent exploration
It does not:
  • ❌ Run your existing regression test suites (use Regression Testing for that)
  • ❌ Test unrelated functionality
  • ❌ Execute pre-written test scripts
Instead, it acts like a product-minded engineer who’s trying your feature for the first time and naturally thinks about what could break.

​
Turn Discoveries into Regression Tests

Here’s where it gets even better: the exploratory testing you get for free can be converted into permanent regression tests. After the exploratory agent tests your new feature and validates it works correctly, you can:
  1. Review the test cases it created during exploration
  2. Convert valuable test scenarios into regression tests with one click
  3. These tests will then run automatically on future PRs to prevent regressions

Exploratory Testing

Tests new features like a human QA engineer discovering them for the first time

Convert to Regression Tests

Transform validated exploratory tests into permanent regression test suites
This creates a virtuous cycle:
  1. PR is created → Exploratory agent tests the new feature
  2. Bugs are caught → Issues are fixed before merge
  3. Tests are saved → Converted to regression tests for future protection
  4. Next PR → Regression tests ensure previous features still work

​
Real-World Testing Scenarios

The exploratory testing agent excels at:
Example: Testing a multi-step checkout process
  • Navigates through each step
  • Tests back button behavior
  • Validates form validation at each stage
  • Checks data persistence between steps
Example: Testing a delete functionality
  • Attempts to delete with dependencies
  • Checks confirmation dialogs
  • Verifies actual deletion occurs
  • Tests error handling for failed deletions
Example: Testing role-based features
  • Logs in with different user roles
  • Verifies correct permissions enforcement
  • Tests edge cases around access control
  • Validates error messages for unauthorized actions
Example: Testing responsive design changes
  • Interacts with the actual UI elements
  • Validates layout and spacing
  • Checks for visual regressions
  • Tests interactive elements (buttons, modals, etc.)

​
Complementary to Code Review Bots

Stably’s exploratory testing is designed to work alongside your existing PR review tools, not replace them:
Tool TypeWhat It CatchesWhat It Misses
Code Review BotsSyntax errors, code style issues, security vulnerabilities in code, logic flawsRuntime bugs, UI issues, edge case failures, UX problems
Stably Exploratory TestingRuntime bugs, UI/UX issues, edge case failures, integration problemsCode quality issues, style violations, potential security vulnerabilities in code
Best Practice: Use both approaches together for comprehensive PR validation:
  • Code review bots catch code-level issues
  • Stably catches runtime and user experience issues

​
Getting Started

Ready to catch bugs that code-only bots miss?

Set Up PR Testing

Learn how to configure exploratory testing for your pull requests in under 5 minutes
Manual triggering: You can also manually trigger exploratory testing on any PR by commenting @stablyai on the pull request.
⌘I