Skip to main content
Perfect for: Augmenting your PR review process and can be used in conjunction with your PR code review bots

​
Overview

Stably’s exploratory testing agent analyzes your PR feature specifications, and automatically perform comprehensive testing:
  • Analyzes code changes & PR description to understand what functionality needs testing
  • Intelligently navigates through your application
  • Tests edge cases and feature userflows
  • Posts detailed test reports with findings and UI/UX feedback
  • Skips testing for changes that it can’t test

​
Setup Process

1

Install GitHub App

The first step is to connect your GitHub repository to Stably:
  1. Navigate to Settings > GitHub PR Testing in your Stably dashboard
  2. Click Install GitHub App
  3. This will connect your GitHub repository to Stably and enable automated testing on pull requests
GitLab and API-based triggers are coming soon for our exploratory testing agent.
2

Configure Repository

Next, configure which repository and branch Stably should monitor:
  1. In the Configure Repository section:
    • GitHub Organization: Select your GitHub organization or user account
    • Repository: Choose the specific repository you want to enable PR testing for
    • Primary Branch: Select the main branch (usually main or master)
  2. Click Save Configuration to apply your settings
3

Configure Exploratory Testing

The final step involves setting up how the exploratory agent will operate:
  • Use credentials from project-level variables: Enable this option to have the agent login using your existing project credentials
  • If unchecked, login information may need to be provided in the PR description or custom prompts
Provide additional instructions or context for the exploratory agent:
  • Add specific testing instructions (e.g., “Always login with TEST_ACCOUNT”)
  • Include additional test data requirements
  • Specify particular areas of focus for testing (e.g. “The color schemes should be consistent across the app”)
Set up a PR template to help the exploratory testing agent understand what to test:
  • Click Update Pull Request Template to configure
  • The template helps the agent understand the context and scope of each pull request
  • This ensures more targeted and relevant testing

​
How Exploratory Testing Works

Once configured, the exploratory agent will:
  • Automatically trigger when new pull requests are created
  • Manually trigger by commenting @stablyai on any pull request
The AI agent performs comprehensive testing by:
  • Understanding the functionality affected by your changes
  • Navigating through your application intelligently
  • Testing edge cases and user workflows
  • Validating both positive and negative scenarios
After testing completes, you’ll receive:
  • Detailed test reports posted as PR comments
  • Summary of testing activities performed
  • Objectives coverage showing what was tested
  • Detailed findings for each test objective
  • UI/UX feedback on user experience observations
  • Status indicators (✅ COMPLETED, ❓ UNSURE, etc.)
The system intelligently skips testing when:
  • Changes are documentation-only (README, .md files, etc.)
  • No user-facing features or UI interactions are affected
  • The PR template indicates no testing is needed

​
Best Practices

Configuration Tips
  • Use descriptive custom prompts to provide context about your application
  • Set up authentication properly so the agent can access all necessary parts
  • Update PR templates regularly to keep them current with your testing needs
Team Workflow
  • Review test reports and check the detailed findings in PR comments
  • Act on feedback and address any issues raised by the exploratory agent
  • Iterate on configuration and refine setup based on testing results
Optimization
  • Monitor test coverage to ensure the agent tests the right areas
  • Adjust custom prompts and fine-tune instructions based on outcomes
  • Keep test accounts and credentials current

​
Troubleshooting

  • Agent not triggering: Check repository and branch configuration
  • Authentication failures: Verify environment-level variables are set correctly
  • Limited test coverage: Review and update custom prompts for better guidance. Also to make sure PR description contains necessary information about the changes.