Perfect for: Augmenting your PR review process and can be used in conjunction with your PR code review bots
Overview
Stably’s exploratory testing agent analyzes your PR feature specifications, and automatically perform comprehensive testing:- Analyzes code changes & PR description to understand what functionality needs testing
- Intelligently navigates through your application
- Tests edge cases and feature userflows
- Posts detailed test reports with findings and UI/UX feedback
- Skips testing for changes that it can’t test
Setup Process
1
Install GitHub App
The first step is to connect your GitHub repository to Stably:
- Navigate to Settings > GitHub PR Testing in your Stably dashboard
- Click Install GitHub App
- This will connect your GitHub repository to Stably and enable automated testing on pull requests
GitLab and API-based triggers are coming soon for our exploratory testing agent.
2
Configure Repository
Next, configure which repository and branch Stably should monitor:
Repository Configuration Details
Repository Configuration Details
-
In the Configure Repository section:
- GitHub Organization: Select your GitHub organization or user account
- Repository: Choose the specific repository you want to enable PR testing for
- Primary Branch: Select the main branch (usually
main
ormaster
)
- Click Save Configuration to apply your settings
3
Configure Exploratory Testing
The final step involves setting up how the exploratory agent will operate:

Authentication Configuration
Authentication Configuration
- Use credentials from project-level variables: Enable this option to have the agent login using your existing project credentials
- If unchecked, login information may need to be provided in the PR description or custom prompts
Custom Prompt Configuration
Custom Prompt Configuration
Provide additional instructions or context for the exploratory agent:
- Add specific testing instructions (e.g., “Always login with TEST_ACCOUNT”)
- Include additional test data requirements
- Specify particular areas of focus for testing (e.g. “The color schemes should be consistent across the app”)
Pull Request Template (Optional)
Pull Request Template (Optional)
Set up a PR template to help the exploratory testing agent understand what to test:
- Click Update Pull Request Template to configure
- The template helps the agent understand the context and scope of each pull request
- This ensures more targeted and relevant testing

How Exploratory Testing Works
Automatic & Manual Triggering
Automatic & Manual Triggering
Once configured, the exploratory agent will:
- Automatically trigger when new pull requests are created
- Manually trigger by commenting
@stablyai
on any pull request
Intelligent Testing
Intelligent Testing
The AI agent performs comprehensive testing by:
- Understanding the functionality affected by your changes
- Navigating through your application intelligently
- Testing edge cases and user workflows
- Validating both positive and negative scenarios
Test Reporting
Test Reporting
After testing completes, you’ll receive:
- Detailed test reports posted as PR comments
- Summary of testing activities performed
- Objectives coverage showing what was tested
- Detailed findings for each test objective
- UI/UX feedback on user experience observations
- Status indicators (✅ COMPLETED, ❓ UNSURE, etc.)
Smart Skipping
Smart Skipping
The system intelligently skips testing when:
- Changes are documentation-only (README, .md files, etc.)
- No user-facing features or UI interactions are affected
- The PR template indicates no testing is needed
Best Practices
Configuration Tips- Use descriptive custom prompts to provide context about your application
- Set up authentication properly so the agent can access all necessary parts
- Update PR templates regularly to keep them current with your testing needs
- Review test reports and check the detailed findings in PR comments
- Act on feedback and address any issues raised by the exploratory agent
- Iterate on configuration and refine setup based on testing results
- Monitor test coverage to ensure the agent tests the right areas
- Adjust custom prompts and fine-tune instructions based on outcomes
- Keep test accounts and credentials current
Troubleshooting
Common Issues
Common Issues
- Agent not triggering: Check repository and branch configuration
- Authentication failures: Verify environment-level variables are set correctly
- Limited test coverage: Review and update custom prompts for better guidance. Also to make sure PR description contains necessary information about the changes.
Related Features
- AI Test Generation - Create comprehensive test suites
- Release Testing - Comprehensive release validation
- Scheduled Checks - Regular automated testing
- CI Integration - Run tests in your deployment pipeline
- Environment Variables - Manage test data and credentials