CI Integration

Running tests on every change you introduce to your software is crucial to ensure your application behaves as expected. Stably allows you to run your tests from any CI / CD environment. You can either use our GitHub Action or call our API to execute a test programmatically.

Get API Key

Go to the settings page on the Web UI to get your API key.

Add to CI / CD

If you use GitHub

Create a secret for your API Key: Go to your GitHub repository, click on “Settings” > “Secrets and variables” > “Actions”. Then create a new Repository Secret named STABLY_API_KEY and paste your API Key into the secret value.
From a test suite’s page, click on the Add to CI button to get the code snippet that contains the test suite ID.
After copying the code snippet, create the file .github/workflows/stably.yaml in your GitHub repository and paste the code snippet there.

The code snippet should look like this example:

name: Stably Test Runner Example

on:
  pull_request:
  push:
    branches:
      - master

# You need to set these permissions if using the `github-comment` option
permissions:
  pull-requests: write
  contents: write

jobs:
  stably-test-action:
    name: Stably Test Runner
    runs-on: ubuntu-latest

    steps:
      - name: Global Setup
        run: echo "Running global setup tasks"

      - name: Stably Runner Action
        id: stably-runner
        uses: stablyhq/stably-runner-action@v3
        with:
          api-key: ${{ secrets.STABLY_API_KEY }}
          test-suite-id: xxxxxxxxxxxxxxx # REPLACE WITH YOUR TEST SUITE ID

      - name: Global Tear Down
        run: echo "Running global tear down tasks"

Testing your Pull Requests

Considering you already have a Test Suite to test a live environment, you can reuse that same test suite to run your Pull Request checks. For this, you must use the url-replacement option:

 - name: Stably Runner Action
   id: stably-runner
   uses: stablyhq/stably-runner-action@v3
   with:
     api-key: ${{ secrets.STABLY_API_KEY }}
     test-suite-id: xxxxxxxxxxxxxxx # REPLACE WITH YOUR TEST SUITE ID
     url-replacement: |-
       https://your-test-target-url # REPLACE WITH THE WEBSITE YOU ARE TESTING IN THE TEST SUITE
       http://localhost:8080 # REPLACE THIS WITH THE LOCAL ADDRESS YOU ARE EXPOSING YOUR APPLICATION

Adding notes to test suite runs

When running multiple test suites concurrently, it can be challenging to identify and track specific test runs. To improve test run identification and organization, you can now attach descriptive notes to test suite runs executed via GitHub Actions using the note parameter.This feature allows you to:

Distinguish between concurrent test runs
Add contextual information to test executions
Improve test result tracking and analysis

 - name: Stably Runner Action
   id: stably-runner
   uses: stablyhq/stably-runner-action@v3
   with:
     api-key: ${{ secrets.STABLY_API_KEY }}
     test-suite-id: xxxxxxxxxxxxxxx # REPLACE WITH YOUR TEST SUITE ID
     note: PR for version 1.2.3

If you use other CI / CD

The Stably API enables you to execute test suites programmatically. To authenticate and run tests, you’ll need two required information:

A Stably API key for authentication
A test suite ID that specifies which tests to run

To execute a test, simply run this curl command:

export TEST_SUITE_ID=your-suite-id
export STABLY_API_KEY=your-api-key # Prefer to use secrets

curl -X 'POST' \
"https://api.stably.ai/testGroup/$TEST_SUITE_ID/run" \
-H "accept: application/json" \
-H "x-stably-api-key: $STABLY_API_KEY"

🚀 Get Started

Core Configuration

Stably Platform

Run Tests

Trouble Shooting

About

Testing your Pull Requests

Adding notes to test suite runs