> ## Documentation Index > Fetch the complete documentation index at: https://docs.stably.ai/llms.txt > Use this file to discover all available pages before exploring further. # Integrate with Test Case Management Systems > Report Stably test results to external test case management systems like Zephyr Scale, Xray, or TestRail Many teams manage their test cases in a dedicated test case management system (TCMS) like **Zephyr Scale**, **Xray**, or **TestRail** while using Stably to author, run, and auto-heal their Playwright tests. This guide shows how to report Stably test results back to your TCMS so that QA leads and PMs see a unified view of test health. The approach depends on where your tests execute: | Execution Model | How Results Are Available | Best Integration Path | | ---------------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------ | | **Your own CI** (GitHub Actions, GitLab, etc.) | Playwright JUnit XML + Stably Reporter | Use Playwright's built-in JUnit reporter — most TCMS tools import JUnit XML natively | | **Stably Cloud** | Stably REST API | Poll the API for results and push to your TCMS via a CI step or webhook script | ## Option 1: Running in Your Own CI with Playwright When you run `stably test` or `npx playwright test` in your own CI environment, Playwright executes locally and you have full control over reporter output. This is the simplest integration path because most TCMS tools — including Zephyr Scale — natively import **JUnit XML**. ### Step 1: Add the JUnit Reporter Add Playwright's built-in JUnit reporter alongside the Stably reporter in your `playwright.config.ts`: ```typescript playwright.config.ts theme={null} import { defineConfig, stablyReporter } from "@stablyai/playwright-test"; export default defineConfig({ reporter: [ ["list"], // Stably reporter — streams results to Stably dashboard stablyReporter({ apiKey: process.env.STABLY_API_KEY, projectId: process.env.STABLY_PROJECT_ID, }), // JUnit reporter — generates XML for your TCMS ["junit", { outputFile: "test-results/junit-results.xml" }], ], use: { trace: "on", }, }); ``` This produces a standard `junit-results.xml` file after every test run, alongside streaming results to the Stably dashboard. ### Step 2: Upload Results to Zephyr Scale After tests complete, upload the JUnit XML to Zephyr Scale using their CLI or API. Here's an example for GitHub Actions: ```yaml .github/workflows/stably-zephyr.yml theme={null} name: Stably Tests → Zephyr Scale on: push: branches: [main] pull_request: jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node uses: actions/setup-node@v4 with: node-version: "20" - name: Install dependencies run: npm ci - name: Install browsers run: npx stably install - name: Run Stably tests env: STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }} STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }} run: npx stably test - name: Upload results to Zephyr Scale if: always() run: | npx @smartbear/zephyr-scale-cli \ --mode junit \ --projectKey "${{ vars.ZEPHYR_PROJECT_KEY }}" \ --junitFile test-results/junit-results.xml \ --autoCreateTestCases env: ZEPHYR_API_TOKEN: ${{ secrets.ZEPHYR_API_TOKEN }} ``` ```yaml .github/workflows/stably-zephyr-api.yml theme={null} name: Stably Tests → Zephyr Scale (API) on: push: branches: [main] pull_request: jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node uses: actions/setup-node@v4 with: node-version: "20" - name: Install dependencies run: npm ci - name: Install browsers run: npx stably install - name: Run Stably tests env: STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }} STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }} run: npx stably test - name: Upload results to Zephyr Scale if: always() run: | curl -X POST \ "https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey=${{ vars.ZEPHYR_PROJECT_KEY }}&autoCreateTestCases=true" \ -H "Authorization: Bearer ${{ secrets.ZEPHYR_API_TOKEN }}" \ -F "file=@test-results/junit-results.xml" ``` The `--autoCreateTestCases` flag (or `autoCreateTestCases=true` query parameter) tells Zephyr to automatically create test cases in your project for any test names it hasn't seen before. This means you don't need to manually pre-create every test case in Zephyr — they're created on first import. ### How Test Names Map to Zephyr Playwright's JUnit XML uses this naming structure: ``` ``` Zephyr Scale uses the `name` attribute to match or create test cases. To keep things clean: * Use descriptive, stable test names in your Playwright tests * Avoid dynamically generated test names that change between runs * Use [Playwright tags](/use-cases/defining-test-groups) to organize tests, and map those to Zephyr folders or labels ### Other TCMS Tools The JUnit XML approach works with any TCMS that supports JUnit import: | TCMS | JUnit Import Method | | ---------------- | ---------------------------------- | | **Zephyr Scale** | CLI tool, REST API, or Jira plugin | | **Xray** | REST API or Jira plugin | | **TestRail** | CLI tool (`trcli`) or API | | **qTest** | API or Pulse integration | ## Option 2: Running on Stably Cloud When tests run on [Stably Cloud](/run-tests/run-tests-on-cloud), you don't have direct access to the file system where tests execute, so you can't grab a JUnit XML file. Instead, use the **Stably REST API** to retrieve structured test results and push them to your TCMS. The Stably API requires an API key. Get yours from the [API Key Dashboard](https://app.stably.ai/settings?tab=api-key). ### Step 1: Trigger a Cloud Run and Get the Run ID Trigger a run using any method — the Web Editor, CLI, API, or scheduled runs. All methods produce a `runId` you can use to fetch results. ```bash theme={null} # Trigger via API curl -X POST "https://api.stably.ai/v1/projects/$STABLY_PROJECT_ID/runs" \ -H "Authorization: Bearer $STABLY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "playwrightProjectName": "my-project" }' # Response: { "runId": "abc123" } ``` ### Step 2: Poll for Results Poll the run status until it completes: ```bash theme={null} curl "https://api.stably.ai/v1/projects/$STABLY_PROJECT_ID/runs/$RUN_ID" \ -H "Authorization: Bearer $STABLY_API_KEY" ``` Once the run finishes, the response includes detailed test results: ```json theme={null} { "status": "FAILED", "startedAt": "2025-09-15T12:00:00.000Z", "finishedAt": "2025-09-15T12:02:30.000Z", "branchName": "main", "results": { "testCases": [ { "title": "should complete purchase", "status": "PASSED", "durationMs": 12340 }, { "title": "should show error on declined card", "status": "FAILED", "durationMs": 8760 } ] } } ``` Possible test case statuses: `PASSED`, `FAILED`, `TIMEDOUT`, `SKIPPED`, `INTERRUPTED`, `FLAKY`. ### Step 3: Push Results to Zephyr Scale Write a script that maps Stably results to Zephyr's test execution API. Here's a complete example: ```bash sync-to-zephyr.sh theme={null} #!/bin/bash set -euo pipefail STABLY_PROJECT_ID="${STABLY_PROJECT_ID:?Set STABLY_PROJECT_ID}" STABLY_API_KEY="${STABLY_API_KEY:?Set STABLY_API_KEY}" ZEPHYR_API_TOKEN="${ZEPHYR_API_TOKEN:?Set ZEPHYR_API_TOKEN}" ZEPHYR_PROJECT_KEY="${ZEPHYR_PROJECT_KEY:?Set ZEPHYR_PROJECT_KEY}" RUN_ID="${1:?Usage: $0 }" # 1. Fetch results from Stably API RESULTS=$(curl -sf \ "https://api.stably.ai/v1/projects/$STABLY_PROJECT_ID/runs/$RUN_ID" \ -H "Authorization: Bearer $STABLY_API_KEY") STATUS=$(echo "$RESULTS" | jq -r '.status') if [ "$STATUS" = "RUNNING" ] || [ "$STATUS" = "QUEUED" ]; then echo "Run $RUN_ID is still $STATUS. Wait for it to complete." exit 1 fi # 2. Convert Stably results to JUnit XML echo "$RESULTS" | jq -r ' .results.testCases as $tests | ($tests | length) as $total | ($tests | map(select(.status == "FAILED" or .status == "TIMEDOUT")) | length) as $failures | "", "", " ", ($tests[] | if .status == "PASSED" or .status == "FLAKY" then " " else " ", " ", " " end ), " ", "" ' > /tmp/stably-junit.xml echo "Generated JUnit XML with $(echo "$RESULTS" | jq '.results.testCases | length') test cases" # 3. Upload to Zephyr Scale curl -sf -X POST \ "https://api.zephyrscale.smartbear.com/v2/automations/executions/junit?projectKey=$ZEPHYR_PROJECT_KEY&autoCreateTestCases=true" \ -H "Authorization: Bearer $ZEPHYR_API_TOKEN" \ -F "file=@/tmp/stably-junit.xml" echo "Results uploaded to Zephyr Scale" ``` ### Using stably runs for Richer Data The `stably runs view --json` command provides more detailed results including error messages, attempt counts, and file locations — useful if your TCMS supports richer metadata: ```bash theme={null} stably runs view --json | jq '.testCases[] | { title: .title, status: .status, location: .location, durationMs: .durationMs, error: .attempts[-1].errorMessage }' ``` ### Automating the Sync in CI Add the sync script as a post-run step in your CI pipeline: ```yaml .github/workflows/stably-cloud-zephyr.yml theme={null} name: Stably Cloud → Zephyr Scale on: schedule: - cron: "0 2 * * *" # nightly jobs: test-and-sync: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Trigger Stably Cloud run id: trigger run: | RESPONSE=$(curl -sf -X POST \ "https://api.stably.ai/v1/projects/${{ secrets.STABLY_PROJECT_ID }}/runs" \ -H "Authorization: Bearer ${{ secrets.STABLY_API_KEY }}" \ -H "Content-Type: application/json" \ -d '{}') echo "run_id=$(echo $RESPONSE | jq -r '.runId')" >> "$GITHUB_OUTPUT" - name: Wait for completion run: | for i in $(seq 1 60); do STATUS=$(curl -sf \ "https://api.stably.ai/v1/projects/${{ secrets.STABLY_PROJECT_ID }}/runs/${{ steps.trigger.outputs.run_id }}" \ -H "Authorization: Bearer ${{ secrets.STABLY_API_KEY }}" \ | jq -r '.status') echo "Status: $STATUS" if [ "$STATUS" != "RUNNING" ] && [ "$STATUS" != "QUEUED" ]; then break fi sleep 30 done - name: Sync results to Zephyr run: bash sync-to-zephyr.sh "${{ steps.trigger.outputs.run_id }}" env: STABLY_PROJECT_ID: ${{ secrets.STABLY_PROJECT_ID }} STABLY_API_KEY: ${{ secrets.STABLY_API_KEY }} ZEPHYR_API_TOKEN: ${{ secrets.ZEPHYR_API_TOKEN }} ZEPHYR_PROJECT_KEY: ${{ vars.ZEPHYR_PROJECT_KEY }} ``` ## Mapping Test Statuses Stably and Zephyr use different status vocabularies. Here's how they map: | Stably Status | Zephyr Scale Status | Notes | | ------------- | ------------------- | --------------------------------------------------- | | `PASSED` | Pass | Direct mapping | | `FAILED` | Fail | Direct mapping | | `FLAKY` | Pass | Passed on retry — typically treated as pass in TCMS | | `TIMEDOUT` | Fail | Test exceeded timeout | | `SKIPPED` | Not Executed | Test was skipped | | `INTERRUPTED` | Blocked | Run was cancelled mid-execution | When using JUnit XML import, Zephyr automatically maps `` (no failure element) to **Pass** and `` with `` to **Fail**. Skipped tests use the `` element. ## Best Practices Your TCMS matches test cases by name. Avoid dynamically generated names that change between runs — this creates duplicate entries in Zephyr. Automate the TCMS sync so results are always up to date. Don't rely on manual uploads. Let Zephyr auto-create test cases on first import. This avoids the overhead of manually creating entries for every Playwright test. Author and maintain tests in Playwright with Stably. Use the TCMS as a reporting and visibility layer, not as the place where test definitions live. ## Next Steps Stream results to the Stably dashboard for AI-powered debugging Full REST API documentation for programmatic access Execute tests on Stably's cloud infrastructure Set up tests in your deployment pipeline