Defining Test Groups - Test Organization

Stably runs your existing Playwright tests. We don’t invent a new test runner or config – we just use your playwright.config.ts.

Overview

Playwright provides many ways to select tests, and Stably supports them all. We recommend using Playwright projects since they let you customize configuration for a logical group of tests.

You can also use projects to run the same tests in different configurations. For example, you can run the same tests in a logged-in and logged-out state.

Defining Projects

Projects are defined in your playwright.config.ts file. Each project represents a logical group of tests with its own configuration.

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  projects: [
    {
      name: 'smoke',
      testMatch: '/smoke/**'
    },
    {
      name: 'release',
      testMatch: '/release/**'
    },
  ],
});

Project Configuration Options

Each project can have its own configuration settings:

Test Selection

testMatch: Pattern to match test files

{
  name: 'smoke',
  testMatch: '/smoke/**/*.spec.ts'
}

testIgnore: Pattern to exclude test files

{
  name: 'all-tests',
  testMatch: '**/*.spec.ts',
  testIgnore: '**/flaky/**'
}

Browser Configuration

use: Browser-specific settings

{
  name: 'chromium-desktop',
  use: {
    browserName: 'chromium',
    viewport: { width: 1920, height: 1080 }
  }
}

Dependencies

dependencies: Run other projects before this one

{
  name: 'authenticated-tests',
  dependencies: ['setup'],
  testMatch: '/tests/**'
}

Common Project Patterns

By Test Type
By Feature Area
By Environment State
By Platform

Organize tests by their purpose:

export default defineConfig({
  projects: [
    {
      name: 'smoke',
      testMatch: '/smoke/**',
      retries: 0
    },
    {
      name: 'regression',
      testMatch: '/regression/**',
      retries: 2
    },
    {
      name: 'e2e',
      testMatch: '/e2e/**',
      timeout: 120000
    }
  ]
});

Organize tests by application features:

export default defineConfig({
  projects: [
    {
      name: 'auth',
      testMatch: '/features/auth/**'
    },
    {
      name: 'checkout',
      testMatch: '/features/checkout/**'
    },
    {
      name: 'dashboard',
      testMatch: '/features/dashboard/**'
    }
  ]
});

Run the same tests in different configurations:

export default defineConfig({
  projects: [
    {
      name: 'logged-out',
      testMatch: '/public/**',
      use: { storageState: undefined }
    },
    {
      name: 'logged-in',
      testMatch: '/authenticated/**',
      dependencies: ['setup-auth'],
      use: { storageState: 'auth.json' }
    }
  ]
});

Test across different browsers and devices:

export default defineConfig({
  projects: [
    {
      name: 'desktop-chrome',
      use: { 
        browserName: 'chromium',
        viewport: { width: 1920, height: 1080 }
      }
    },
    {
      name: 'mobile-safari',
      use: {
        browserName: 'webkit',
        ...devices['iPhone 13']
      }
    }
  ]
});

Running Specific Projects

Once you’ve defined your projects, you can run them selectively:

Locally

# Run a single project
npx playwright test --project=smoke

# Run multiple projects
npx playwright test --project=smoke --project=critical

On Stably Cloud

# Run specific projects in the cloud
npx stably cloud test --project=smoke --project=release

Advanced Project Setup

Setup and Teardown Projects

Use special projects for setup and teardown:

export default defineConfig({
  projects: [
    {
      name: 'setup',
      testMatch: '/setup/**'
    },
    {
      name: 'tests',
      dependencies: ['setup'],
      testMatch: '/tests/**'
    },
    {
      name: 'teardown',
      testMatch: '/teardown/**'
    }
  ]
});

Conditional Project Execution

Use environment variables to conditionally run projects:

export default defineConfig({
  projects: [
    {
      name: 'smoke',
      testMatch: '/smoke/**'
    },
    ...(process.env.RUN_FULL_SUITE ? [{
      name: 'full-regression',
      testMatch: '/regression/**'
    }] : [])
  ]
});

Tagging with Grep

Combine projects with grep patterns for fine-grained control:

// In your test files
test('@p0 critical login flow', async ({ page }) => {
  // test implementation
});

Then run:

npx stably cloud test --project=smoke --grep @p0

Best Practices

Keep Projects Focused

Each project should have a clear purpose and contain related tests. Avoid catch-all projects.

Use Descriptive Names

Name projects after their purpose (smoke, critical) rather than technical details.

Balance Granularity

Too many projects add complexity. Too few limit flexibility. Find the right balance for your team.

Document Your Structure

Add comments in your config explaining the purpose of each project and when to use it.

Next Steps

Running Test Groups

Learn how to run your test groups locally, in the cloud, or in CI/CD pipelines

​Overview

​Defining Projects

​Project Configuration Options

​Common Project Patterns

​Running Specific Projects

​Locally

​On Stably Cloud

​Advanced Project Setup

​Best Practices

Keep Projects Focused

Use Descriptive Names

Balance Granularity

Document Your Structure

​Next Steps

Running Test Groups

Overview

Defining Projects

Project Configuration Options

Common Project Patterns

Running Specific Projects

Locally

On Stably Cloud

Advanced Project Setup

Best Practices

Next Steps