Customization

Flakiness.io reporters share a set of common configuration options across all supported test runners. This page documents each option with examples for every reporter.

Title

Environment variable: FLAKINESS_TITLE

The run title is a human-readable label for the test run. It appears in the Report Viewer header and, when uploaded to the Analytics Platform, in the run list and history views.

Use the title to distinguish runs — for example, by CI job name, target platform, or test suite:

Linux — Playwright Tests
Nightly E2E
PR #1234 Smoke Tests

export default defineConfig({
  reporter: [
    ['@flakiness/playwright', { title: 'Playwright Tests' }]
  ],
});

export default defineConfig({
  test: {
    reporters: [
      ['@flakiness/vitest', { title: 'Unit Tests' }]
    ],
  },
});

export default {
  format: [['@flakiness/cucumberjs', '.flakiness/formatter.log']],
  formatOptions: {
    title: 'BDD Tests',
  },
};

pytest --flakiness-title "Integration Tests"

flakiness convert-junit ./test-results --title "Java Tests"

Dynamic titles in GitHub Actions

Since all reporters support the FLAKINESS_TITLE environment variable, you can set it dynamically in your workflow. This is especially useful for matrix builds where each job should produce a distinctly named report:

jobs:
  test:
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    name: "Tests (${{ matrix.os }}, shard ${{ matrix.shard }}/4)"
    env:
      FLAKINESS_TITLE: "Tests (${{ matrix.os }}, shard ${{ matrix.shard }}/4)"
    steps:
      - uses: actions/checkout@v4
      - run: npx playwright test --shard=${{ matrix.shard }}/4

This produces titles like Tests (ubuntu-latest, shard 2/4) — making it easy to identify each job’s report in the run list.

Flakiness Project

Environment variable: FLAKINESS_PROJECT

The Flakiness.io project identifier in orgSlug/projectSlug format. This connects your report to a project on the Analytics Platform, enabling regression detection, flakiness tracking, and duration trends.

This option is also required for GitHub OIDC authentication — the recommended upload method when running in GitHub Actions.

export default defineConfig({
  reporter: [
    ['@flakiness/playwright', { flakinessProject: 'my-org/my-project' }]
  ],
});

export default defineConfig({
  test: {
    reporters: [
      ['@flakiness/vitest', { flakinessProject: 'my-org/my-project' }]
    ],
  },
});

export default {
  format: [['@flakiness/cucumberjs', '.flakiness/formatter.log']],
  formatOptions: {
    flakinessProject: 'my-org/my-project',
  },
};

pytest --flakiness-project my-org/my-project

flakiness convert-junit ./test-results --project my-org/my-project

Custom Environment Metadata

Environment variables prefixed with FK_ENV_ are automatically included in the report’s environment metadata. The prefix is stripped and the key is converted to lowercase.

This is useful for recording deployment target, region, GPU type, or any other context that varies between test runs.

export FK_ENV_DEPLOYMENT=staging
export FK_ENV_REGION=us-east-1

This produces the following metadata in the report:

{
  "metadata": {
    "deployment": "staging",
    "region": "us-east-1"
  }
}

When uploaded to the Analytics Platform, each unique environment gets its own test history timeline — so tests run with FK_ENV_DEPLOYMENT=staging are tracked separately from FK_ENV_DEPLOYMENT=production.

All reporters support FK_ENV_* variables automatically — no reporter-specific configuration is needed.

Output Folder

Environment variable: FLAKINESS_OUTPUT_DIR

The directory where the reporter writes the Flakiness report. Defaults to flakiness-report in the current working directory.

export default defineConfig({
  reporter: [
    ['@flakiness/playwright', { outputFolder: './test-results/flakiness' }]
  ],
});

export default defineConfig({
  test: {
    reporters: [
      ['@flakiness/vitest', { outputFolder: './test-results/flakiness' }]
    ],
  },
});

export default {
  format: [['@flakiness/cucumberjs', '.flakiness/formatter.log']],
  formatOptions: {
    outputFolder: './test-results/flakiness',
  },
};

pytest --flakiness-output-dir ./test-results/flakiness

flakiness convert-junit ./test-results --output-dir ./test-results/flakiness