Commands Reference

This page documents all Flakiness CLI commands. For authentication commands, see Authentication.

show

View a local Flakiness report in the browser.

flakiness show [path]

Arguments:

• [path] — Path to a Flakiness folder containing report.json (default: flakiness-report)

Options:

• -e, --viewer-url <url> — Report viewer URL (default: https://report.flakiness.io, env: FLAKINESS_ENDPOINT)

Example:

flakiness show
flakiness show ./run-123

access Auth required

Check your access to a Flakiness.io project.

flakiness access --project <org/project>

Options:

• --project <org/project> — Flakiness.io project slug, required (env: FLAKINESS_PROJECT)
• -t, --access-token <token> — Access token (env: FLAKINESS_ACCESS_TOKEN)
• -e, --endpoint <url> — Service endpoint (default: https://flakiness.io, env: FLAKINESS_ENDPOINT)
• --json — Output result as JSON
• -q, --quiet — Suppress output, only set exit code

Exits with code 1 if access is denied.

Example:

flakiness access --project myorg/myproject

upload Auth required

Upload Flakiness report(s) to the Flakiness.io service.

flakiness upload <relative-paths...>

Arguments:

• <relative-paths...> — One or more paths to Flakiness report directories

By default, prints the run URL for each uploaded report to stdout. Use --progress to show a spinner with upload progress instead.

Options:

• --project <org/project> — Flakiness.io project slug; overrides the project embedded in the report (env: FLAKINESS_PROJECT)
• -t, --access-token <token> — Access token (env: FLAKINESS_ACCESS_TOKEN)
• -e, --endpoint <url> — Service endpoint (default: https://flakiness.io, env: FLAKINESS_ENDPOINT)
• --progress — Show a progress spinner instead of printing run URLs

Upload Method	Status	Guide
GitHub OIDC	Supported	GitHub Actions
Access Token	Supported	GitHub Actions
Other CI	Supported	Other CI Providers

Tip

GitHub OIDC is the recommended method — no secrets to manage. It requires the report to contain a flakinessProject field, which is set automatically by reporters (e.g. Playwright, Pytest) or via convert-junit --project. See the GitHub Actions guide for setup instructions.

Examples:

# Upload a report — prints the run URL
flakiness upload ./flakiness-report

# Upload multiple reports with a progress spinner
flakiness upload --progress ./report-1 ./report-2 ./report-3

download Auth required

Download Flakiness reports from the Flakiness.io service.

flakiness download --project <org/project>

Options:

• --project <org/project> — Flakiness.io project slug, required (env: FLAKINESS_PROJECT)
• --run-id <runId> — Download a specific run by its ID
• --since <date> — Download all runs uploaded since the specified date
• -j, --parallel <number> — Number of parallel downloads (default: 1)
• -t, --access-token <token> — Access token (env: FLAKINESS_ACCESS_TOKEN)
• -e, --endpoint <url> — Service endpoint (default: https://flakiness.io, env: FLAKINESS_ENDPOINT)

Downloaded reports are saved to run-<runId>/ directories, each containing:

• report.json — The Flakiness report file
• attachments/ — Directory with all attachments (if any)

Already-downloaded runs are skipped automatically.

Examples:

# Download a specific run
flakiness download --project myorg/myproject --run-id 123

# Download all runs since a date
flakiness download --project myorg/myproject --since 2024-01-01

# Download with parallel processing
flakiness download --project myorg/myproject --since 2024-01-01 --parallel 4

list tests Auth required

Query test data from the Flakiness.io platform.

flakiness list tests --project <org/project> [options]

By default, queries the default branch and shows tests from the day of its head commit (in the project’s timezone). Use --pr to query a specific pull request (shows tests from the merge-commit of the PR into the target branch), or --branch to query a named branch (shows tests from the day of its head commit).

When querying a PR with --pr, the status values have specific meaning:

• regressed — test was passing on the target branch but fails in this PR (caused by the PR)
• failed — test also fails on the target branch (pre-existing failure, not caused by the PR)
• flaked — test failed but passed on retry (flaky)

Options:

• --project <org/project> — Flakiness.io project slug, required (env: FLAKINESS_PROJECT)
• --pr <number> — Show tests from a specific pull request (mutually exclusive with --branch)
• --branch <name> — Show tests from a specific branch (mutually exclusive with --pr)
• --fql <query> — FQL filter expression
• --sort <axis> — Sort axis: outcome, flip_rate, duration, duration_trend, name (default: outcome)
• --sort-dir <dir> — Sort direction: asc or desc (default: desc)
• --page <n> — Page number, 1-based (default: 1)
• --page-size <n> — Tests per page, 1–300 (default: 20)
• --env-name <value> — Filter by environment name (repeatable)
• --env-path <value> — Filter by config file path (repeatable)
• --env-category <value> — Filter by test category (repeatable)
• --env-os <value> — Filter by OS name + version (repeatable)
• --env-arch <value> — Filter by CPU architecture (repeatable)
• --env-metadata <key=value> — Filter by user-supplied environment metadata (repeatable)
• -t, --access-token <token> — Access token (env: FLAKINESS_ACCESS_TOKEN)
• -e, --endpoint <url> — Service endpoint (default: https://flakiness.io, env: FLAKINESS_ENDPOINT)

All --env-* flags are repeatable. Multiple values for the same flag are OR’d (match any); different flags are AND’d (must match all).

Output includes each test’s name, file location, environment (with key=value pairs matching the --env-* flag suffixes), user-supplied metadata (if any), status, duration, flip rate, and errors.

Examples:

# List all tests sorted by outcome (default branch)
flakiness list tests --project myorg/myproject

# Find regressions caused by PR #42
flakiness list tests --project myorg/myproject --pr 42 --fql 's:regressed'

# Show all broken tests in a PR (regressions + pre-existing failures)
flakiness list tests --project myorg/myproject --pr 42 --fql 'status:(failed, regressed)'

# Show tests from a feature branch
flakiness list tests --project myorg/myproject --branch feature/login

# Find flaky tests
flakiness list tests --project myorg/myproject --fql 'flip>0%' --sort flip_rate --sort-dir desc

# Find failed tests in e2e files
flakiness list tests --project myorg/myproject --fql 's:failed f:e2e'

# Find slow tests
flakiness list tests --project myorg/myproject --fql 'd>5s' --sort duration --sort-dir desc

# Find tests by error message
flakiness list tests --project myorg/myproject --fql '$timeout'

# Combine filters: failed e2e tests, excluding smoke-tagged
flakiness list tests --project myorg/myproject --fql 's:failed f:e2e -#smoke'

# Filter by OS
flakiness list tests --project myorg/myproject --env-os "Ubuntu 22.04"

# Filter by environment name
flakiness list tests --project myorg/myproject --env-name chromium

# Multiple OSes (OR) combined with FQL (AND)
flakiness list tests --project myorg/myproject --env-os "Ubuntu 22.04" --env-os "Darwin 24.0" --fql 's:failed'

# Filter by user-supplied metadata
flakiness list tests --project myorg/myproject --env-metadata browser=firefox

See the FQL reference for the full filter syntax.

convert-junit

Convert JUnit XML report(s) to Flakiness report format.

flakiness convert-junit <junit-root-dir-path>

Arguments:

• <junit-root-dir-path> — Path to JUnit XML file or directory containing XML files

Options:

• --output-dir <dir> — Output directory for the report (default: flakiness-report)
• --env-name <name> — Environment name for the report (default: junit)
• --commit-id <id> — Git commit ID (auto-detected if not provided)
• --project <org/project> — Flakiness.io project slug for OIDC uploads (env: FLAKINESS_PROJECT)

Tip

Set --project (or FLAKINESS_PROJECT env var) to enable GitHub OIDC uploads without an access token.

Example:

flakiness convert-junit ./test-results --env-name ci --output-dir ./flakiness-report
flakiness upload ./flakiness-report/report.json

See the Migrating from JUnit XML guide for more details.

skills install

Install bundled skills for AI coding agents. By default, skills are installed into the user home directory (e.g. ~/.claude/skills/), making them available across all projects. See Agentic Workflows for details.

flakiness skills install --agent <claude|codex|cursor>

Options:

• --agent <agent> — Target agent: claude, codex, or cursor
• --project — Install into the project directory instead of the user home directory