48.4k Log in Get started
Metric · CI/CD

What is build success rate, and how do you measure it in Metabase?

Build success rate is the share of completed CI runs that succeeded — successful runs divided by all completed runs, per pipeline and per branch. It's the baseline health check for a delivery pipeline: when it slips, everything downstream — deployment frequency, lead time for changes — slips with it. Measure it in Metabase from run history synced from Jenkins, CircleCI, Buildkite, or GitHub Actions.

TL;DR — successful runs ÷ all completed runs, per pipeline and per branch. Canceled and skipped runs stay out of the denominator, and the main-branch rate — not the blended number — is the headline.

What does build success rate measure?

It measures whether the pipeline is doing its job or getting in the way. Read it in segments, because the blended number hides both stories:

  • Main branch — the headline. Main should be green the vast majority of the time; every red run there blocks the whole team and stalls deployment frequency.
  • PR branches — failures here are the system working. CI exists to catch broken changes before merge, so a lower PR-branch rate is expected and healthy.
  • First attempt vs. after retry — a run that fails and then passes on the same commit succeeded by luck, not by correctness. The gap between first-attempt and after-retry success rates is a direct read on your flaky test rate.

Trend it weekly. Day-to-day the rate is noisy — one broken dependency bump can sink an afternoon — but a weekly trend per pipeline shows whether the suite is getting more or less trustworthy, and it belongs next to the DORA metrics in any software delivery analytics setup.

What data does it need?

  • A pipeline_runs table: pipeline_name, branch, result, started_at, duration_seconds.
  • A retry link — retry_of_run_id or a rebuild flag — so first-attempt success can be separated from success-after-retry.
  • A result taxonomy that distinguishes failure from canceled and skipped; only completed runs belong in the metric.

SQL patterns

Main-branch success rate by weekPostgreSQL
SELECT
  date_trunc('week', started_at) AS week,
  COUNT(*) AS completed_runs,
  ROUND(
    100.0 * COUNT(*) FILTER (WHERE result = 'success')
    / NULLIF(COUNT(*), 0), 1
  ) AS success_rate_pct
FROM pipeline_runs
WHERE branch = 'main'
  AND result IN ('success', 'failure')
GROUP BY 1
ORDER BY 1;
Success rate by pipeline (trailing 30 days)PostgreSQL
SELECT
  pipeline_name,
  COUNT(*) AS completed_runs,
  COUNT(*) FILTER (WHERE result = 'failure') AS failures,
  ROUND(
    100.0 * COUNT(*) FILTER (WHERE result = 'success')
    / NULLIF(COUNT(*), 0), 1
  ) AS success_rate_pct
FROM pipeline_runs
WHERE started_at >= CURRENT_DATE - INTERVAL '30 days'
  AND result IN ('success', 'failure')
GROUP BY pipeline_name
HAVING COUNT(*) >= 10
ORDER BY success_rate_pct;
First-attempt vs. after-retry success on mainPostgreSQL
SELECT
  date_trunc('week', started_at) AS week,
  ROUND(
    100.0 * COUNT(*) FILTER (
      WHERE result = 'success' AND retry_of_run_id IS NULL
    )
    / NULLIF(COUNT(*) FILTER (WHERE retry_of_run_id IS NULL), 0), 1
  ) AS first_attempt_rate_pct,
  ROUND(
    100.0 * COUNT(*) FILTER (WHERE result = 'success')
    / NULLIF(COUNT(*), 0), 1
  ) AS all_attempts_rate_pct
FROM pipeline_runs
WHERE branch = 'main'
  AND result IN ('success', 'failure')
GROUP BY 1
ORDER BY 1;

Pitfalls

Counting canceled runs as failures.→ A canceled run says nothing about the code — someone pushed a newer commit or stopped a duplicate. Scope the denominator to completed runs (result IN ('success', 'failure')) or the rate swings with push cadence, not code quality.
Blending PR-branch and main-branch runs.→ PR failures are CI doing its job; main failures are outages for the team. One blended number makes a healthy repo with strict checks look broken and a broken main look tolerable. Always segment by branch.
Celebrating a rate propped up by retries.→ If everyone hammers the retry button until the build goes green, the after-retry rate looks great while every merge gets slower and flaky tests multiply unpunished. First-attempt success rate is the honest number.

Where this metric applies

Metrics

Pipeline durationFlaky test rateDORA metrics overview

Dashboards

CI pipeline health dashboardDORA metrics dashboard

FAQ

How do you calculate build success rate?
Divide successful runs by all completed runs: COUNT(*) FILTER (WHERE result = 'success') / COUNT(*), scoped to result IN ('success', 'failure') so canceled and skipped runs never enter the denominator. Compute it per pipeline and per branch — the main-branch number is the one that belongs on a CI pipeline health dashboard — and trend it weekly rather than staring at a single day's runs.
What is a good build success rate?
There is no universal target — pipelines, test suites, and merge policies differ too much for a cross-team benchmark to mean anything. Trend the rate within a team instead: the main branch should be green the large majority of the time, because a red main blocks every developer at once, while PR-branch runs are allowed to fail — that is what they are for. If main-branch health is slipping, read it next to pipeline duration and flaky test rate to find whether the cause is broken code or unreliable tests.
How is build success rate different from change failure rate?
They sit on opposite sides of the release. Build success rate measures failures caught before production — a failed CI run costs a developer some time and nothing more. Change failure rate, one of the four DORA metrics, measures deployments that caused failures in production. A healthy pipeline fails builds so that changes don't fail later; a falling build success rate paired with a falling change failure rate can even be a good trade.
Do builds that pass after a retry count as successes?
Count them, but separately. A run that failed and then passed on retry against the same commit is a success for the team's throughput and a red flag for test reliability — by construction it points at a flaky test or unstable infrastructure. Track first-attempt success rate next to the after-retry rate: a wide gap between the two is the clearest quantitative signal of flakiness you can get from run data alone.
How do you track build success rate in Metabase?
CI tools don't expose a SQL interface directly, so sync run history from Jenkins, CircleCI, Buildkite, or GitHub Actions into a database via their APIs or an ETL tool, model a pipeline_runs table with pipeline, branch, result, and retry lineage, and define the rate once as a saved question. Put the main-branch trend and the per-pipeline table on a CI pipeline health dashboard next to pipeline duration.