48.3k Log in Get started
Metric · DORA

What is change failure rate, and how do you measure it in Metabase?

Change failure rate is the share of production deployments that cause a failure — one needing a rollback, hotfix, or incident. It's a DORA stability metric that balances the throughput metrics (deployment frequency and lead time for changes). Measure it in Metabase by joining deploys from GitHub or GitLab to incident data.

TL;DR — Failed production deploys ÷ total production deploys. A "failure" means it required remediation (rollback/hotfix/incident), not just a red CI run.

What change failure rate measures

It's the percentage of deployments to production that result in degraded service. It guards against optimizing speed at the expense of quality — a rising frequency with a rising failure rate is a warning, not a win. Report monthly to smooth out small denominators.

  • Numerator: deploys linked to an incident, rollback, or hotfix.
  • Denominator: all successful production deploys.
  • A failed CI run that never reached production is not a change failure.

What data does it need?

  • deployments: environment, status, created_at, repository_id.
  • A way to mark failures: an incidents table with deployment_id, or is_rollback/is_hotfix flags on deploys. Incident data usually comes from PagerDuty, Opsgenie, incident.io, or a labeled issue.

SQL patterns

Change failure rate by month (incident-linked)PostgreSQL
SELECT
  date_trunc('month', d.created_at) AS month,
  COUNT(*)                                          AS deployments,
  COUNT(DISTINCT i.deployment_id)                   AS failed_deployments,
  COUNT(DISTINCT i.deployment_id)::numeric
    / NULLIF(COUNT(*), 0)                           AS change_failure_rate
FROM deployments d
LEFT JOIN incidents i ON i.deployment_id = d.id
WHERE d.environment = 'production'
  AND d.status = 'success'
GROUP BY 1
ORDER BY 1;
Rollback/hotfix fallbackPostgreSQL
-- Fallback when incidents aren't linked: count deploys followed
-- by a rollback or hotfix deploy within 1 hour
SELECT
  date_trunc('month', d.created_at) AS month,
  AVG(
    CASE WHEN EXISTS (
      SELECT 1 FROM deployments r
      WHERE r.repository_id = d.repository_id
        AND r.environment = 'production'
        AND (r.is_rollback OR r.is_hotfix)
        AND r.created_at BETWEEN d.created_at AND d.created_at + INTERVAL '1 hour'
    ) THEN 1.0 ELSE 0.0 END
  ) AS change_failure_rate
FROM deployments d
WHERE d.environment = 'production' AND d.status = 'success'
GROUP BY 1
ORDER BY 1;

Pitfalls

Counting CI failures.→ Only count failures that hit production and needed remediation.
Tiny denominators.→ With few deploys, use monthly windows or a rolling rate so one incident doesn't swing the number.
Double-counting one incident.→ Attribute each failure to a single deploy; COUNT(DISTINCT deployment_id).
No link between deploy and incident.→ Without a join key, fall back to rollback/hotfix proximity — but invest in the link.

Where this metric applies

Metrics

DORA metrics overviewTime to restore (MTTR)Deployment frequencyLead time for changes

Integrations

GitHub + MetabaseGitLab + MetabaseSentry + MetabasePagerDuty + Metabase

Dashboards

Software delivery dashboardRelease quality dashboard

FAQ

What is change failure rate?
Change failure rate is the share of production deployments that cause a failure needing remediation — a rollback, hotfix, or incident. It is one of the two DORA stability metrics, balancing the throughput pair of deployment frequency and lead time for changes. Compute it as failed production deploys divided by total successful production deploys, reported monthly to smooth out small denominators. See the DORA overview for how the four keys fit together.
What counts as a failure?
A production deployment that degrades service and requires remediation — a rollback, hotfix, patch, or a triggered incident. A red CI run that blocks a deploy from ever reaching production doesn't count, and neither do staging failures. Keeping the definition tight matters because the numerator drives the whole metric. Pair it with MTTR so you see how often changes fail and how fast you recover, and track regressions per release on a release quality dashboard.
How do I link a deploy to an incident?
Best: store a deployment_id (or release tag/SHA) on the incident record when it's opened in PagerDuty or your incident tool. Fallback: infer failures from a rollback or hotfix deploy shortly after the original release — the second SQL pattern above does this with a one-hour window. Deploy records come from GitHub Deployments/Actions or GitLab environments; attribute each incident to a single deploy to avoid double-counting.
What's a good change failure rate?
DORA's elite and high performers sit around 15% or lower, medium runs roughly 16–30%, and low is above 30%. Treat the bands as directional, not a scoreboard — thresholds shift with each annual report, and small deploy counts make the rate noisy. Compare a team against its own trend, and read it next to deployment frequency: rising frequency with a rising failure rate is a warning, not a win. See the DORA overview for the full bands.
How do you track change failure rate in Metabase?
Sync deploys from GitHub or GitLab and incidents from your on-call tool into a SQL database, model deployments and incidents tables with a join key, and define the rate once as a saved question — failed deploys over total production deploys by month. The SQL patterns above cover both the incident-linked and rollback-fallback versions. Put the result on a software delivery dashboard next to the other DORA keys.