Log in Get started
Metric · DORA

What is change failure rate, and how do you measure it in Metabase?

Change failure rate is the share of production deployments that cause a failure — one needing a rollback, hotfix, or incident. It's a DORA stability metric that balances the throughput metrics (deployment frequency and lead time for changes). Measure it in Metabase by joining deploys from GitHub or GitLab to incident data.

TL;DR — Failed production deploys ÷ total production deploys. A "failure" means it required remediation (rollback/hotfix/incident), not just a red CI run.

What change failure rate measures

It's the percentage of deployments to production that result in degraded service. It guards against optimizing speed at the expense of quality — a rising frequency with a rising failure rate is a warning, not a win. Report monthly to smooth out small denominators.

  • Numerator: deploys linked to an incident, rollback, or hotfix.
  • Denominator: all successful production deploys.
  • A failed CI run that never reached production is not a change failure.

What data does it need?

  • deployments: environment, status, created_at, repository_id.
  • A way to mark failures: an incidents table with deployment_id, or is_rollback/is_hotfix flags on deploys. Incident data usually comes from PagerDuty, Opsgenie, incident.io, or a labeled issue.

SQL patterns

Change failure rate by month (incident-linked)PostgreSQL
SELECT
  date_trunc('month', d.created_at) AS month,
  COUNT(*)                                          AS deployments,
  COUNT(DISTINCT i.deployment_id)                   AS failed_deployments,
  COUNT(DISTINCT i.deployment_id)::numeric
    / NULLIF(COUNT(*), 0)                           AS change_failure_rate
FROM deployments d
LEFT JOIN incidents i ON i.deployment_id = d.id
WHERE d.environment = 'production'
  AND d.status = 'success'
GROUP BY 1
ORDER BY 1;
Rollback/hotfix fallbackPostgreSQL
-- Fallback when incidents aren't linked: count deploys followed
-- by a rollback or hotfix deploy within 1 hour
SELECT
  date_trunc('month', d.created_at) AS month,
  AVG(
    CASE WHEN EXISTS (
      SELECT 1 FROM deployments r
      WHERE r.repository_id = d.repository_id
        AND r.environment = 'production'
        AND (r.is_rollback OR r.is_hotfix)
        AND r.created_at BETWEEN d.created_at AND d.created_at + INTERVAL '1 hour'
    ) THEN 1.0 ELSE 0.0 END
  ) AS change_failure_rate
FROM deployments d
WHERE d.environment = 'production' AND d.status = 'success'
GROUP BY 1
ORDER BY 1;

Pitfalls

Counting CI failures.→ Only count failures that hit production and needed remediation.
Tiny denominators.→ With few deploys, use monthly windows or a rolling rate so one incident doesn't swing the number.
Double-counting one incident.→ Attribute each failure to a single deploy; COUNT(DISTINCT deployment_id).
No link between deploy and incident.→ Without a join key, fall back to rollback/hotfix proximity — but invest in the link.

Where this metric applies

Metrics

DORA metrics overviewTime to restore (MTTR)Deployment frequencyLead time for changes

Integrations

GitHub + MetabaseGitLab + Metabase

Dashboards

Software delivery dashboard

FAQ

What counts as a failure?
A production deployment that degrades service and requires remediation — a rollback, hotfix, patch, or triggers an incident. A red CI run that blocks the deploy doesn't count.
How do I link a deploy to an incident?
Best: a deployment_id (or release tag/SHA) on the incident record. Fallback: a rollback/hotfix deploy shortly after the original release.
What's a good change failure rate?
DORA elite/high performers sit around ≤15%. Treat bands as directional — see the DORA overview.