48.9k Log in Get started
Dashboard

What goes in a data science dashboard in Metabase?

A data science dashboard is the program view of an ML practice: model performance and drift over time, experiment velocity, prediction volume, feature freshness, and the model inventory by stage. It's built from the model registry, prediction logs, and experiment trackers like Weights & Biases or Braintrust, landed in the warehouse and queried with plain SQL.

For: data science leads and the teams depending on model output. Grain: one row per prediction, per experiment run, and per registry entry. Source: model registry, prediction logs joined to outcomes, and experiment-tracker exports.

What does a data science dashboard look like?

Here’s the layout this guide builds. Program-level counts sit at the top — models in production, prediction volume, open drift alerts; performance and drift trends come next because they are the cards that page someone; experiment throughput, serving volume, and the feature-freshness table sit at the bottom for the weekly program review.

Data science dashboard in Metabase showing model performance, feature drift, experiment velocity, prediction volume, and model inventory.
An example data science dashboard in Metabase, built from model registry, prediction, and experiment data. Figures are illustrative.

Which cards belong on a data science dashboard?

The eight below cover model health, input health, and program throughput — performing models, fresh features, and experiments that actually conclude.

  • Model performance over time — weekly AUC per production model (line)
  • Feature drift — PSI for the most-drifted features, against the 0.2 threshold (line)
  • Days since last retrain, by model (row)
  • Drift alerts opened per week (bar)
  • Experiments started versus concluded, monthly (bar)
  • Prediction volume per day (area)
  • Model inventory by stage — production, staging, development, retired (donut)
  • Feature-group freshness — refresh SLA versus last refresh (table)

What data does the dashboard need?

  • A prediction log — prediction_id, model, stage, predicted label, confidence, and timestamp — joined to an outcomes table as ground truth arrives.
  • The model registry: model, version, lifecycle stage, and last_trained_at, synced from an MLflow-style store.
  • Per-feature drift scores (PSI against the training baseline), computed on a schedule by a monitoring job.
  • Experiment-tracker exports — run, start and conclusion dates, and outcome (shipped, negative, inconclusive).
  • A feature-group reference table with refresh SLAs, plus each group’s actual last-refresh timestamp.

How do you build it?

  1. Log every production prediction to the warehouse with model, version, confidence, and features hash — and backfill outcomes into ml.outcomes as labels arrive.
  2. Sync the registry and experiment tracker nightly, so stage changes, retrain dates, and experiment conclusions are queryable tables.
  3. Schedule the drift job: compute PSI per feature against each model’s training baseline and write one row per feature per day.
  4. Build the eight cards, cohorting accuracy by prediction date with an awaiting-labels count so young cohorts aren’t misread.
  5. Add filters for model, stage, and feature group, then subscribe the DS channel to a weekly snapshot for the program review.

Example card SQL

Weekly production accuracy per model, with label lag made visible PostgreSQL 
SELECT
DATE_TRUNC('week', p.predicted_at)                    AS week,
p.model_name,
COUNT(*)                                              AS predictions,
ROUND(AVG(CASE WHEN o.actual_label = p.predicted_label
          THEN 1.0 ELSE 0.0 END), 3)                  AS accuracy,
ROUND(AVG(p.confidence), 3)                           AS avg_confidence,
COUNT(*) FILTER (WHERE o.actual_label IS NULL)        AS awaiting_labels
FROM ml.predictions p
LEFT JOIN ml.outcomes o USING (prediction_id)
WHERE p.predicted_at >= CURRENT_DATE - INTERVAL '12 weeks'
AND p.model_stage = 'production'
GROUP BY 1, 2
ORDER BY 1, 2;

Metrics

Eval pass rate Data freshness Error rate p95 LLM latency

Integrations

Weights & Biases + Metabase Braintrust + Metabase Arize Phoenix + Metabase Databricks data source

Dashboards

Eval quality tracking LLM latency & errors Model usage & cost Data warehouse

FAQ

What is a data science dashboard?
A data science dashboard is the program view of an ML practice: how production models are performing over time, whether their input features are drifting or going stale, how fast experiments move from started to concluded, how much prediction volume the models serve, and what the model inventory looks like by lifecycle stage. It is read by DS leads and the teams that depend on model output — a shared answer to "are the models healthy and is the practice shipping", built from registry, prediction, and monitoring data landed in the warehouse.
Where does the data come from?
Three places, all landable in a warehouse. The model registry — an MLflow-style store with each model's stage, version, and last-trained date. The prediction log — one row per scored request with model, features hash, prediction, confidence, and timestamp, joined later to actual outcomes. And the experiment tracker — platforms like Weights & Biases or Arize Phoenix expose runs and evaluations through APIs you can sync nightly. Metabase then queries all three with plain SQL, next to the business tables the models affect.
How do I track production model performance when labels arrive late?
Cohort predictions by the date they were made, and let each cohort's accuracy fill in as ground truth arrives — churn labels land 30 days later, fraud chargebacks 60+. The card should always show an awaiting_labels count so nobody misreads a young cohort's early numbers as a crash. In between, watch leading indicators that need no labels at all: score-distribution shifts, confidence drops, and feature drift. A model whose inputs moved is usually a model whose accuracy is about to move — that ordering is why the drift card sits next to the performance card.
What drift metric should I use, and what threshold?
Population stability index is the workhorse: compare each feature's current distribution against its training baseline. The usual reading is under 0.1 stable, 0.1–0.25 worth watching, above 0.25 significant drift — but treat those as triage buckets, not truth. Drift is a symptom, not a verdict: a feature can drift because the business genuinely changed (a pricing change shifts every transaction feature at once) and the model may still perform. So alert on drift, then check the performance cohorts before retraining. Chart PSI over time for the top-drifting features rather than a single snapshot — the trajectory tells you whether it's a step change or a slide.
Why track experiments started and concluded separately?
Because the gap between them is the program's real bottleneck. Starting experiments is cheap; concluding them — a decision to ship, kill, or iterate — is where value gets realized, and where queues form. A month with 18 starts and 9 conclusions means work in progress is piling up, usually in review or in waiting-for-significance limbo. Track conclusions by outcome too (shipped, negative, inconclusive): a healthy program kills things — an 80% ship rate usually means experiments are confirmatory theater rather than genuine bets, while a high inconclusive rate points at underpowered designs.
Why does feature freshness get its own card?
Because stale features degrade models silently. If the pipeline feeding days_since_last_order stops updating, the scoring service keeps serving — same latency, no errors — while every prediction quietly uses yesterday's world. That is training-serving skew in its most common form, and no accuracy card will catch it until labels arrive weeks later. Give every feature group a refresh SLA in a reference table and chart last-refresh against it, exactly like a data freshness check for ETL — because that is what it is, with a model instead of a dashboard downstream.
We mostly ship LLM features now — is this still the right dashboard?
Keep this one for the classic ML estate and add the LLM-specific views, because the failure modes differ. LLM applications are evaluated with rubric and judge scores rather than AUC — that's the eval quality tracking dashboard — and their operational health is latency and token cost, covered by LLM latency & errors and model usage & cost. The program-level cards here — inventory by stage, experiment velocity, feature freshness for anything retrieval-augmented — apply to both estates unchanged.