48.9k Log in Get started
Dashboard

What goes in a real-time analytics dashboard in Metabase?

A real-time analytics dashboard tracks event throughput, consumer lag, end-to-end latency percentiles, late and dropped events, top topics, and sink freshness for a Kafka-style streaming platform. Metabase doesn't consume the stream itself — it queries the stores the stream lands in, like ClickHouse or Druid, plus broker metrics landed as one-minute rollups.

For: streaming platform engineers and every team consuming the stream. Grain: one-minute rollups of broker, consumer, and sink metrics. Refresh: auto-refresh at one minute — freshness is refresh interval plus sink latency.

What does a real-time analytics dashboard look like?

Here’s the layout this guide builds. Current throughput, lag, and latency sit at the top because streaming incidents unfold in minutes; the throughput and latency trends come next to show whether the platform is keeping up; topics, delivery SLO, and per-sink freshness fill the bottom for diagnosis.

Real-time analytics dashboard in Metabase showing event throughput, consumer lag, latency percentiles, late events, top topics, and sink freshness.
An example real-time analytics dashboard in Metabase, built from streaming metrics landed in a real-time store. Figures are illustrative.

Which cards belong on a real-time analytics dashboard?

The eight below cover flow, delay, and correctness — is data moving, how far behind is it, and is it right.

  • Events per second by topic group (line)
  • End-to-end latency, p50 / p95 / p99, hourly (line)
  • Consumer lag by group, hourly (line)
  • Late and dropped events per hour (stacked bar)
  • Top topics by daily volume (row)
  • Events delivered within the 5-second SLO (gauge)
  • Consumer group rebalances per day (bar)
  • Sink freshness — last event and p95 lag per sink (table)

What data does the dashboard need?

  • Broker metrics — messages in per topic, bytes, and partition counts — from the platform’s metrics exporter, rolled up per minute.
  • Consumer group offsets and computed lag per group and topic, sampled on the same schedule.
  • Event-level timestamps in the sink: producer event_ts and sink ingested_at, for latency percentiles and late-event counts.
  • A sink registry — sink, type, freshness SLO — so the freshness table is a join against live watermarks.
  • Consumer group lifecycle events (rebalances, restarts) if the platform exposes them; rebalance storms explain most lag mysteries.

How do you build it?

  1. Land broker and consumer metrics into your real-time store as one-minute rollups — a small consumer job or the metrics exporter’s remote-write does this continuously.
  2. Make sure every event carries a producer timestamp, and record ingestion time in the sink — the latency and late-event cards are differences between the two.
  3. Connect the store to Metabase (see the ClickHouse guide) and build shared models for the rollups: one for flow, one for lag, one for event timing.
  4. Build the eight cards, converting lag to time-to-drain in SQL, and derive sink status from each sink’s own SLO rather than a global threshold.
  5. Add filters for cluster, topic, and consumer group, set auto-refresh to one minute, and alert the platform channel when lag growth or the late share crosses its threshold.

Example card SQL

End-to-end latency percentiles and late events per minute PostgreSQL 
SELECT
toStartOfMinute(ingested_at)                          AS minute,
count()                                               AS events,
round(quantile(0.50)(
  dateDiff('millisecond', event_ts, ingested_at)) / 1000.0, 2)
                                                      AS p50_s,
round(quantile(0.95)(
  dateDiff('millisecond', event_ts, ingested_at)) / 1000.0, 2)
                                                      AS p95_s,
round(quantile(0.99)(
  dateDiff('millisecond', event_ts, ingested_at)) / 1000.0, 2)
                                                      AS p99_s,
countIf(event_ts < ingested_at - INTERVAL 5 MINUTE)   AS late_events
FROM events.stream_landing
WHERE ingested_at >= now() - INTERVAL 24 HOUR
GROUP BY minute
ORDER BY minute;

Metrics

Data freshness Error rate Uptime Queue time

Integrations

ClickHouse data source Druid data source Build a data pipeline

Dashboards

ETL monitoring Application monitoring CDP sync health Data warehouse

FAQ

What is a real-time analytics dashboard?
A real-time analytics dashboard tracks the health of a streaming platform: how many events flow per second, whether consumers are keeping up, how long an event takes from production to being queryable, and whether events arrive late or get dropped on the way. It is the shared answer to "is the stream healthy and is the data current" — for the platform team that runs the brokers and for every team whose dashboards sit on top of the stream's output.
Can Metabase read Kafka directly?
No — Metabase queries databases, not broker protocols, and that turns out to be the right architecture anyway. Streams become analyzable where they land: a real-time OLAP store like ClickHouse or Druid ingests topics continuously and answers SQL in milliseconds, which is exactly what dashboard cards need. Broker-side metrics — throughput, consumer offsets, rebalances — come from the platform's metrics exporter, landed in the same store as one-minute rollups. Every card on this dashboard is then plain SQL.
How do I measure end-to-end latency?
Stamp events with a producer-side event_ts, record an ingested_at when the sink writes them, and chart percentiles of the difference. Percentiles, not averages: streaming latency is long-tailed, and an average hides the p99 spike that pages someone. The p50 tells you the pipeline's cruising speed, p95 what most consumers experience, p99 what breaks SLOs. Watch for the two failure shapes: all percentiles rising together means the pipeline is saturated; p99 alone rising means one partition, broker, or consumer is struggling while the rest keep up.
How much consumer lag is too much?
Messages of lag alone mean little — 1.9 million messages is minutes of lag for a group consuming 20k/s and a day for one consuming 20/s. Convert lag to time: divide by the group's consumption rate, and alert when time-to-drain exceeds the freshness promise of whatever the consumer feeds. The trend matters more than the level: flat lag at any size is a steady state, while monotonically growing lag means the consumer has fallen behind its partition's production rate and will never catch up without scaling — the chart's job is to make that divergence visible within minutes of a bad deploy.
What should I do about late and dropped events?
First separate them, because the causes differ. Late events — event time far behind arrival time — usually trace to producer clock skew, mobile clients flushing buffered events, or a stalled upstream; they corrupt time-windowed aggregates silently, so track the late share and rebuild affected windows past your allowed-lateness threshold. Dropped events — sent but never landed — point to serialization failures or full dead-letter queues, and belong near zero. The card exists because both failure modes leave every run "green": the pipeline works, the data is quietly wrong.
How fresh can a Metabase dashboard actually be?
Set the dashboard to auto-refresh at one minute, and the freshness you see is refresh interval plus sink latency — with a ClickHouse or Druid sink ingesting continuously, that means figures roughly a minute old, which is what "real time" means for human decision-making. Metabase is not the tool for sub-second machine reactions; that's what stream processors and alerting are for. For a wall display, use a dedicated dashboard with a small number of fast cards so the refresh doesn't hammer the store with expensive queries every 60 seconds.
How is this different from an application monitoring dashboard?
Subject and unit. An application monitoring dashboard watches services — request rates, error rates, saturation — and its unit is the request. This dashboard watches the event pipeline — topics, consumers, sinks — and its unit is the event, with data correctness (late, dropped, stale) as a first-class failure mode alongside availability. They meet during incidents: a consumer-lag spike here often shows up as elevated latency there. Keep both, and link the lag card's drill-through to the affected service's view.