48.4k Log in Get started
Dashboard

What goes in a model usage and cost dashboard?

A model usage and cost dashboard answers the model-portfolio questions: which models handle the traffic, what each one really costs per token, and whether the mix is drifting toward expensive defaults. It's the page that turns 'should we switch models?' from a debate into a query.

For: AI engineers, platform teams, and engineering leadership. Refresh: daily. Source: modeled LLM usage tables in a Metabase-connected database.

Which cards belong on this dashboard?

  • Requests by model (stacked bar over time)
  • Prompt vs. completion tokens by model (stacked bar)
  • Cost per 1M tokens by model (bar)
  • Model mix share over time (area)
  • Cache hit rate by model or app (line)
  • Traffic still on deprecated or replaced models (table)
  • Reasoning-token share where models report it (line)
  • Cost per request by model (table)

What data does this dashboard need?

  • Daily rollups per model: requests, prompt/completion (and reasoning) tokens, cost
  • Model metadata: provider, family, deprecation status, list price
  • Cache hit counts or cached-token fields where the gateway records them
  • App tags so model mix can be split by workload

How do you build it?

  1. Sync usage rollups from your LLM gateway, tracing, or eval tool into a database, keeping stable IDs, timestamps, token counts, and costs.
  2. Build clean models at the grain this dashboard requires — daily per app and model is the workhorse.
  3. Create one saved question per card and certify the shared models and metric definitions.
  4. Add dashboard filters for Date range, model, provider, app, environment.
  5. Exclude test, staging, and playground traffic from headline cards, and show the refresh time.

Example card SQL

Cost per 1M tokens and traffic share by model, trailing 30 daysPostgreSQL
WITH totals AS (
  SELECT
    model,
    SUM(requests) AS requests,
    SUM(prompt_tokens + completion_tokens) AS total_tokens,
    SUM(cost_usd) AS cost_usd
  FROM llm_request_rollups
  WHERE window_start >= CURRENT_DATE - INTERVAL '30 days'
    AND environment = 'production'
  GROUP BY model
)
SELECT
  model,
  requests,
  ROUND(100.0 * requests / NULLIF(SUM(requests) OVER (), 0), 1)
    AS traffic_share_pct,
  ROUND(cost_usd, 2) AS cost_usd,
  ROUND(1e6 * cost_usd / NULLIF(total_tokens, 0), 2)
    AS cost_per_1m_tokens
FROM totals
ORDER BY cost_usd DESC;

Dashboards

LLM spend overview dashboardLLM latency & errors dashboardLLM adoption & unit economics dashboardEval quality tracking dashboard

Integrations

AI & LLM analytics integrationsLangfuse + MetabaseLangSmith + MetabaseHelicone + MetabaseBraintrust + MetabaseArize Phoenix + MetabaseW&B Weave + MetabaseOpenRouter + Metabase

Metrics

Token usageCost per 1M tokensLLM cost per userp95 LLM latencyLLM error rateCache hit rateEval pass rateLLM requests per day

Analytics

LLM analytics

FAQ

Why compute cost per 1M tokens when providers publish prices?
Published prices are per-direction list rates; your blended rate mixes prompt and completion tokens, cache discounts, batch pricing, and retries. Cost per 1M tokens computed from your own rollups is the effective rate you actually pay — and drift between it and the list price is itself a finding (usually cache regressions or prompt bloat).
How do I catch model-mix drift?
Chart each model's share of requests as an area over time and alert on the expensive models' share. Drift usually enters through defaults — a new feature ships pointing at the flagship model, or a fallback quietly becomes the primary. The deprecated-model table catches the opposite failure: traffic still burning on models you meant to retire.
Should I compare models on cost alone?
No — pair this dashboard with eval quality tracking. The decision is quality per dollar: a model that costs 40% less but fails your evals is not cheaper, it's broken. Put pass rate and cost per 1M tokens side by side for candidate models on the same dataset.