How to build Braintrust dashboards in Metabase

Braintrust is an AI evals and observability platform for scoring experiments, catching regressions, and logging LLM interactions in production. Metabase is where you turn that LLM telemetry into shared, trustworthy dashboards. This guide covers two complementary paths: a lightweight MCP + CLI route that pulls live data with the Braintrust MCP server and loads a CSV into Metabase with the Metabase CLI, and a durable pipeline route that syncs Braintrust rollups into a database so you can build dashboards anyone can read.

How do you connect Braintrust to Metabase?

Most teams combine both routes: use quick exports and CLI uploads for a fast first pass, then move recurring cost and quality reporting to a warehouse-backed model.

What can you analyze from Braintrust data in Metabase?

Each view below is built from experiments and eval scores, plus the logs, datasets, and spans your sync exposes:

• Eval pass rate by experiment
• Score regressions between runs
• Model comparison on shared datasets
• Production log quality trends
• Eval coverage by feature

Which Braintrust dashboards should you build in Metabase?

How do you use the Braintrust MCP server with the Metabase CLI?

Pair the Braintrust MCP server with the Metabase CLI for fast, hands-on analysis. MCP is useful for scoped lookups and summarized exports; the Metabase CLI's upload command loads CSV data into Metabase and creates a ready-to-query table and model.

Example workflow

• Ask the MCP server for the latest experiments and eval scores with scores, pass/fail flags, and model versions.
• Export the result as CSV, keeping stable IDs, apps, models, environments, token counts, costs, and timestamps.
• Run mb upload csv to load it into Metabase as a table and model, then build questions and dashboards on top.

Be honest about the limits

• MCP lookups are excellent for exploration, not scheduled reporting.
• A CSV upload is a snapshot; refresh it with mb upload replace or move to the pipeline for real history.
• Experiment timestamps and score history are required for regression and quality trends.
• mb upload csv needs an uploads database configured under Admin → Settings → Uploads.

How do you set up Braintrust MCP and the Metabase CLI?

{
  "mcpServers": {
    "braintrust": {
      "url": "https://api.braintrust.dev/mcp"
    }
  }
}

EU data plane: api-eu.braintrust.dev/mcp. SSE-only clients are not supported. The standout tool is sql_query — SQL directly over experiments and logs — alongside summarize_experiment and infer_schema. Results over 1 MB come back as a signed-URL envelope.

# Install the Metabase CLI
npm install -g @metabase/cli

# Log in (opens your browser; requires Metabase v62+)
mb auth login --url https://your-metabase.example.com

# Load a experiments-and-eval-scores export — creates a table AND a model
mb upload csv --file braintrust-experiments-and-eval-scores.csv --collection root

# Refresh that same table later from a new export
mb upload replace <table-id> --file braintrust-experiments-and-eval-scores.csv

Can you generate a Braintrust dashboard with AI?

Yes. Use the prompt below with any assistant that can run the Braintrust MCP server and the Metabase CLI. It works end to end: if Braintrust tables already exist in Metabase it analyzes those; otherwise it pulls scoped, summarized data, loads it with mb upload csv, then builds the dashboard and caveats any metric that needs missing history.

Create a polished Metabase dashboard for Braintrust llm evaluation analytics.
Work end to end: get the data into Metabase if it isn't there yet, then build.

Goal: Help engineering, product, and finance partners understand pass rates, score trends, regressions, model comparisons, and eval coverage from Braintrust data.

Step 1 — Find or load the data:
- First, check what already exists in Metabase (search for braintrust tables and
  models). If durable Braintrust data is already present — synced from a warehouse
  or uploaded earlier — use it and skip to Step 2.
- If nothing is there, pull a scoped, summarized export with the Braintrust MCP server:
  experiments and eval scores, plus logs, datasets, spans.
  Prefer aggregated or rollup views over raw traces. Write each result to a CSV,
  then load it with the Metabase CLI — run "mb upload csv --file <export>.csv" so
  each upload creates a table and a ready-to-query model. Use "mb upload replace
  <table-id> --file <export>.csv" to refresh an existing table instead of creating
  duplicates.

Step 2 — Inspect before querying:
Do not assume exact table or column names. Inspect available fields, apps, models,
environments, timestamps, and whether rollups or history exist before creating
duration or trend cards.

Important:
- Build on whatever data is present; don't claim Metabase connects natively to
  Braintrust — it reads a database or CLI-uploaded tables.
- Never load raw prompt/response payloads into Metabase; use rollups, trace
  summaries, and experiment- or score-grain data.
- Only compute latency percentiles and durations when the required timestamps
  or pre-aggregated percentile fields exist.
- Exclude test, staging, and playground traffic from headline cost and quality
  cards, and segment by environment where the field exists.
- A single CSV is a point-in-time snapshot: only build trend cards if there is a
  usable date column or multiple periods have been uploaded.

Dashboard title: Braintrust LLM Evaluation Overview

Sections:
1. Executive summary: Latest pass rate; Score delta vs. previous run;
   Experiments last 30 days; Regressed examples; Eval spend.
2. Eval health: Pass rate and mean score by experiment over time.
3. Regressions: Score deltas between runs; regressed examples by dataset.
4. Comparisons: Pass rate by model and prompt version on shared datasets.
5. Production quality: Online scores; user feedback vs. eval score; coverage.

Filters: Date range, App, Model, Environment, User or team, Status.

Output: Build the dashboard if you have permission; otherwise provide the exact
questions, SQL, model definitions, and layout. Include caveats for any metric
that cannot be calculated from the available data.

How do you sync Braintrust data into a database or warehouse?

For dashboards that need history and reliability, land Braintrust rollups and metadata in a database first, then connect Metabase to that database.

Connector options

• Direct database or export pipeline — use the vendor's storage or export path when it reaches a database you control.
• Custom pipeline — use the Braintrust query API (SQL) for control over rollup grain, fields, and refresh cadence.
• MCP + CSV — use this for quick exploration and one-off slices.

Export automations (Enterprise plan) land logs in S3 or GCS as Hive-partitioned Parquet or JSONL on a 5-minute-to-daily cadence — BigQuery, Snowflake, and DuckDB read them directly. On other plans, script the query API — standard SQL, with the older BTQL syntax still supported — for experiment and score rollups.

Notes

• Decide the rollup grain first (daily per app/model is the workhorse) — it drives warehouse cost and every trend card.
• Land raw entity tables first, then build clean Metabase models on top.
• Normalize experiment, dataset, scorer, model, prompt-version, and created-at fields.
• Strip or hash prompt/response payloads and user identifiers before they reach the warehouse — cost and quality analytics don't need them.

How should you model Braintrust data in Metabase?

Core tables

Modeling advice

• Build a clean eval_results model with common columns across tools, so multi-source dashboards don't fork definitions.
• Separate entity tables (apps, models, datasets, prompts) from time-series rollups and trace- or score-grain tables.
• Exclude test, staging, and playground traffic from headline cost and quality metrics; keep environment as an explicit column.
• Use stable IDs for app, model, and experiment joins; display names and model aliases change.

Which Braintrust metrics should you track in Metabase?

What SQL powers Braintrust dashboards in Metabase?

These assume a cleaned analytical model in a warehouse (PostgreSQL dialect). Adjust table and column names to match your pipeline.

SELECT
  experiment,
  date_trunc('week', created_at) AS week,
  COUNT(*) AS examples,
  ROUND(
    100.0 * COUNT(*) FILTER (WHERE passed) / NULLIF(COUNT(*), 0), 1
  ) AS pass_rate_pct
FROM eval_results
GROUP BY experiment, 2
ORDER BY 2, experiment;

WITH ranked AS (
  SELECT
    dataset,
    experiment,
    AVG(score) AS mean_score,
    ROW_NUMBER() OVER (
      PARTITION BY dataset ORDER BY MIN(created_at) DESC
    ) AS recency
  FROM eval_results
  GROUP BY dataset, experiment
)
SELECT
  latest.dataset,
  latest.experiment AS latest_experiment,
  ROUND(latest.mean_score, 3) AS latest_score,
  ROUND(previous.mean_score, 3) AS previous_score,
  ROUND(latest.mean_score - previous.mean_score, 3) AS delta
FROM ranked latest
JOIN ranked previous
  ON previous.dataset = latest.dataset AND previous.recency = 2
WHERE latest.recency = 1
ORDER BY delta ASC;

SELECT
  scorer,
  model,
  COUNT(*) AS examples,
  ROUND(
    100.0 * COUNT(*) FILTER (WHERE passed) / NULLIF(COUNT(*), 0), 1
  ) AS pass_rate_pct,
  ROUND(AVG(score), 3) AS mean_score
FROM eval_results
WHERE created_at >= CURRENT_DATE - INTERVAL '90 days'
GROUP BY scorer, model
ORDER BY scorer, pass_rate_pct DESC;

What are common mistakes when analyzing Braintrust in Metabase?

Related

Related analytics

Related dashboards

Related integrations

FAQ