How to build W&B Weave dashboards in Metabase

W&B Weave is Weights & Biases' toolkit for tracing, evaluating, and monitoring LLM applications alongside your ML experiments. Metabase is where you turn that LLM telemetry into shared, trustworthy dashboards. This guide covers two complementary paths: a lightweight MCP + CLI route that pulls live data with the W&B MCP server and loads a CSV into Metabase with the Metabase CLI, and a durable pipeline route that syncs W&B Weave rollups into a database so you can build dashboards anyone can read.

How do you connect W&B Weave to Metabase?

Most teams combine both routes: use quick exports and CLI uploads for a fast first pass, then move recurring cost and quality reporting to a warehouse-backed model.

What can you analyze from W&B Weave data in Metabase?

Each view below is built from Weave traces and evaluations, plus the calls and ops, evaluation runs, and W&B runs and sweeps your sync exposes:

• Trace volume and latency by op
• Token usage by model
• Evaluation scores over time
• Error and exception trends
• LLM usage next to training runs

Which W&B Weave dashboards should you build in Metabase?

How do you use the W&B MCP server with the Metabase CLI?

Pair the W&B MCP server with the Metabase CLI for fast, hands-on analysis. MCP is useful for scoped lookups and summarized exports; the Metabase CLI's upload command loads CSV data into Metabase and creates a ready-to-query table and model.

Example workflow

• Ask the MCP server for a summarized slice of Weave traces and evaluations for the apps and models you care about.
• Export the result as CSV, keeping stable IDs, apps, models, environments, token counts, costs, and timestamps.
• Run mb upload csv to load it into Metabase as a table and model, then build questions and dashboards on top.

Be honest about the limits

• MCP lookups are excellent for exploration, not scheduled reporting.
• A CSV upload is a snapshot; refresh it with mb upload replace or move to the pipeline for real history.
• Trace timestamps or daily rollups are required for latency and error-rate trends.
• mb upload csv needs an uploads database configured under Admin → Settings → Uploads.

How do you set up W&B Weave MCP and the Metabase CLI?

{
  "mcpServers": {
    "wandb": {
      "url": "https://mcp.withwandb.com/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_WANDB_API_KEY"
      }
    }
  }
}

Key tools: query_weave_traces_tool, count_weave_traces_tool, and query_wandb_tool for runs and sweeps. There is no dedicated cost tool — cost fields ride along in trace summaries where recorded. Dedicated or on-prem deployments set WANDB_BASE_URL; an optional read-only flag exists for analysis use.

# Install the Metabase CLI
npm install -g @metabase/cli

# Log in (opens your browser; requires Metabase v62+)
mb auth login --url https://your-metabase.example.com

# Load a Weave-traces-and-evaluations export — creates a table AND a model
mb upload csv --file wandb-weave-Weave-traces-and-evaluations.csv --collection root

# Refresh that same table later from a new export
mb upload replace <table-id> --file wandb-weave-Weave-traces-and-evaluations.csv

Can you generate a W&B Weave dashboard with AI?

Yes. Use the prompt below with any assistant that can run the W&B MCP server and the Metabase CLI. It works end to end: if W&B Weave tables already exist in Metabase it analyzes those; otherwise it pulls scoped, summarized data, loads it with mb upload csv, then builds the dashboard and caveats any metric that needs missing history.

Create a polished Metabase dashboard for W&B Weave llm observability analytics.
Work end to end: get the data into Metabase if it isn't there yet, then build.

Goal: Help engineering, product, and finance partners understand traffic, latency, error rates, token usage, cost, and adoption from W&B Weave data.

Step 1 — Find or load the data:
- First, check what already exists in Metabase (search for wandb-weave tables and
  models). If durable W&B Weave data is already present — synced from a warehouse
  or uploaded earlier — use it and skip to Step 2.
- If nothing is there, pull a scoped, summarized export with the W&B MCP server:
  Weave traces and evaluations, plus calls and ops, evaluation runs, W&B runs and sweeps.
  Prefer aggregated or rollup views over raw traces. Write each result to a CSV,
  then load it with the Metabase CLI — run "mb upload csv --file <export>.csv" so
  each upload creates a table and a ready-to-query model. Use "mb upload replace
  <table-id> --file <export>.csv" to refresh an existing table instead of creating
  duplicates.

Step 2 — Inspect before querying:
Do not assume exact table or column names. Inspect available fields, apps, models,
environments, timestamps, and whether rollups or history exist before creating
duration or trend cards.

Important:
- Build on whatever data is present; don't claim Metabase connects natively to
  W&B Weave — it reads a database or CLI-uploaded tables.
- Never load raw prompt/response payloads into Metabase; use rollups, trace
  summaries, and experiment- or score-grain data.
- Only compute latency percentiles and durations when the required timestamps
  or pre-aggregated percentile fields exist.
- Exclude test, staging, and playground traffic from headline cost and quality
  cards, and segment by environment where the field exists.
- A single CSV is a point-in-time snapshot: only build trend cards if there is a
  usable date column or multiple periods have been uploaded.

Dashboard title: W&B Weave LLM Observability Overview

Sections:
1. Executive summary: Requests last 7 days; p95 latency; Error rate;
   Total cost last 30 days; Active users.
2. Traffic: Requests per day by app and environment; tokens per day.
3. Latency: p50/p95 by model by week; slowest ops or chains.
4. Errors: Error rate by model and app; failures by type; retries.
5. Cost and adoption: Cost by model; cost per request; active users of
   LLM features.

Filters: Date range, App, Model, Environment, User or team, Status.

Output: Build the dashboard if you have permission; otherwise provide the exact
questions, SQL, model definitions, and layout. Include caveats for any metric
that cannot be calculated from the available data.

How do you sync W&B Weave data into a database or warehouse?

For dashboards that need history and reliability, land W&B Weave rollups and metadata in a database first, then connect Metabase to that database.

Connector options

• Direct database or export pipeline — use the vendor's storage or export path when it reaches a database you control.
• Custom pipeline — use the Weave API for control over rollup grain, fields, and refresh cadence.
• MCP + CSV — use this for quick exploration and one-off slices.

No warehouse export pipeline exists — exports are CSV, TSV, or JSONL from the UI and API. Pull scoped trace and evaluation summaries over MCP or the API and load CSVs with the Metabase CLI; treat Weave as an exploration-first source and keep durable history in whichever warehouse your other LLM logs land in.

Notes

• Decide the rollup grain first (daily per app/model is the workhorse) — it drives warehouse cost and every trend card.
• Land raw entity tables first, then build clean Metabase models on top.
• Normalize app, model, environment, status, duration, token-count, and window-start fields.
• Strip or hash prompt/response payloads and user identifiers before they reach the warehouse — cost and quality analytics don't need them.

How should you model W&B Weave data in Metabase?

Core tables

Modeling advice

• Build a clean llm_traces model with common columns across tools, so multi-source dashboards don't fork definitions.
• Separate entity tables (apps, models, datasets, prompts) from time-series rollups and trace- or score-grain tables.
• Exclude test, staging, and playground traffic from headline cost and quality metrics; keep environment as an explicit column.
• Use stable IDs for app, model, and experiment joins; display names and model aliases change.

Which W&B Weave metrics should you track in Metabase?

What SQL powers W&B Weave dashboards in Metabase?

These assume a cleaned analytical model in a warehouse (PostgreSQL dialect). Adjust table and column names to match your pipeline.

-- Keep p95 at the rollup grain it was computed. Do not MAX/AVG percentiles
-- across windows — that invents a fake weekly p95.
SELECT
  model,
  window_start,
  p95_latency_ms,
  requests
FROM llm_request_rollups
WHERE environment = 'production'
  AND window_start >= CURRENT_DATE - INTERVAL '30 days'
ORDER BY window_start, model;

SELECT
  app_name,
  date_trunc('week', window_start) AS week,
  SUM(requests) AS requests,
  SUM(error_count) AS errors,
  ROUND(
    100.0 * SUM(error_count) / NULLIF(SUM(requests), 0), 2
  ) AS error_rate_pct
FROM llm_request_rollups
WHERE environment = 'production'
GROUP BY app_name, 2
ORDER BY 2, error_rate_pct DESC;

SELECT
  date_trunc('day', started_at) AS day,
  app_name,
  COUNT(*) AS traces,
  SUM(total_tokens) AS total_tokens,
  ROUND(SUM(cost_usd), 2) AS cost_usd
FROM llm_traces
WHERE started_at >= CURRENT_DATE - INTERVAL '30 days'
GROUP BY 1, app_name
ORDER BY 1, cost_usd DESC;

What are common mistakes when analyzing W&B Weave in Metabase?

Related

Related analytics

Related dashboards

Related integrations

FAQ