AI usage auditing

Admin > AI > Usage auditing

Admins can get an overview of human-robot interactions across Metabase, from high-level stats like total token counts down to visibility into actual conversations. These interactions include the in-product Metabot chat sidebar, Metabot conversations in Documents, Slack chats, and inline SQL editing.

The usage auditing section includes:

• Stats: aggregate charts across all Metabot activity.
• Conversations: a filterable list of every conversation, with a detail view for each.

You can also build your own questions on top of the Usage Analytics views that back these reports.

Stats

Admin > AI > Usage auditing > Stats

The Stats page shows aggregate Metabot activity over a date range you choose, defaulting to the previous 30 days.

Filters

• Date range: the time window the charts cover.
• User: limit to a single person (or All users).
• Group: limit to a single group (or All groups).
• Tenant: limit to a single tenant. Only shows up if tenants are enabled.

Metrics

Pick what you want to count:

• Conversations: one row per distinct Metabot chat, across every surface listed above (chat sidebar, Documents, Slack, inline SQL). Not MCP conversations.
• Tokens: total tokens (input and output) consumed by LLM calls.
• Messages: every message exchanged, both from people and from Metabot.

For each metric, you’ll see the same set of charts:

• By time: a time-series chart that buckets by hour or day depending on the date range you filter. Defaults to day.
• By source: where in Metabase the request came from. See Sources and profiles.
• By profile: which Metabot persona answered. See Sources and profiles.
• Users with most …, Groups with most …, IP addresses with most …: top-N rankings.
• Tenants with most …: only shown when tenants are enabled.

You can drill through in the By day, Groups with most …, Users with most …, or Tenants with most … charts to see the Conversations list with the corresponding filter applied. The By source, By profile, and IP addresses with most … charts are display-only.

Sources and profiles

• Source is where the request came from in Metabase.
• Profile is which agent persona answered it.

They often line up (a conversation that started in Slack is handled by the Slackbot profile), but they don’t have to. A conversation started from the Metabot chat sidebar might be handled by the Internal, NLQ, or SQL profile depending on what the person asked.

The Conversations admin page only shows Profile. Source is visible in the Stats charts and in the Usage Analytics models used to build custom reports.

Sources

Each conversation is tagged with a source, i.e. where in Metabase the conversation took place. The By source chart in Stats groups conversations by a human-readable source_name, and the AI Usage Log model exposes both source_name and a raw source ID (e.g. metabot_agent, oss-sql-gen, document_generate_content) for custom reports. Conversations Metabase couldn’t classify show up as (empty) on the chart.

Source name	Where it comes from
Metabot	The Metabot chat sidebar inside Metabase.
Documents	Content generation inside Documents.
Suggested Prompts	Background generation of suggested prompts.
Slackbot	Conversations that started in Slack.
SQL	Inline SQL editing in the native editor.
Unknown	A conversation Metabase couldn’t classify (distinct from no-source conversations, which appear as (empty) on charts).

Profiles

A profile is the configuration Metabot uses for a conversation: which prompt, which tools, and what it’s allowed to do. The Conversations admin page, the By profile chart in Stats, and the Metabot Conversations model (see Building custom reports) all show the human-readable profile name. The AI Usage Log model exposes the raw profile_id instead (e.g. internal, transforms_codegen, embedding_next).

Profile	What it does
Internal	The default Metabot in the chat sidebar. Can build query-builder questions and write SQL.
NLQ	Natural-language querying only. Always returns a query builder result, never SQL.
SQL	SQL writing only. Used by inline SQL editing and similar surfaces.
Slackbot	The persona behind Metabot in Slack.
Embedding	The Metabot persona used inside embedded Metabase.
Transforms codegen	Generates transform, SQL, or Python.
Documents	Generates content inside Documents.

Conversations

Admin > AI > Usage auditing > Conversations

The Conversations page lists every Metabot conversation Metabase has on file, newest first.

Filters

• Date range, User, Group, Tenant: same as the Stats filters.

Columns

Each row shows:

• User: who started the conversation.
• Profile: which Metabot persona answered.
• Date: when the conversation started.
• Messages: total messages, including both sides.
• Tokens: total LLM tokens spent.
• Queries: how many queries (SQL or query-builder) Metabot generated during the conversation.
• Searches: how many search-tool calls Metabot made.
• IP: the IP address the request came from.

You can sort by Date, Messages, or Tokens. Click any row to open the conversation detail.

Conversation detail

The detail view is a full audit of a single conversation. It includes:

• Header: start date, the person who chatted with Metabot, the profile Metabot used, the person’s groups (including whether they’re an admin), and tenant if applicable. From the … menu next to the person’s name you can jump to all of their conversations, or to their account details.
• Stat tiles: Messages, Total tokens, Queries run, Searches.
• Feedback (if any): thumbs-up or thumbs-down and comments. The agent response that triggered the feedback is shown alongside.
• Conversation transcript: the full message-by-message exchange. Tool calls (search calls, query construction, etc.) are inlined. You can click “View” to open a modal with the info.
• Queries generated: every SQL or query builder (MBQL) query Metabot wrote during the conversation, with the referenced tables listed underneath. Hit Visit on a query to open the item in a new tab and run it yourself. Transform code-gen queries are shown read-only and can’t be re-run from here.

The /inspect shortcut

If you’re an admin chatting with Metabot, type /inspect in the chat to jump straight from the current conversation to its detail page in Usage auditing.

Building custom reports

Three Usage Analytics models back the Usage auditing pages.

• AI Usage Log: one row per LLM call.
• Metabot Conversations: one row per conversation.
• Metabot Messages: one row per message.

Save your custom questions in the Custom reports sub-collection so the reports inherit the right permissions.

What isn’t tracked

MCP activity isn’t included in Usage auditing. MCP requests don’t go through Metabot’s conversation pipeline, so they don’t generate conversations or token rows.

Further reading

• AI usage controls
• AI settings
• Metabot
• Metabot in Slack
• Usage analytics
• Permissions overview