Serialization
Serialization is only available on Pro and Enterprise plans (both self-hosted and on Metabase Cloud).
Once you really get rolling with Metabase, it’s often the case that you’ll have more than one Metabase instance spun up. You might have a couple of testing or development instances and a few production ones, or maybe you have a separate Metabase per office or region.
To help you out in situations like this, Metabase has a serialization feature which lets you create an export of the contents of a Metabase that can then be imported into one or more Metabases.
Export will serialize the contents of your source Metabase as YAML files.
Import will read those exported YAML files and create or update items in the target Metabase based on the contents serialized in those YAML files.
There are two ways to run these export
and import
commands:
We’re interested in how we can improve serialization to suit your workflow. Upvote an existing issue to let us know it’s important to you. If a relevant issue doesn’t yet exist, please create one and tell us what you need.
Serialization use cases
- Staging environments. Enable a staging-to-production workflow for important dashboards by exporting from a staging instance of Metabase and then importing them into your production instance(s).
- Version control. Check the exported files into version control and audit changes to them, as the YAML files contained within the export are pretty readable.
- Duplicating assets to other Metabase instances. Export the “template” data from a source Metabase and import them to one or more target instances.
Check out our guides for:
Serialization isn’t intended for use cases like duplicating assets or swapping data sources within the same Metabase instance. If you’re using serialization for duplicating assets within the same instance, check out How export works, How import works, and the directions for your use case in Other uses of serialization
How export works
- What gets exported
- General Metabase settings that get exported
- Customize what gets exported
- Example of a serialized question
- Metabase uses Entity IDs to identify and reference items
What gets exported
Metabase will only include some artifacts in its exports:
- Collections (but personal collections don’t get exported unless explicitly specified them through export options)
- Dashboards
- Saved questions
- Actions
- Models
- SQL Snippets
- Data model and table metadata
- Segments and Metrics defined in the Table Metadata
- Public sharing settings for questions and dashboards
- General Metabase settings
- Events and timelines
- Database connection strings (only if specified through export options) (#customize-what-gets-exported).
Metabase will export its artifacts to a directory of YAML files. The export includes:
-
Directories that contain YAML files for various Metabase entities. An example export could include the following directories, depending on what you exported and the contents of your Metabase:
- actions
- collections
- cards
- dashboards
- timelines
- databases
When serializing through the API, the export directory will be a compressed into a .tar.gz file.
-
A
settings.yaml
file that includes some Metabase-wide settings
Database connection details are not included by default, so you but you can configure your export to include them.
General Metabase settings that are exported
Here’s the list of the general settings Metabase exports in the settings.yaml
file. For more on Metabase settings, see Configuring Metabase.
humanization-strategy
native-query-autocomplete-match-style
site-locale
report-timezone-short
report-timezone-long
application-name
enable-xrays
show-homepage-pin-message
source-address-header
enable-nested-queries
custom-geojson-enabled
start-of-week
custom-geojson
available-timezones
unaggregated-query-row-limit
aggregated-query-row-limit
hide-embed-branding?
search-typeahead-enabled
enable-sandboxes?
application-font
available-locales
landing-page
enable-embedding
application-colors
application-logo-url
application-favicon-url
show-homepage-xrays
show-metabot
enable-whitelabeling?
show-homepage-data
site-name
application-font-files
loading-message
report-timezone
persisted-models-enabled
enable-content-management?
subscription-allowed-domains
breakout-bins-num
available-fonts
custom-formatting
Customize what gets exported
You can customize what gets exported. You can tell Metabase:
- Export specific collections
- Do not export collections
- Do not export Metabase settings
- Do not export table metadata
- Include sample field values (excluded by default)
- Include database connection details (excluded by default)
See export parameters in CLI commands or export parameters in API calls.
Example of a serialized question
Questions can be found in the cards
directory of a collection directory. Here’s an example card YAML file for a question written with SQL that uses a field filter and has an area chart visualization.
To preserve a native query’s multi-line format, remove trailing whitespace from native queries. If your native query has trailing whitespace, YAML will convert your query to a single string literal (which only affects presentation, not functionality).
name: Products by week
description: Area chart of products created by week
entity_id: r6vC_vLmo9zG6_r9sAuYG
created_at: "2024-05-08T19:10:24.348808Z"
creator_id: admin@metabase.local
display: area
archived: false
collection_id: onou5H28Wvy3kWnjxxdKQ
collection_preview: true
collection_position: null
query_type: native
dataset: false
cache_ttl: null
database_id: Sample Database
table_id: null
enable_embedding: false
embedding_params: null
made_public_by_id: null
public_uuid: null
parameters:
- default:
- Gizmo
id: c37d2f38-05fa-48c4-a208-19d9dba803c6
name: Pick a category
slug: category_filter
target:
- dimension
- - template-tag
- category_filter
type: string/=
parameter_mappings: []
dataset_query:
database: Sample Database
native:
query: |-
SELECT
category,
date_trunc ('week', created_at) AS "Week",
count(*) AS "Count"
FROM
products
WHERE
GROUP BY
category,
"Week"
template-tags:
category_filter:
default:
- Gizmo
dimension:
- field
- - Sample Database
- PUBLIC
- PRODUCTS
- CATEGORY
- base-type: type/Text
display-name: Pick a category
id: c37d2f38-05fa-48c4-a208-19d9dba803c6
name: category_filter
type: dimension
widget-type: string/=
type: native
result_metadata:
- base_type: type/Text
display_name: CATEGORY
effective_type: type/Text
field_ref:
- field
- CATEGORY
- base-type: type/Text
name: CATEGORY
semantic_type: null
- base_type: type/DateTime
display_name: Week
effective_type: type/DateTime
field_ref:
- field
- Week
- base-type: type/DateTime
name: Week
semantic_type: null
- base_type: type/BigInteger
display_name: Count
effective_type: type/BigInteger
field_ref:
- field
- Count
- base-type: type/BigInteger
name: Count
semantic_type: type/Quantity
visualization_settings:
column_settings: null
graph.dimensions:
- Week
- CATEGORY
graph.metrics:
- Count
serdes/meta:
- id: r6vC_vLmo9zG6_r9sAuYG
label: products_created_by_week
model: Card
initially_published_at: null
metabase_version: v1.49.7 (f0ff786)
type: question
Metabase uses Entity IDs to identify and reference Metabase items
Metabase assigns a unique entity ID to every Metabase item (a dashboard, question, model, collection, etc.). Entity IDs use the NanoID format.
You can see the entity IDs of items in the exported YAML files in the entity_id
field. For example, in the Example of a serialized question, you’ll see the Entity ID of that question:
entity_id: r6vC_vLmo9zG6_r9sAuYG
This ID also appears in the serdes/meta → id
field (these IDs must match):
serdes/meta:
- id: r6vC_vLmo9zG6_r9sAuYG
To disambiguate entities that share the same name, Metabase includes entity IDs in the file and directory names for exported entities.
r6vC_vLmo9zG6_r9sAuYG_products_by_week.yaml
IA96oUzmUbYfNFl0GzhRj_accounts_model.yaml
KUEGiWvoBFEc5oGQCEnPg_converted_customers.yaml
For example, in the Example of a serialized question above, you can see the field collection_id
:
collection_id: onou5H28Wvy3kWnjxxdKQ
This ID refers to the collection where the question was saved. In a real export, you’d be able to find a YAML file for this collection whose name starts with its ID: onou5H28Wvy3kWnjxxdKQ
.
Databases, schemas, tables, and fields are identified by name
By default, Metabase exports some database and data model settings. Exports exclude database connection strings by default. You can explicitly include database connection strings. You can also choose to exclude the data model entirely.
Metabase serializes databases and tables in the databases
directory. It will include YAML files for every database, table, field, segment, and metric.
Databases, tables, and fields are referred to by their names (unlike Metabase-specific items, which are referred to by entity IDs).
For example, in the Example of a serialized question, there are several YAML keys that reference Sample Database:
database_id: Sample Database
---
dataset_query:
database: Sample Database
In the description of the field filter (category_filter:
) in that example, you can see the reference to the field that’s used to populate the filter options:
dimension:
- field
- - Sample Database
- PUBLIC
- PRODUCTS
- CATEGORY
It refers to the CATEGORY
field in the PRODUCTS
table in the PUBLIC
schema in Sample Database
. The serialized Sample Database
in the databases
directory will also include YAML files for this field and table.
How import works
During import, Metabase will read the provided YAML files and create items according to the YAML specs. Example of a serialized question how Metabase records information it needs to reconstruct an item.
Metabase will not delete items from target instance during import, but it will overwrite items that already exist.
Metabase relies on Entity IDs to figure out which items to create or overwrite, and what are the relationships between items. When importing into an instance that already has some content in it, keep in mind:
-
If you import an item with an
entity_id
that doesn’t exist in your target Metabase, Metabase will create a new item. -
If you import an item with an
entity_id
that already exists in your target Metabase, the existing item will be overwritten.In particular, this means that if you export a question, then make a change in an exported YAML file — like rename a question by directly editing the
name
field — and then import the edited file back, Metabase will try to apply the changes you made to the YAML. -
If you import an item with blank
entity_id
(and blankserdes/meta → id
), Metabase will create a new item. -
All items and data sources referenced in YAML must either exist in the target Metabase already, or be included in the import.
For example, if an exported YAML has the field
collection_id: onou5H28Wvy3kWnjxxdKQ
, then the collectiononou5H28Wvy3kWnjxxdKQ
must already exist in target instance, or there must be a YAML file with the export of a collection that has this ID.
Serialization best practices
Use the same Metabase version for source and target instance
Currently, serialization only works if source and target Metabase have the same major version. If you are using the CLI serialization commands, the version of the .jar file that you are using to run the serialization commands should match both the source and target Metabase versions as well.
If you’re using H2 as your application database, you’ll need to stop Metabase before importing or exporting
If you’re using Postgres or MySQL as your application database, you can import and export while your Metabase is still running.
Avoid using serialization for backups
Just a note: serialization is not meant to back up your Metabase.
See Backing up Metabase.
If you’re instead looking to do a one-time migration from the default H2 database included with Metabase to a MySQL/Postgres, then use the migration guide instead.
You’ll need to manually add license tokens
Metabase excludes your license token from exports, so if you’re running multiple environments of Metabase Enterprise Edition, you’ll need to manually add your license token to the target Metabase(s), either via the Metabase user interface, or via an environment variable.
Metabase adds logs to exports and imports
Exports: Metabase adds logs to the compressed directory as export.log
.
Imports: You can add the -o -
flag to export logs directly into the terminal, or -o import.log
to save to a file.
Serialization with CLI commands
To serialize data on Metabase Cloud, use the import and export API endpoints
Metabase provides export
and import
CLI commands.
See How export works, How import works, and Serialization best practices for general information about serialization.
Exporting with CLI
To export the contents of a Metabase instance, change into the directory where you’re running the Metabase JAR and run:
java -jar metabase.jar export dir_name
Where dir_name
can be whatever you want to call the directory.
export
options
To view a list of export
options, use the help
command:
java -jar metabase.jar help export
Which will run and then print something like:
export path & options
Serialize Metabase instance into directory at `path`.
Options:
-c, --collection ID Export only specified ID; may occur multiple times.
-C, --no-collections Do not export any content in collections.
-S, --no-settings Do not export settings.yaml
-D, --no-data-model Do not export any data model entities; useful for subsequent exports.
-f, --include-field-values Include field values along with field metadata.
-s, --include-database-secrets Include database connection details (in plain text; use caution).
--collection
By default, Metabase will include all collections (except for personal collections) in the export. To include personal collections, you must explicitly add them with the --collection
flag.
The --collection
flag (alias -c
) lets you specify by ID one or more collections to include in the export. You can find the collection ID in the collection’s URL, e.g., for a collection at: your-metabase.com/collection/42-terraforming-progress
, the ID would be 42
.
If you want to specify multiple collections, separate the IDs with commas. E.g.,
java -jar metabase.jar export export_name --collection 1,2,3
--no-collections
The --no-collections
flag (alias -C
) tells Metabase to exclude all collections from the export.
--no-settings
The --no-settings
flag (alias -S
) tells Metabase to exclude the settings.yaml
file that includes site-wide settings, which is exported by default.
--no-data-model
The --no-data-model
flag (alias -D
) tells Metabase to exclude the Table Metadata settings from the export. Admins define the metadata settings in the Table Metadata tab of the Admin settings.
--include-field-values
The --include-field-values
flag (alias -f
) tells Metabase to include the sample values for field values, which Metabase uses to present dropdown menus. By default, Metabase excludes these sample field values.
--include-database-secrets
The --include-database-secrets
flag (alias -s
) tells Metabase to include connection details, including the database user name and password. By default, Metabase excludes these database connection secrets. If you don’t use this flag, you’ll need to manually input the credentials in the target Metabase.
Importing with CLI
To import exported artifacts into a Metabase instance, go to the directory where you’re running your target Metabase (the Metabase you want to import into) and use the following command, where path_to_export
is the path to the export that you want to import:
java -jar metabase.jar import path_to_export
Currently, you can only import exported artifacts into a Metabase instance that was created from the same version of Metabase.
import
options
Most options are defined when exporting data from a Metabase. To view a list of import flags, run:
java -jar metabase.jar help import
Which prints out:
import path & options
Load serialized Metabase instance as created by the [[export]] command from directory `path`.
Serialization via the API
Just like the CLI serialization commands, these endpoints are only available for Pro and Enterprise plans.
You can import and export serialized Metabase data via Metabase’s API, which makes serialization possible for Metabase Cloud deployments.
There are two endpoints:
POST /api/ee/serialization/export
POST /api/ee/serialization/import
We use
POST
, notGET
, for the/export
endpoint. The export operation does not modify your Metabase, but it’s long and intensive, so we usePOST
to prevent accidental exports.
For now, these endpoints are synchronous. If the serialization process takes too long, the request can time out. In this case, we suggest using the CLI commands.
See How export works, How import works, and Serialization best practices for general information about serialization.
API export parameters
You can append optional parameters to tell Metabase what to include or exclude from the export. You can also combine parameters (excluding, of course, all_collections
and selective collections).
So, assuming you’re testing on localhost
, and you want to exclude all collections from the export, you’d format the URL like so:
http://localhost:3000/api/ee/serialization/export?all_collections=false
You can include multiple parameters, separated by &
. For example, to exclude both the settings and the data model from the export:
http://localhost:3000/api/ee/serialization/export?data_model=false&settings=false
collection
Type: Array of integers.
Default value: Metabase will export all collections, unless all_collections
is set to false
.
To select which collections to export, include the collection IDs. For example, to include collections 1
and 2
:
collection=1&collection=2
all_collections
Type: Boolean
Default: true
(unless you specify a subset of collections with collection
).
To exclude all collections:
all_collections=false
settings
Type: Boolean.
Default: true
.
To exclude the settings.yaml
file that contains site-wide settings:
settings=false
data_model
Type: Boolean.
Default: true
.
To exclude the Table Metadata:
data_model=false
field_values
Type: Boolean.
Default: false
.
To include the sample values for field values, which Metabase uses to present dropdown menus:
field_values=true
database_secrets
Type: Boolean.
Default: false
.
To include database connection details, like the database username and password:
database_secrets=true
dirname
Type: String.
Default: <instance-name>-<YYYY-MM-dd_HH_mm>
To specify a different directory:
dirname=name_of_your_directory
You must compress your files when serializing via API calls
To keep file sizes over the network under control, both the export
and import
endpoints expect GZIP-compressed Tar files (.tgz
).
Compress a directory
To compress a directory (e.g., a directory named metabase_data
).
tar -czf metabase_data.tgz metabase_data
Extract a directory
To extract/unzip a directory:
tar -xvf metabase_data.tgz
Serialization API example
Step 1: Set up an API key
- Create an API key.
- Assign the key to the Admin group
Step 2: Export
- Send a
curl
request to export data:
curl \
-H 'x-api-key: YOUR_API_KEY' \
-X POST 'http://your-metabase-url/api/ee/serialization/export' \
-o metabase_data.tgz
substituting YOUR_API_KEY
with your API key and your-metabase-url
with the URL of your Metabase instance.
We use
POST
, notGET
, for the/export
endpoint.
This command will download the files as a GZIP-compressed Tar file named metabase_data.tgz
.
- Unzip the compressed file:
tar -xvf metabase_data.tgz
The extracted directory will be called something like metabase-yyyy-MM-dd_HH-mm
, with the date and time of the export.
Step 3: Import
- Compress the directory containing serialized Metabase application data
Let’s say you have your YAML files with Metabase application data in a directory called metabase_data
. Before importing those files to your target Metabase, you’ll need to compress those files.
tar -czf metabase_data.tgz metabase_data
- POST to
/api/ee/serialization/import
.
From the directory where you’ve stored your GZIP-compressed file, run:
curl -X POST \
-H 'x-api-key: YOUR_API_KEY' \
-F file=@metabase_data.tgz \
'http://your-metabase-url/api/ee/serialization/import' \
-o -
substituting YOUR_API_KEY
with your API key and your-metabase-url
with your Metabase instance URL.
The -o -
option will output logs in the terminal.
If you import Metabase data into the same Metabase as you exported it from, you will overwrite your existing questions, dashboards, etc. See How import works.
Other uses of serialization
Serialization is intended for version control, staging-to-production workflows, and duplicating assets to other Metabase instances. While it’s possible to use serialization for other use cases (like duplicating assets within a single instance), we don’t officially support these use cases.
We’re providing some directions on how to approach these unsupported use cases, but you should use them at your own risk. We strongly recommend that you test any process involving serialization on a non-production instance first, and reach out to help@metabase.com if you have any questions.
Using serialization for duplicating content within the same Metabase
Using serialization to duplicate content is not trivial, because you’ll need to wrangle Entity IDs for all the items you want to duplicate — and the IDs for all the items that are related to those items — to avoid overwriting existing data.
Before starting this perilous journey, review how export works and how import works, and contact help@metabase.com if you have any questions.
You’ll need to keep in mind:
- Importing an item with an entity ID that already exists will overwrite the existing item. To use an existing YAML file to create a new item, you’ll need to either a) create a new entity ID or b) clear the Entity ID.
- Two items cannot have the same entity IDs.
entity_id
andserdes/meta → id
fields in the YAML file should match.- If the
entity_id
andserdes/meta → id
fields in a YAML file for an item are blank, Metabase will create a new item with a new Entity ID. -
All items and data sources referenced by an item should either already exist in target Metabase or be included in the import.
For example, a collection can contain a dashboard that contains a question that is built on a model that references a data source. All of those dependencies must be either included in the import or already exist in the target instance.
This means that you might need a multi-stage export/import: create some of the items you need (like collections) in Metabase first, export them to get their entity IDs, then export the stuff that you want to duplicate and use those IDs in items that reference them.
For example, to duplicate a collection that contains only questions that are built directly on raw data (not on models or other saved questions), without changing the data source for the questions, you can use a process like this:
- In Metabase, create a “template” collection and add the items you’d like to duplicate.
- In Metabase, create a new collection which will serve as the target for duplicated items.
- Export the template collection and the target collection (you can use export parameters to export only a few collections). The YAML files for template questions in the export will have their own Entity IDs and reference the entity ID of the template collection.
- Get the entity ID of the target collection from its export.
-
In the YAML files for questions in the template collection export:
- Clear the values for the fields
entity_id
andserdes/meta → id
for questions. This will ensure that the template questions don’t get overwritten, and instead Metabase will create new questions. - Replace
collection_id
references to the template collection with the ID of the new collection
- Clear the values for the fields
- Import the edited files.
This process assumes that your duplicated questions will all use the same data source. You can combine this with switching the data source to use a different data source for every duplicated collection.
If you want to create multiple copies of a collection at once, then instead of repeating this process for every copy, you could create your own target entity IDs (they can be any string that uses the NanoID format), duplicate all the template YAML files, and replace template entity IDs and any references to them with your created entity IDs.
If your collections contains dashboards, models, and other items that can add dependencies, this process can become even more complicated – you need to handle every dependency. We strongly recommend that you first test your serialization on a non-production Metabase, and reach out to help@metabase.com if you need any help.
Using serialization to swap the data source for questions within one instance
If you want to change the data source for some of the questions in your Metabase — for example, just for questions in a single collection - you can serialize the questions manually, then edit the exported YAML files.
If you want to switch every question built on database A to use database B instead, and database B has exactly the same schema as database A, you don’t need to use serialization: you can just swap the connection string in Admin > Databases
Your databases must have the same engine, and ideally they should have the same schema.
You’ll need to keep in mind:
- Databases, tables and fields are referred to in Metabase by name
- Database connection details are not exported by default. To export database connection details, you’ll need to specify this in export parameters.
- Databases, tables and fields referenced by an item should either already exist in the target Metabase, or be included in the import.
For example, if you want to switch all questions in the Movie reviews
collection to use the Romance
database instead of the Horror
database, and both databases have the same schema, you could follow a process like this:
- In Metabase, add a new database connection in Admin > Databases and name it
Romance
. -
Export the collection
Movie reviews
.You can tell Metabase to export a single collection, or you can export all the collections and just work with files in the folder for the
Movie reviews
collection - In the YAML files for items from this collection, replace all references to
Horror
database with references toRomance
- Import the edited files.
Importing will overwrite the original questions. If you’re looking to create new questions that use a different data source, you can combine this process with Using serialization for duplicating assets.
This process assumes that your new data source has exactly the same schema. If the schema is different, then you will also need to replace all references to all tables and fields. This process can be complicated and error-prone, so we strongly recommend that you test your serialization on a non-production instance first, and reach out to help@metabase.com if you need any help.
Migrating from the old serialization commands
If you’re upgrading from Metabase version 46.X or older, here’s what you need to know:
- The
export
command replaces thedump
command. - The
import
command replace theload
command.
A few other changes to call out:
- The exported YAML files have a slightly different structure:
- Metabase will prefix each file with a 24-character entity ID (like
IA96oUzmUbYfNFl0GzhRj_accounts_model.yaml
). You can run a Metabase command to drop entity IDs before exporting. - The file tree is slightly different.
- Metabase will prefix each file with a 24-character entity ID (like
- To serialize personal collections, you just need to include the personal collection IDs in the list of comma-separated IDs following the
-c
option (short for--collection
).
If you’ve written scripts to automate serialization, you’ll need to:
- Reserialize your Metabase using the upgraded Metabase (which uses the new
export
andimport
commands). Note that serialization will only work if you export and import your Metabase using the same Metabase version. - Update those scripts with the new commands. See the new export options.
- If your scripts do any post-processing of the exported YAML files, you may need to update your scripts to accommodate the slightly different directory and YAML file structures.
Further reading
- Serialization tutorial.
- Multiple environments
- Setting up a git-based workflow.
- Need help? Contact support@metabase.com.
Read docs for other versions of Metabase.