In this blog we focus on how we can dump and restore data stored in Tigris Server, an Open Source Database, using Tigris CLI. This will be demonstrated using our public beta environment.

Overview

At a high level the process looks as follows:

Setting up Tigris CLI
Authentication via the CLI
Performing the data dump or restore

Setting up Tigris CLI

We will need to grab the CLI binary that fits your platform and architecture. Several operating systems and architectures are supported.

I'm going to demonstrate the process using an M1 Mac laptop but the process should be similar on other platforms.

Mac

$ curl -sSL https://tigris.dev/cli-macos | sudo tar -xz -C /usr/local/bin

Alternatively, if you are a Homebrew user you may also use:

$ brew install tigrisdata/tigris/tigris-cli

Linux

$ curl -sSL https://tigris.dev/cli-linux | sudo tar -xz -C /usr/local/bin

Windows

C:\Users\robert> curl -sSLO https://tigris.dev/cli-windows
C:\Users\robert> tar xvzf cli-windows

Authentication

In order to interact with the public beta environment, we need to first authenticate our client with Tigris.

You can run the following CLI command to initiate login:

$ tigris login

This should initiate browser based authentication:

Pick whichever authentication method is applicable to your user to authenticate.

A successful setup should yield the following message in the browser:

On the command line the utility should also inform you that authentication was successful and your credentials have been saved:

❯ tigris login
Opening login page in the browser. Please continue login flow there.
Successfully logged in

❯ cat ~/.tigris/tigris-cli.yaml
token: <redacted>
url: api.preview.tigrisdata.cloud

ℹ️ Your authentication process may look different if you are self-hosting Tigris. For instance, you will not have to authenticate if authentication has not been setup.

Performing Data Dump

We are going to use a simple script to dump all the contents into individual JSON files in the current working directory:

$ cat dump_all.sh
#!/usr/bin/env bash

# Increase Tigris Timeout
TIGRIS_TIMEOUT=1h
export TIGRIS_TIMEOUT

for DATABASE in `tigris list databases`
do
  echo " [.] Dumping schema of db ${DATABASE}"
    tigris describe database "${DATABASE}" --schema-only > "${DATABASE}.schema"
    for COLLECTION in `tigris list collections "${DATABASE}"`
    do
        echo " [*] Backing up collection ${DATABASE}:${COLLECTION}"
        tigris read "${DATABASE}" "${COLLECTION}" > "${DATABASE}.${COLLECTION}.json"
    done
done

Above script increases the timeout and dumps the data in all the databases and their collections that you have access to into an individual JSON file, one by one.

For example:

❯ ./dump_all.sh
 [.] Dumping schema of db tigris_netlify_starter
 [*] Backing up collection tigris_netlify_starter:todoItems
 [.] Dumping schema of db tigris_starter_ts
 [*] Backing up collection tigris_starter_ts:orders
 [*] Backing up collection tigris_starter_ts:products
 [*] Backing up collection tigris_starter_ts:social_messages
 [*] Backing up collection tigris_starter_ts:user_events
 [*] Backing up collection tigris_starter_ts:users
 [.] Dumping schema of db tigris_vercel_starter
 [*] Backing up collection tigris_vercel_starter:todoItems
 [.] Dumping schema of db tigris_starter_java
 [*] Backing up collection tigris_starter_java:orders
 [*] Backing up collection tigris_starter_java:product_collection
 [*] Backing up collection tigris_starter_java:users
 [.] Dumping schema of db meteorites
 [*] Backing up collection meteorites:landings
 [.] Dumping schema of db ycsb_tigris
 [*] Backing up collection ycsb_tigris:user_tables
 [.] Dumping schema of db auth0
 [*] Backing up collection auth0:users
 [.] Dumping schema of db catalog
 [*] Backing up collection catalog:products

❯ ls -la *.json *.schema
-rw-r--r--  1 rbarabas  staff      619 Nov  8 16:02 auth0.schema
-rw-r--r--  1 rbarabas  staff    49234 Nov  8 16:02 auth0.users.json
-rw-r--r--  1 rbarabas  staff    13866 Nov  8 16:02 catalog.products.json
-rw-r--r--  1 rbarabas  staff      424 Nov  8 16:02 catalog.schema
-rw-r--r--  1 rbarabas  staff  8892587 Nov  8 16:02 meteorites.landings.json
-rw-r--r--  1 rbarabas  staff      419 Nov  8 16:02 meteorites.schema
-rw-r--r--  1 rbarabas  staff      250 Nov  8 16:01 tigris_netlify_starter.schema
-rw-r--r--  1 rbarabas  staff        0 Nov  8 16:01 tigris_netlify_starter.todoItems.json
-rw-r--r--  1 rbarabas  staff        0 Nov  8 16:02 tigris_starter_java.orders.json
-rw-r--r--  1 rbarabas  staff        0 Nov  8 16:02 tigris_starter_java.product_collection.json
-rw-r--r--  1 rbarabas  staff     1722 Nov  8 16:02 tigris_starter_java.schema
-rw-r--r--  1 rbarabas  staff      173 Nov  8 16:02 tigris_starter_java.users.json
-rw-r--r--  1 rbarabas  staff        0 Nov  8 16:01 tigris_starter_ts.orders.json
-rw-r--r--  1 rbarabas  staff        0 Nov  8 16:01 tigris_starter_ts.products.json
-rw-r--r--  1 rbarabas  staff     1400 Nov  8 16:01 tigris_starter_ts.schema
-rw-r--r--  1 rbarabas  staff      553 Nov  8 16:01 tigris_starter_ts.social_messages.json
-rw-r--r--  1 rbarabas  staff     1161 Nov  8 16:02 tigris_starter_ts.user_events.json
-rw-r--r--  1 rbarabas  staff      563 Nov  8 16:02 tigris_starter_ts.users.json
-rw-r--r--  1 rbarabas  staff      250 Nov  8 16:02 tigris_vercel_starter.schema
-rw-r--r--  1 rbarabas  staff      172 Nov  8 16:02 tigris_vercel_starter.todoItems.json
-rw-r--r--  1 rbarabas  staff      545 Nov  8 16:02 ycsb_tigris.schema
-rw-r--r--  1 rbarabas  staff  7570000 Nov  8 16:02 ycsb_tigris.user_tables.json

Restore

To make the process a bit more interesting, in this example we are going to do a partial restore to an empty database. Full restore can be scripted similar to the data dump.

❯ tigris list databases❯

First, let's make sure we will not time out during restore:

❯ export TIGRIS_TIMEOUT=1h

Next, let's create the database using the schema we captured prior:

❯ tigris create database tigris_vercel_starter
❯ tigris list databases
tigris_vercel_starter

Also, its collections

❯ cat tigris_vercel_starter.schema | tigris create collection tigris_vercel_starter
❯ tigris list collections tigris_vercel_starter
todoItems

Lastly, load the data into the database:

❯ cat tigris_vercel_starter.todoItems.json | tigris insert tigris_vercel_starter todoItems
❯ tigris read tigris_vercel_starter todoItems
{"text":"Pasta","completed":true,"id":6}
{"text":"Celery","completed":true,"id":10}
{"text":"Bread","completed":false,"id":12}
{"text":"Grocery","completed":false,"id":13}

Summary

I hope above process gave you a good enough idea on how to perform data dumps out an into Tigris. If this article piqued your interest and would like to read more on operational matters, please let us know on our community interfaces! Join our Tigris Community Slack or Tigris Discord Server!

Tigris is the data platform built for developers! Use it as a scalable, ACID transactional, real-time backend for your serverless applications. Build data-rich features without worrying about slow queries or missing indexes. Seamlessly implement search within your applications with its embedded search engine. Connect serverless functions with its event streams to build highly responsive applications that scale automatically.

Dump and Restore Your Data In Tigris