KWhen I entered the Web3 space in 2021, my first question was: What is the database of choice?
The answers were a bit unsatisfying.
There were the omnipresent blockchain networks, which were expensive and slow, and there were decentralized storage networks that focused on object storage at that time. Their more granular APIs could have been more convenient to use. And finally, there were p2p databases that stored all data on the client, which didn't seem optimal to me either.
Much has changed in the last few years, so I wanted to check out the decentralized database products.
I wanted to do a series on decentralized databases, where I would build a backend for a blog with different database solutions—starting with ComposeDB.
What is Compose DB?
ComposeDB is a graph database created by 3BoxLabs, a company well-known in the Web3 ecosystem for their work on decentralized identifiers (DIDs) and their main product the Ceramic network. Ceramic is a network of nodes that store and share composable data streams on top of libp2p, the network stack that also powers IPFS.
The goal of Ceramic is to enable decentralized storage independent from a specific blockchain network. This way, you can store your data in a decentralized manner and use it on DApps independently of what blockchain powers them.
Ceramic is the low-level infrastructure, and ComposeDB gives you a graph database on top of it. So, ComposeDB is part of every Ceramic node.
You define models with GraphQL schemas, so you can query the data from a node via GraphQL too.
When two or more nodes use the same model, they start synchronizing their data, which leads to decentralization.
To get data into the network, you must host your own node and add your schema. If others think your models are worth their resources, they can load your schema and start mirroring it.
There is no incentivization layer in the form of tokens. While on a blockchain network, you have to pay fees for data you want to store, the node operators have to interact "manually" to figure out what to keep and who they allow to upload it.
So, let's build a decentralized blog with it!
Features
- Authentication
- Profile creation and update
- Article creation and update
- Commenting on articles
- Reading profiles, articles, and comments
Architecture
The architecture of a ComposeDB powered backend would look something like this:
The orange Ceramic nodes share a model and synchronize their data, the red ones are Ceramic nodes without, or with different models.
Prerequisites
Implementation
Let's start by creating a new Node.js project and installing all dependencies with these commands:
$ mkdir composedb && cd composedb
$ npm init -y
$ npm i @ceramicnetwork/cli @ceramicnetwork/http-client @composedb/cli @composedb/devtools @composedb/devtools-node dids key-did-provider-ed25519 key-did-resolver uint8arrays
Creating the Ceramic Config File
After this, we can create an NPM script in the package.json to start a Ceramic node.
"scripts": {
"ceramic": "ceramic daemon --config=config/daemon.config.json"
},
This script starts the Ceramic node and creates a new config file. You have to stop it with ctrl+c since we will edit the config file later and have to restart it with the updated config file.
$ npm run ceramic
Creating the Admin Account
To be able to modify the database via its API, we need an admin account. Let's create a script for it, and make sure you use the .mjs extension.
File scripts/setup-account.mjs:
import { mkdirSync, readFileSync, writeFileSync } from "fs"
import { execSync } from "child_process"
const composeDb = (...args) =>
execSync("composedb " + args.join(" "), {
encoding: "utf-8"
}).trim()
mkdirSync("account")
const adminKey = composeDb("did:generate-private-key")
writeFileSync("account/admin-key", adminKey)
const adminDid = composeDb("did:from-private-key", adminKey)
writeFileSync("account/admin-did", adminDid)
const daemonConfig = JSON.parse(
readFileSync("config/daemon.config.json", {
encoding: "utf-8"
})
)
daemonConfig["http-api"]["admin-dids"].push(adminDid)
writeFileSync(
"config/daemon.config.json",
JSON.stringify(daemonConfig)
)
This script will generate a random private key and use it to generate a DID. Both get saved into the account directory and added to the Ceramic config file we created before.
To run the account creation, execute the following command:
$ node scripts/setup-account.mjs
Creating the Schema
The schema consists of multiple GraphQL files; they all get indexed by the Ceramic node and then linked to each other.
ComposeDB comes with DID-based accounts out of the box, so we don't have to implement them ourselves.
Creating the Profile Model
Having more info about a user than just their DID would be nice, so let's start with a profile model.
File schema/profile.graphql:
type Profile
@createModel(accountRelation: SINGLE, description: "Author profile") {
author: DID! @documentAccount
name: String! @string(minLength: 1, maxLength: 50)
bio: String @string(maxLength: 100000)
}
-
Profileuses the@createModeldirective, which marks the type as a model for ComposeDB -
accountRelationdescribes how many instances of this model a ComposeDB account can have; in this case, one -
descriptionhelps people who find the model in the registry -
authorfield adds the owning account to theProfile -
nameandbiofields are just text with a length definition
Creating the Article Model
What is a blog without articles? I know, I know, most of us never get to the actual writing because we are preoccupied with building blogs... just like right now, lol.
File schema/article.graphql:
type Profile @loadModel(id: "$PROFILE_ID") {
id: ID!
}
type Article
@createModel(
accountRelation: LIST
description: "Text written by an author"
) {
author: DID! @documentAccount
title: String! @string(maxLength: 50)
date: DateTime!
content: String! @string(maxLength: 100000)
profileId: StreamID! @documentReference(model: "Profile")
}
-
@loadModelwill load a model with a specific ID from the Ceramic node. We don't have that ID because we didn't indexProfileyet, so we put in a placeholder that will be replaced by a script later -
accountRelationis aLISTbecause an account can have more than one article, even if it has a low probability of happening. -
authorAgain, the ID of the account that created the record -
titleandcontentfor the actual content of the article -
dateto log when the article was created -
profileIdto embed theProfilefrom before; that way we can load it as part of theArticlein one query
Creating the Comment Model
Now, what are articles without flame wars, right? So, let's add a Comment model to spice things up a bit!
File schema/comment.graphql:
type Article @loadModel(id: "$ARTICLE_ID") {
id: ID!
}
type Comment
@createModel(
accountRelation: LIST
description: "A comment on an article"
) {
author: DID! @documentAccount
content: String! @string(maxLength: 5000)
articleId: StreamID! @documentReference(model: "Article")
}
-
@loadModelto get the indexedArticlemodel so that we can link it to each comment -
accountRelationis aLISTbecause an account can have more than one comment -
authorAgain, the ID of the account that created the record -
contentfor the actual content of the comment -
articleIdto embed theArticlefrom before; that way, we can load it as part of theCommentin one query
Link Article and Comment Models
Now, we can show a Profile on an article and an Article on a comment, but it would be much more interesting to show multiple Comment instances on an Article.
For this, we have to create another schema.
File schema/article.comment.graphql:
type Comment @loadModel(id: "$COMMENT_ID") {
id: ID!
}
type Article @loadModel(id: "$ARTICLE_ID") {
comments: [Comment] @relationFrom(model: "Comment", property: "articleId")
}
We load the indexed Comment model from our Ceramic node and the indexed Article model, but this time we extend the Article model with a relation. The @relationFrom directive will create a field that checks the articleId of our Comment models match the ID of a corresponding Article when requested.
Indexing the Schema
We need to get all these models loaded by our Ceramic node, so we create a script that replaces our ID placeholders before it loads a model. This way, we can link our models with the correct model IDs without manually adding them to the schema definitions.
Again, make sure you use the .mjs extension.
File scripts/setup-schema.mjs:
import { execSync } from "child_process"
import { readFileSync } from "fs"
import { CeramicClient } from "@ceramicnetwork/http-client"
import { DID } from "dids"
import { Ed25519Provider } from "key-did-provider-ed25519"
import { getResolver } from "key-did-resolver"
import { fromString } from "uint8arrays/from-string"
import { Composite } from "@composedb/devtools"
import {
createComposite,
writeEncodedComposite,
} from "@composedb/devtools-node"
const privateKey = fromString(
readFileSync("account/admin-key", { encoding: "utf-8" }).trim(),
"base16"
)
const did = new DID({
resolver: getResolver(),
provider: new Ed25519Provider(privateKey),
})
await did.authenticate()
const ceramic = new CeramicClient("http://127.0.0.1:7007")
ceramic.did = did
const profileComposite = await createComposite(
ceramic,
"schema/profile.graphql"
)
const articleSchema = readFileSync("schema/article.graphql", {
encoding: "utf-8",
}).replace("$PROFILE_ID", profileComposite.modelIDs[0])
const articleComposite = await Composite.create({
ceramic,
schema: articleSchema,
})
const commentSchema = readFileSync("schema/comment.graphql", {
encoding: "utf-8",
}).replace("$ARTICLE_ID", articleComposite.modelIDs[1])
const commentComposite = await Composite.create({
ceramic,
schema: commentSchema,
})
const articleCommentSchema = readFileSync("schema/article.comment.graphql", {
encoding: "utf-8",
})
.replace("$ARTICLE_ID", articleComposite.modelIDs[1])
.replace("$COMMENT_ID", commentComposite.modelIDs[1])
const articleCommentComposite = await Composite.create({
ceramic,
schema: articleCommentSchema,
})
const composite = Composite.from([
profileComposite,
articleComposite,
commentComposite,
articleCommentComposite,
])
await writeEncodedComposite(composite, "composites/blog.json")
await composite.startIndexingOn(ceramic)
execSync(
"composedb composite:compile composites/blog.json composites/blog.runtime.json --ceramic-url=http://0.0.0.0:7007"
)
execSync(
"composedb composite:compile composites/blog.json composites/blog.runtime.js --ceramic-url=http://0.0.0.0:7007"
)
First, we load the private key to authenticate with the Ceramic node.
Then, we load each model, replace the ID placeholders with stream IDs, and create a composite.
After creating and linking all models, we merge them so our node can index them.
We then save the merged composite to disk at composites/blog.json and compile two runtime composites from it that go into the same directory. Our GraphQL server will use these runtime composites later.
To run the script, we have to start the Ceramic node again and execute the script, each in different command lines.
Start the node with this command:
$ npm run ceramic
Open another command line and run the script with the following command:
$ node scripts/setup-schema.mjs
After this, our models should be queryable.
Testing the Schema
We can test the schema using the GraphQL server that ships with the ComposeDB CLI.
Let's add an NPM script for this:
File package.json:
...
"scripts": {
...
"graphql": "composedb graphql:server --graphiql composites/blog.runtime.json --ceramic-url=http://0.0.0.0:7007 --did-private-key=$(cat account/admin-key) --port=5005"
},
...
The GraphQL server will use the runtime composite we generated and connect to the Ceramic node.
If you open http://localhost:5005/graphql you'll see the Yoga GraphiQL interface and can start issuing queries.
Querying your Account
The most straightforward query is for the currently active account, the viewer.
Query:
{
viewer {
id
isViewer
}
}
Response:
{
"data": {
"viewer": {
"id": "did:key:...",
"isViewer": true
}
}
}
In the case of the GraphQL server, the viewer is the admin account we created. Obviously, isViewer is true when we query our current account.
Creating a Profile
If we use the auto-completion of GraphiQL, we see the models we indexed are already available.
But we didn't create records for our models.
Records are called nodes and edges in a graph database, but I will use records to prevent confusion with the Ceramic nodes.
So, let's start by creating a Profile for our admin. To do so, run this mutation:
Mutation:
mutation {
createProfile(
input: {
content: {
name: "Admin"
bio: "The creator of this blog."
}
}
) {
profile: document {
name
bio
}
}
}
Response:
{
"data": {
"createProfile": {
"profile": {
"name": "Admin",
"bio": "The creator of this blog."
}
}
}
}
Loading the Profile
Now, we can use the viewer to receive the profile we created.
Query:
{
viewer {
profile {
name
bio
}
}
}
Response:
{
"data": {
"viewer": {
"profile": {
"name": "Admin",
"bio": "The creator of this blog."
}
}
}
}
If we want to load the profile for another user, we can use ComposeDB's built-in Node type. Every document is a Node; we must use inline fragments to define which kind of nodes we want.
Query:
{
profile: node(id: "...") {
... on Profile {
name
bio
}
}
}
Response:
{
"data": {
"profile": {
"name": "Admin",
"bio": "The creator of this blog."
}
}
}
Creating and Updating an Article
Now that we have a profile, we can start writing something!
Mutation:
mutation {
createArticle(
input: {
content: {
date: "2023-03-21T12:19:06.089Z"
title: "First Article!"
content: "This is some content."
profileId: ""
}
}
) {
article: document {
date
title
content
}
}
}
Response:
{
"data": {
"createArticle": {
"article": {
"date": "2023-03-21T12:19:06.089Z",
"title": "First Article!",
"content": "This is some content."
}
}
}
}
If we wanted to update the article's title, the mutation would look like this:
Mutation:
mutation {
updateArticle(
input: {
id: "..."
content: {
title: "New title!"
}
}
) {
article: document {
title
}
}
}
Response:
{
"data": {
"updateArticle": {
"article": {
"title": "New title!"
}
}
}
}
Loading Profile with Articles and Article with Profile
Now, in a blog UI, we want to show the profile of a user and all of their articles.
Depending on what ID we have at hand, we can either query the profile by ID and traverse to the article list of the owner of the profile:
Query:
{
profile: node(id: "") {
... on Profile {
name
bio
author {
articleList(last: 10) {
articles: edges {
article: node {
id
date
title
}
}
}
}
}
}
}
Response
{
"data": {
"profile": {
"name": "Admin",
"bio": "The creator of this blog.",
"author": {
"articleList": {
"articles": [
{
"article": {
"id": "...",
"date": "2023-03-21T12:19:06.089Z",
"title": "New title!"
}
}
]
}
}
}
}
}
We can also start from the account and get the profile and article record.
Query:
{
account: node(id: "did:key:...") {
... on CeramicAccount {
profile {
name
bio
}
articleList(last: 10) {
articles: edges {
article: node {
id
date
title
}
}
}
}
}
}
Response:
{
"data": {
"account": {
"profile": {
"name": "Admin",
"bio": "The creator of this blog."
},
"articleList": {
"articles": [
{
"article": {
"id": "...",
"date": "2023-03-21T12:19:06.089Z",
"title": "New title!"
}
}
]
}
}
}
}
Creating and Loading Comments
Let's start with adding comments to an article.
Mutation:
mutation {
createComment(
input: {
content: {
content: "Hello!"
articleId: "..."
}
}
) {
comment: document {
content
}
}
}
Response:
{
"data": {
"createComment": {
"comment": {
"content": "Hello!"
}
}
}
}
Now that we have an article with a comment, we can query the article, with the author's name and all related comments with their author's name.
Query:
{
article: node(id: "...") {
... on Article {
author {
profile {
name
}
}
date
title
content
comments(last: 10) {
edges {
node {
author {
profile {
name
}
}
content
}
}
}
}
}
}
Response:
{
"data": {
"article": {
"author": {
"profile": {
"name": "Admin"
}
},
"date": "2023-03-21T12:19:06.089Z",
"title": "New title!",
"content": "This is some content.",
"comments": {
"edges": [
{
"node": {
"author": {
"profile": {
"name": "Admin"
}
},
"content": "Hello!"
}
}
]
}
}
}
}
No Data Deletion
There is no way to delete data on IPFS for sure, so we'll be missing out on that feature of our blog. If the network participants deem it necessary enough, they will pin it all over the place and keep it from getting deleted.
We can write update methods that remove content and mark it as deleted for the UX. But remember that users might be surprised when they find their content elsewhere.
Summary
Even though ComposeDB is still in beta, it looks like a promising database solution for your DApp.
It distributes data via the Ceramic Network, which, in turn, is built on top of IPFS. It doesn't have any token-based incentivization layer (yet?), but that is a plus for some people.
If someone cares about a DApp, they can host a Ceramic node and import the relevant models to mirror the data.
ComposeDB is a graph database that can be confusing when you come from a relational database, but its GraphQL interface might still be familiar to the average developer.
