No BS monorepo - Part 1

Matti Bar-Zeev - Dec 10 '21 - - Dev Community

In this post join me as I build my own monorepo called “pedalboard” with a single package in it, using Yarn workspaces and Lerna.
Since there is a lot to cover I decided to divide this post into 2 parts:

  • Part 1 (this one) - where I build the monorepo using Yarn workspaces and Lerna to the point I can bump a new version of my packages
  • Part 2 (next one) - where I will join the outcome of this part with GitHub actions in order to publish my package automatically to NPM

Some considerations first

This article is not about what technology should you choose to build and publish your monorepo’s packages, but I feel an explanation is deserved as to why I went with Yarn workspaces and Lerna -

Why Lerna and not Nx?
I try to avoid “code magic” when I can, and Nx sure smells like magic. The generator for different types of packages and complex configuration appears to me as something that can quite quickly get out of hand.
There is a shallow comparison between the two on LibHunt and also a detailed reply on StackOverflow on how to choose between the two

Since writing this article Lerna has announced that the project is no longer actively maintained, so I started followup series of articles on migrating from Lerna here.

Why Yarn workspaces and not NPM workspaces?
Well, from what I read, they both are pretty much the same. Yarn has some more time on the market (since Sep, 2017) and better documentation (which also has details about working with Lerna). I don't think there is a big difference here, so I will go with the more battle-tested solution of Yarn.

Creating my “Pedalboard” monorepo

A guitar “pedalboard” is a board (wait for it…) which you can mount any effect pedal onto, and then plug your guitar on one end, the amp on the other and use these effects to express yourself better. Maybe the analogy for monorepo and packages is a bit of a stretch but I like it so… pedalboard it is :)

Let’s get started

For the workspace I am creating a new directory representing the root project, called “pedalboard”. I then initialize yarn with the workspace flag in it:

yarn init -w
Enter fullscreen mode Exit fullscreen mode

Answering a few prompt questions and we are on our way. I don’t see anything special on the package.json that was generated on the workspace root, though. This is how my package.json looks like now:

{
   "name": "pedalboard",
   "version": "1.0.0",
   "description": "A collection of packages to help you express you software better",
   "main": "index.js",
   "author": "Matti Bar-Zeev",
   "license": "MIT",
   "private": true,
   "workspaces": [],
   "scripts": {}
}
Enter fullscreen mode Exit fullscreen mode

(I’ve added the “workspaces” and “scripts” manually)

My first package is an ESlint plugin with a single rule. I will call this package “eslint-plugin-craftsmanlint” (🥂).
Following the Lerna convention I will create a “packages” directory and put it there.
Now I can add this package name to my root package.json, but in order to make it a bit more elegant and robust I will add a glob for all the packages under the “packages” directory to be considered as workspaces:

{
   "name": "pedalboard",
   "version": "1.0.0",
   "description": "A collection of packages to help you express you software better",
   "main": "index.js",
   "author": "Matti Bar-Zeev",
   "license": "MIT",
   "private": true,
   "workspaces": [
       "packages/*"
   ],
   "scripts": {}
}
Enter fullscreen mode Exit fullscreen mode

Now i will initialize Yarn on that package:

cd packages/eslint-plugin-craftsmanlint && yarn init
Enter fullscreen mode Exit fullscreen mode

And after a few CLI questions I now have a package.json file for that newly created package:

{
 "name": "@pedalboard/eslint-plugin-craftsmanlint",
 "version": "1.0.0",
 "description": "A set of ESlint rules",
 "main": "index.js",
 "author": "Matti Bar-Zeev",
 "license": "MIT"
}
Enter fullscreen mode Exit fullscreen mode

Notice that I’m using the “@pedalboard” namespace for the package name.
Now that I have this set, it is time to put some content into the package. I will add the rule I’ve created in a previous post of mine (Creating a Custom ESLint Rule with TDD) to the package.
Navigating back to the root of the project, I run “yarn” and this is the output I get:

➜  pedalboard yarn
yarn install v1.22.17
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 0.07s.
Enter fullscreen mode Exit fullscreen mode

There is a new node_modules residing on the root project, and it has my eslint-plugin-craftsmanlint package, sym-linked to the actual code on the package:

eslint-plugin-craftsmanlint package sym-linked to the actual code on the package

(That little arrow marks that it is sym-linked).

You know me - tests are something I care deeply about, but before I jump into running test scripts from the root project, let’s step into the package itself and run the tests from there.

cd packages/eslint-plugin-craftsmanlint && yarn test
Enter fullscreen mode Exit fullscreen mode

And I get this error:

error Command "test" not found.
Enter fullscreen mode Exit fullscreen mode

Yes, of course it does not exist. Let’s create it in the package.json of that package. I am using Jest to test it so I first install Jest in that package:

yarn add jest -D
Enter fullscreen mode Exit fullscreen mode

Wow 😲, what just happened?
If I open the node_modules of my eslint-plugin package, I see that there is a “jest” package registered there, but it is sym-linkd to… the root project’s node_modules!

Jest sym-linkd to the root project’s node_modules

And indeed in the root project we have the entire dependencies of Jest in its node_modules. Nice.

Now I will add the “test” script to the eslint-plugin package.json and attempt to run the tests:

{
   "name": "@pedalboard/eslint-plugin-craftsmanlint",
   "version": "1.0.0",
   "description": "A set of ESlint rules",
   "main": "index.js",
   "author": "Matti Bar-Zeev",
   "license": "MIT",
   "scripts": {
       "test": "jest"
   },
   "devDependencies": {
       "jest": "^27.4.3"
   }
}
Enter fullscreen mode Exit fullscreen mode

Running the tests, I find out that I’m missing yet another dependency - eslint itself. Let’s add that as well.

yarn add eslint -D
Enter fullscreen mode Exit fullscreen mode

The same happens - the eslint package is installed on the root project and there is a sym-link between the inner package node_modules to the node_modules on the root project.

eslint sym-linkd to the root project’s node_modules

Yep, tests are running now and everything passes with flying colors.

So at this stage we have a root project called “pedalboard” with a single package in it named “eslint-plugin-craftsmanlint” (🥂) and the dependencies are all being taken care of by Yarn workspecs.

Adding Lerna to the pot

I have 2 more goals right now:

  • I want to be able to launch npm/yarn scripts from the root project which will run on all the packages on my monorepo
  • I want to be able to bump the package to version, along with generating a CHANGELOG.md file and git tagging it

This is where Lerna comes in.

I will start by installing and then initializing Lerna on the project. I’m using the independent mode so that each package will have it’s own version.
The “-W” is for allowing a dependency to be installed on the workspace root, and Lerna should obviously be there.

yarn add lerna -D -W
Enter fullscreen mode Exit fullscreen mode

Now I will initialize Lerna and this will create the lerna.json file for me:

npx lerna init --independent
Enter fullscreen mode Exit fullscreen mode

The "independent" param means that I would like each package to be independent and have its own separated version.

Since I would like my conventional commits to determine the version of my packages, I will add the “version” command to the lerna.json and set it as such - I will be also allowing version changes only from the “master” branch.

{
   "npmClient": "yarn",
   "command": {
       "publish": {
           "ignoreChanges": ["ignored-file", "*.md"],
           "message": "chore(release): publish %s",
           "registry": "https://registry.npmjs.org/"
       },
       "version": {
       "message": "chore(release): version %s",
       "allowBranch": "master",
           "conventionalCommits": true
       },
       "bootstrap": {
           "npmClientArgs": ["--no-package-lock"]
       }
   },
   "packages": ["packages/*"],
   "version": "independent",
}
Enter fullscreen mode Exit fullscreen mode

Notice that when you initialize Lerna for it takes a "0.0.0" version as a default, also I'm not using Lerna bootstrap (cause I have Yarn workspaces taking care of that) but I left the default configuration for it ATM.
You can check out the docs to further understand what I’ve added on top of the basic configuration

Running the tests for all packages

Ok, let’s add the “test” script to the root project’s package.json and in it we will use lerna in order to run the script on all the packages.

"scripts": {
       "test": "lerna run test"
   },
Enter fullscreen mode Exit fullscreen mode

“lerna run” will attempt to run the following script name in each package. So if I now do a yarn test on the root project, it will run the “test” script under the eslint-plugin directory.
Great! The tests are running as expected. Now it is time to move to bumping a version.

Bumping the version

The single package I have at the moment is currently on version 1.0.0 and I modified the rule code to rephrase the error message the lint rule outputs. Once done I committed that using the following conventional commit:

fix: Rephrase the lint error message
Enter fullscreen mode Exit fullscreen mode

I will run npx lerna changed from the root project to see what changed. I expect it to pick-up the single package change. Here is the outcome:

lerna notice cli v4.0.0
lerna info Assuming all packages changed
@pedalboard/eslint-plugin-craftsmanlint
lerna success found 1 package ready to publish
Enter fullscreen mode Exit fullscreen mode

Awesome! “lerna success found 1 package ready to publish”, so if I now run npx lerna version I’m supposed to see that the version is bumped in a “fix” version increment.

lerna notice cli v4.0.0
lerna info current version 0.0.0
lerna info Assuming all packages changed
lerna info getChangelogConfig Successfully resolved preset "conventional-changelog-angular"

Changes:
 - @pedalboard/eslint-plugin-craftsmanlint: 1.0.0 => 1.0.1

? Are you sure you want to create these versions? (ynH) 
Enter fullscreen mode Exit fullscreen mode

As you can see, Lerna has found my change and is about to bump the version from 1.0.0 to 1.0.1. If I confirm this action a few things will happen -
Lerna will modify the eslint-plugin-craftsmanlint package.json file with and the Lerna.json file with the new version.
Lerna will also create a change.log file with my recent change documented, both on the package and on the root project and add a git tag for this version, named v1.0.1

At the end, Lerna will push the commit and tag containing all these changes with the message that is defined on the lerna.json file: "message": "chore(release): version %s". It will replace the %s with the full version tag name, which should now be “v1.0.1”.

Once completed I have a CHANGELOG.md with the following content:

# Change Log

All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.

## 1.0.1 (2021-12-09)


### Bug Fixes

* Rephrase the lint error message ([3486b18](https://github.com/mbarzeev/pedalboard/commit/3486b1831b1891c01cb9a784253c8569ace3bc91))
Enter fullscreen mode Exit fullscreen mode

Please note that the version command does not publish anything to your selected registry (NPM, GitHub etc.) and usually it is not invoked on its own but as a part of invoking Lerna's “publish” command, but it provides a good milestone to check that bumping the versions acts as expected.

And So at this stage we have the following:

  • A root project called “pedalboard” with a single package in it named “eslint-plugin-craftsmanlint”
  • Dependencies are all being taken care of by Yarn workspecs.
  • Lerna manages npm script executions and version bumping on the monorepo’s nested packages.

As always, if you have any ideas on how to make this better or any other technique, be sure to share with the rest of us!

Coming up in the next part -

I will go for completing the E2E flow of publishing my package to NPM using GitHub actions, which basically means that when I push my commits to the master branch it will trigger a build pipeline which will test my package, bump the version and publish it to NPM automatically.

Stay tuned ;)

Hey! If you liked what you've just read check out @mattibarzeev on Twitter 🍻

Photo by Kelly Sikkema on Unsplash

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .