didi-ops

Recently the didi repository has gone a bit quiet, this in part is due to the frustrations I have been having around monorepo tooling and the enormous headache that comes with stitching it all together. I had trouble with:

• developing typescript packages.
• linking the distributed code all together and keeping the versions in sync.
• preprocessing release output so I could vet the contents of package.json's
• adding boilerpalte without copy and pasting over and over and over (adding new packages became a drag)

I wont name names but let's just say even with the popular tools out there, this pipeline is hard to stitch together. What I wanted was a tool that could generate my packages with sensible defaults and know the distinction between libs and clients (functionality and consumers of functionality), an important architectural decision didi subscribes too.

I wanted to use just NPM scripts to do all the heavy lifting, but it became crazy complex pretty quickly, scripts all over the place, 9 or so individual scripts per package, a ton at the root as well. I then stumbled upon something tucked away in the NPM docs.

process.env.npm_lifecycle_event // magic

With this, I can detect the key of the NPM script and place a big switch statement in a single file, and so didi-ops was born. didi-ops has been quite a lot of work but the result is pretty great, let me show you.

In one package.json at the root of the didi mono-repo we have just the following scripts:

Some of you will know that $npm_package_config is just a package.json variable.

"config": {
    "ops": "node ./ops/ops.js"
},

So you might be thinking, hold on, all those scripts point to the same file? Yep, I hid the mess in a massive node script 537 lines of dev-ops quality scripting (in other words, its enough to work). How then does one script do all this for any package I have added to didi or will add in the future?

Prompts, lots of prompts.

So how does it know what package is installed?
There is a .json manifest keeping track of clients and libs, this means that there can be several differences in the prompts and wizards depending on the package type.
Here I'll show you what sort of things are tracked.

But its not just tracking, didi uses template-dirs to generate all the files and folders a client or a libs package needs, all the test setup, the tsconfig, the package.json, README and more, even the comments inside these files.
Additionally bases files have been used where possible to inherit tsconfig and others, this keeps package configuration lean.

Docs

So now didi has its own package generator, is this enough to attract developers? I mean, it's about as easy as it gets to create new packages? No, this is not enough, because docs.

That leads me to the other reason I have been so quiet, I have been exploring static documentation generators, given that I can already generate the API Reference using typedoc, I really needed a more uniform way of integrating this with something I could write guides for. I tried Docz, It was promising but buggy and just took me to long to work with. you just know when something isn't working its time to stop. So I stopped and looked around again, I found Docusaurus 2. Firstly its fitting for a dinosaur like didi! But most importantly it was working very well out of the box! I had a lot more to show for myself in 5 mins of Docusaurus.

Docusaurus sadly doesn't really know the distinction between API References and Docs, so I had to do some acrobatics to make this work. Then tie this up to didi-ops and now I have a wizard to select which packages to build docs for, using typedoc and the markdown plugin.

Oodles of content!

(it's a bit rough around the edges, we have an issue open for that)

Is this work upstream yet? Not yet yes take a look at master branch, I have done so much of this work off of tickets and branches that I now need to get integrate this back into an issue, this will be a challenge, but it was worth it. Then I can finally say that didi's infrastructure is done and get some features sorted beyond its current feature set.

npm install -g @didi-js/client-didi-cli

# or

yarn global add @didi-js/client-didi-cli

const colorThief = require('colorThief'); // base specifier example, no paths needed

import colorThief from "color-thief"; // still no path?

http://didi.land < website coming soonish.

progress report