As part of Lab 3 for my Topics in Open Source course, I embarked on a journey to elevate my GitHub project ReadCraft. ReadCraft, my trusty README generator, had served me well, but it was time for an upgrade. The mission? Add JSON output support, real-time streaming, and the ability to handle multiple files and directories. Sounds simple, right? Spoiler alert: it wasn’t.

The Quest Begins: JSON and Streaming Together?

When I first decided to add both JSON output and streaming capabilities, I underestimated how tricky things would get. I thought, "I'll just merge the features and it’ll work perfectly!" But ReadCraft had other plans. Merging these two features felt like trying to walk a tightrope while juggling fire—one wrong move, and you’re toast.

I started by making ReadCraft output JSON data, which seemed harmless enough. But once I integrated streaming, chaos ensued. I suddenly had to make sure the tool printed results line-by-line while also generating Markdown (README.md) and JSON files without missing a beat.

Things got even crazier when I introduced the ability to read multiple files or entire directories at once. Each file needed its own output, but at the same time, it had to be part of a larger stream that was showing up in real-time in the terminal. My terminal became a circus of outputs!

The Battle with Git: Merge Conflicts Galore!

While coding was one battle, managing my Git repository was another beast. Merging the json-output and streaming branches became a nightmare. My code threw merge conflicts at me like confetti, except this confetti was not fun. Every time I thought I resolved one conflict, three more popped up—there were issues in nearly every file!

The readme_generator.py file saw the most action. In one of my commits (092caeb), I integrated streaming, but I overlooked some JSON output logic, causing ReadCraft to output in a strange format. Once I fixed that, another conflict appeared where JSON wasn’t correctly saving output to files, as seen in commit 4b8f74e.

Finally, Victory: Merging JSON, Streaming, and Directories

After hours of testing, breaking, and fixing, I finally got ReadCraft to behave. Now, it could:

1. Generate both .md and .json files from multiple source files.
2. Stream results to the terminal in real-time, so you can see the README as it’s being created.
3. Process entire directories, handling multiple files effortlessly without crashing.

Running this command now:

python readme_generator.py ./my_files --model mixtral-8x7b-32768 --api-key YOUR_API_KEY --json --stream --output-dir ./output

streams the output while saving both .md and .json files into a directory.

What I Learned: Git is Your Best Frenemy

The journey was both humbling and exhilarating. Git taught me patience—every merge conflict was like a mini-boss battle, forcing me to pay attention to every change. It’s safe to say I’ve leveled up my Git conflict-resolution skills after this lab. Also, when working with streams and JSON, timing and ordering are everything. One missing line of code or a wrongly placed argument, and the whole thing crumbles.

If you want to check out the project, head over to the ReadCraft GitHub repo. You’ll find all the commits that document the wild ride, including commit 092caeb and commit 4b8f74e.

Happy coding, and may your merge conflicts be few!