For the last week of Hacktoberfest, I wanted to do something special. I wanted to contribute to a much larger repository than I had contributed to so far!

The Issue

What project would fit better than one where the theme is similar to our labs? On to Hugging Face repositories!

After browsing their repositories, I set my mind on the chat-ui project. Chat-ui is the open-source codebase powering the HuggingChat app! The HuggingChat app is a chat-based web app that allows interaction with a variety of different language models, such as llama, cohere, and mistralai models, and more!

The issue I chose discussed implementing a new feature for the existing model configuration in the app. While this is something I personally did not know before, apparently some models do not support system role prompts, but rather only user role prompts. Therefore, models that do not support system role prompts failed to work on the app.

My task was to implement a new model configuration option that allows the user to specify whether the model in use supports or does not support system role prompts.

Process

The setup process took quite a long time (another downside of being a Windows user), even though I was using WSL, which powers an Ubuntu subsystem on my PC. Not only that, but building and running the llama.cpp server was not only time-consuming but also extremely resource-intensive. I could feel the hit my PC's performance took when I got it up and running. Nevertheless, once it was all set up, and I made sure everything worked, it was time to start development!

The very first thing I did was search for modelConfig in all the files, because it was pointed out in the issue that the new option should be added there. I immediately found the file in question.

The next step was to familiarize myself with the code. I noticed they are using Zod for validation and type inference, and I could see where all the user prompts are being passed, and system prompts are being set.

After identifying where I needed to insert the new feature, I made sure to make the least amount of changes possible to support this new feature. I added a new variable to the Zod validator, systemRoleSupported which is true by default, and simply filtered out all system prompts before they were passed to the models.

I also ensured that hard-coded system prompts preferred user prompts if the user specified that the model does not support the system prompt.

After a lot of code reading and testing, my pull request was up. I got one tiny requested change from the repo maintainer asking for a better name for the new model config option, but once that was addressed, all checks passed, and the pull request is now ready to be merged!

Hacktoberfest Conclusion

Overall, I would say this was one of the most fun and engaging experiences I've had in my program yet. The idea of contributing to random projects supported by strangers with similar goals is exhilarating. Seeing every open-source contributor and maintainer following the best practices that we were taught, and communicating effectively, was inspiring, and yet another proof that I was born to do this, because it's just so much fun!