You should (probably) fire your Kafka developer

Thomas Hansen - Jul 10 '22 - - Dev Community

We've all seen some of our fellow colleagues entering euphoria and la-la-land as we mention technologies such as ...

  • Kafka
  • Pulsar
  • ActiveMQ
  • Solace
  • Etc ...

However, the dirty secret of our industry is that 99.999999999999% of the times some developer implements a message broker, the result becomes 1,000 times more unnecessary complexity, implying your project becomes undeliverable, and your employer ends up wasting money for nothing.

Say after me; We don't need a message broker!

I have seen the same thing happen over and over again. Some developer with a hole in his CV is given the task of solving some problem. The problem could easily have been solved with a MySQL database and a couple of CRUD endpoints, but because the developer is trying to fill holes in his CV, he ends up implementing it using 37 micro services, with cross process communicating, through some "hot and cool message broker thing", orchestrating this architectural nightmare from an astronaut's perspective. Using a message broker when you don't need it, is like building a space shuttle to go shopping at SafeWay.

Kafka is MADNESS unless you need it, and you almost never need it!

Kafka, Solace, ActiveMQ, and all associated technologies results in added complexity, and unless you absolutely need it, this added complexity is literally like stealing from your employer. Your employer pays you for your time. You wasting time configuring Pulsar when it is not needed, ensures your employer is literally flushing his money down the toilet, while you mess around with 100+ YAML files, each 1,000 lines of code, forcing you to spend 3 weeks simply to read the documentation before you can apply things correctly. And yes, simply configuring Pulsar to run correctly easily takes a month, even for a senior developer.

For the record, I don't mean to pick on Pulsar specifically here, it's an amazing piece of tech, however just like all the other message broker implementations, is is almost never, ever, ever, ever, ever, ever, ever, ever needed!

The above implies that unless you really need a message broker, you have effectively stolen your salary from your employer, by wasting your time doing unnecessary things. Quite frankly, I find it fascinating that employers even pay people such as this ...

You should be ashamed of yourself, and somebody should fire you!

For the record, I will be sending this article to your manager ... ;)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .