has a vendor ever told you that open source Kafka is expensive?...

That was perplexing... 🤔
Confluent had recently released a competitor product called Confluent Freigh which promised to be up to 90% cheaper than Kafka.
So we weren't comparing the right thing.
It is a super confusing input that can subtly change.
But let's call that my bad.
I don’t think Kafka would work at all with that many brokers.
And if it did - it would be operationally impossible to manage.
But it’s true that without tiered storage you need a lot of instances (or use JBOD - multiple EBS disks per node)
Kafka was made to run on HDDs. Each broker in this deployment writes just 11.1 MiB/s to the disk.
and the cheapest AWS HDD (sc1) gives you 192 MiB/s at these sizes.
it's also 6.66x times cheaper 😈
• sc1s give you just 192 IOPS
• Kafka has the OS batch larger IO writes from pagecache into disk due to the ample memory its deployed with and pagecache
• EBS HDDs have their own batching - they merge sequential IOs
but chances are if you're evaluating WarpStream, you don't care that much about latency.
(sc1 will usually give you better latency than S3)
And 1.3x cheaper in GCP
So think again before you believe a marketing narrative 💡
source: gist.github.com/stanislavkozlo…
I spent the last two months coding up an in-depth Apache Kafka deployment calculator, and today I’m releasing it.
See it here: 2minutestreaming.com/apache-kafka-c…
I worked in Confluent for 6 years. Some people may see it as a bit of a backstab to speak out against a company they acquired.
I wanted to write out my thoughts in detail so there aren’t any questions about intention, malice, agenda, etc.
• I have only good things to say about Confluent. The company was always transparent, the founders are great people and the culture in there was great.
• I worked with talented, honest co-workers and friends there.
• I have equally published multiple posts praising Confluent's product and the company. And will continue to when the opportunity arises.
• I assume best intentions, namely that Confluent didn't know/notice, and that they will correct this.
• I see WarpStream and Confluent as separate, at least in the actions they’ve taken and the culture they've shown.
• I do think WarpStream has a very strong team and an amazing product, it’s a true testament of excellence what they managed to build.
• I think what WarpStream did in marketing was not right with relation to the open source Apache Kafka project.
I therefore believe a call out and a bit of parody in response is therefore fair game.
I don’t mean to build my brand off drama and controversy. I’ll be going back to my regularly-scheduled programming right after this post.
I realize the initial disclaimers on my newsletters were too vague regarding my intention, so I have updated them there too.
I don’t have a bone to pick. I realize other companies make similar non-truthful claims in their marketing too. And I think they should be called out too. 👮♂️
Especially if it goes too far.
I tried very hard to keep everything objective and backed by data. I have only used data I've publicly observed and expressed my opinion based on it.
I don't think I should censor myself simply and cherry-pick who I call out simply because I have some indirect history with them.
If there are any strict errors - please tell me, I will correct them and apologize.
I would prefer to say the truth. I would say it to a friend too.
If you don’t agree, then perhaps we can agree to disagree. 🤷♂️
You’re free to unfollow me and hold your own views. ✌️
It’s just some pixels of a post on the internet after all.
But I hold no negativity. Hopefully you don’t either. 😇
And if you side with the truth? Smash the repost button :)





































