Welcome to the third issue of the WarpStream Newsletter!

We have added a ton of new features since our last newsletter, and we’re excited to share them all with you in this update. We also started a YouTube channel that we’ll be using to share video content. Subscribe to our channel to stay up to date on conference talks, informational videos, interviews, and other content.

Also, a few of us are in Bangalore for Kafka Summit this week. If you are attending Kafka Summit Bangalore, drop by our booth and say hello!

Join us on X , Slack, and ** Discord **!

What’s new

Blog post: Tiered Storage Won’t Fix Kafka

The tiered storage architecture has been proposed as a solution to many of Kafka’s problems. Unfortunately, it doesn’t really live up to the hype.

We released a blog post this week discussing the fundamental flaws with tiered storage for Kafka. In this post, we argue in favor of a disk-less architecture, not a bolt-on solution that causes at least as many problems as it aims to solve. Check it out, and let us know what you think.

Blog post: Cloud Disks are (Really!) Expensive

It’s a fact: disks are expensive in the cloud. But many Kafka users underestimate just how expensive the disks actually are. So we wrote a blog post that breaks down the cost of the local storage for a Kafka cluster.

Support for multiple control plane regions

You can now configure your BYOC clusters in WarpStream to communicate with an instance of the WarpStream control plane in three additional globally distributed regions in AWS. The supported regions are:

• us-east-1
• us-west-2
• eu-central-1
• ap-southeast-1

To reduce round-trip request latency, configure your WarpStream agents to communicate with the control plane region that is geographically closest to where your agents are deployed. Review the documents to learn more.

We also now support deploying Serverless clusters in both us-east-1 and ap-southeast-1.

Setting up a new control plane region is straightforward, so you’re interested in another region, let us know and we’ll deploy the control plane there.

Add your team

You can now invite your team members to your WarpStream account so you can collaborate in the same workspace. Navigate to the Teams page in the console to invite your team.

Support for mTLS

WarpStream now supports mTLS between your clients and WarpStream agents. To learn more about how to configure this feature, review the docs.

Let us know if you have any questions!

Benthos integration (beta)

The WarpStream Agent now has built-in support for Benthos, an open-source framework for streaming transformations and integrations. You can use Benthos to integrate WarpStream with other systems, such as databases and other messaging systems, using a simple, easy to use configuration.

You can read more about WarpStream’s built-in Benthos support on our blog. Or, check out our docs to learn how to configure Benthos on WarpStream. This feature is currently in beta, so feel free to try it out and give us feedback.

Interested in learning more about WarpStream?

Book a call

What’s next

Kafka Transactions

The team is currently working on support for Transactions for Kafka. This is the last remaining major Kafka protocol feature missing from WarpStream. We expect to release support for Transactions in the next few weeks.

Transactions are often used in stream processing workloads, so we are excited to be able to onboard those use cases! If you’re interested in learning more about our work in this area, contact us and we would be more than happy to tell you about it.

Built-in offset preserving replication

One of the biggest hurdles to migrating to WarpStream is the ability to transparently migrate clients from a Kafka cluster to a WarpStream cluster. We are currently working on a solution that will provide replication for both data and metadata, so you can directly mirror a Kafka topic (or topics) from any Kafka (or Kafka-compatible) cluster to your WarpStream cluster. Unlike existing replication solutions, such as MirrorMaker, WarpStream’s built-in replication will also preserve offsets, so you will be able to transparently switch consumer clients from your Kafka cluster to WarpStream.

Schema Validation and Schema Registry

We are currently working on building full schema validation into the Agent. This feature will work with an external schema registry. We will then follow up with our own implementation of a WarpStream schema registry, so soon you won’t have to run an external system to ensure that your data conforms to the expected schema.

Sign up and try WarpStream for free!

WarpStream is free to try.Sign up now and discover how you can save 80% on your total cost of running Kafka.