DDD Europe 2025 - Program

Getting Your Kafka Costs Under Control: A Quest for More Transparency and Accountability

EventCentric - Talk (50min)

Speakers

Alexander Rovner and Ralph M. Debusmann

Alexander RovnerRalph M. Debusmann
Date
Description

Especially if used at enterprise scale, much of the conversation in the Kafka space is centered around reducing the overall costs. Whatever costs are incurred after these cost reduction efforts should be passed on to the applications and teams incurring them.

Surprisingly, neither self-hosted Kafka deployments (including Strimzi) nor managed products from popular vendors (e.g. Confluent) allow to distribute these costs in a fair, fine-granular manner based on the usage by the individual development teams.

In this presentation, Alexander Rovner (SPOUD) and Ralph Debusmann (MGB) will describe their journey towards the usage-based distribution of Kafka costs among teams within a large organization - which can serve as a blueprint also for Kafka cost distribution at smaller companies. We will also show that having a fair, usage-based cost distribution scheme in place can help the development teams to focus on more sensibly using Kafka (e.g. no overpartitioning) and thus lead to additional benefits beyond just the cost aspect.

About Alexander Rovner

Alexander Rovner is a software engineer at SPOUD, Switzerland's leading consultancy for event streaming. His first contact with event streaming was using Apache Kafka to build microservices at Helvetia Insurance Group. Since joining SPOUD in 2023, he has supported some of the largest companies in Switzerland, such as MGB and Swisscom, in using Kafka to process terabytes of data every day and deliver timely insights. His particular interest is discovering cost-efficient ways to analyze and visualize stream data. When not thinking about event streaming, he enjoys hiking in the Alps and playing board games.

About Ralph M. Debusmann

Ralph is a former AI/NLP researcher turned software engineer, solution architect and technologist, now acting as the Lead Enterprise Kafka Engineer at Migros-Genossenschafts-Bund in Zuerich, Switzerland. He has received his PhD in computer science focusing on Natural Language Processing and Artificial Intelligence in 2006 (Saarbruecken University and University of Edinburgh) and has spent 15 years at SAP, Bosch and Forecasty.AI/BASF SE before joining Migros-Genossenschafts-Bund in 2023.