Dapr and Kafka-easy binding

Lucas Jellema
7 min readJan 9, 2022

Dapr is a runtime framework for distributed applications. Dapr also provides a personal assistant to any application and environment that offers standardized access to different facilities (state, secrets, configuration, pub/sub, actors) and technologies. Simply by using Dapr we can leverage a large number of Dapr components that through the Dapr runtime provide access to a wide variety to cloud services and on prem technologies. The next figure shows some of these.

Some of the Cloud Services and On Prem Technologies that Dapr makes readily available to applications

Some of the heavy lifting an application developer typically has to implement for interacting with the specific APIs and intricacies of technologies is taken over by Dapr, allowing the application developers to focus more on business functionality instead of what is really often not much more than plumbing.

In this article, I will show a quick and simple example of interaction with Kafka. It arose from a session on Microcks and AsyncAPI and I needed a quick way to:

  • consume the events published by the Mock producer generated by Microcks (from the AsyncAPI interface definition) to a Kafka Topic
  • publish events according to the same AsyncAPI contract to a Kafka Topic to demonstrate how Microcks performs a test on these events

I have worked with Kafka before, but quickly ramping up client applications — and configuring their connection to the Kafka broker — always takes me longer than I would hope for. For what I needed, Dapr offers a simple and quick solution. And it also demonstrates quite well how the Dapr approach of components-configured-outside-of-applications works.

Consume Events Published to Kafka Topic

The first case implemented using Dapr can be visualized like this:

Dapr used to consume events from Kafka Topic (on behalf of Node application with no Kafka-knowledge)

The consuming application is shown here on the far right as a Node application. It could have been any technology that can handle an HTTP POST request — any semi modern programming language would do. The code required for receiving the event from Dapr (after Dapr has taken care of the interaction with Kafka) is very, very simple (and of course using Express for such a simple…

--

--

Lucas Jellema

Lucas Jellema is CTO and IT architect at Conclusion, The Netherlands. He is Oracle ACE Director, one time JavaOne Rockstar and programmer