Kafka Streaming connection with Event Hub
In trying to be more efficient with read/writes into CosmosDB my customer is requesting that having a Kafka Stream connect to Event Hub directly will provide some signficant savings. Also Kafka is very commonly used for Streaming across various enterprises and so hopefully supporting this connectivity for Event Hub might mean a good thing.
Some more details with an example below, there are workarounds I am sure but hope this helps:
o We currently have several read services and several write services. With Kafka Streaming the reads are consolidated in a Kafka queue and published to Event Hub as a single read or write to CosmosDB. Without this in place, there will be several reads or several writes with Event Hub to CosmosDB.
Increased traffic will have a slower throughput
Resulting traffic will have an increased cost in CosmosDB
Event Hub's support for Kafka Streaming can possibly help.
Siju T Thomas commented
In addition to the above-mentioned points, if Kafka Stream can read topics from Event hub and do the transformation on multiple messages before writing to CosmosDB, it could avoid several IOs to CosmosDB. Considering the read/write from/to CosmosDB is a costlier operation, it is important to enabling Kafka Stream capability with Event Hub to optimize the cost.