Allow more than 10 hubs per event hub namespace
Event hub namespaces have a hard limit of 10 hubs. It would be great to be able to raise that limit to 100 or 1000 or whatever via a support request.
We use Azure Event Hubs with Kafka. A namespace for us is a kafka server and a hub is a kafka topic. Now we have to split topics per kafka server based on Azure's 10 hubs hard limit, which is counterintuitive and complicates our setup.
Nikolaos Georgiou
shared this idea
9 comments
-
Anonymous
commented
Is there any update on this?
-
Stuart
commented
One of the most common uses in Kafka is to split each discrete type of event to its own Topic. A typical enterprise can have 100's or 1000's of event types. This allows very selective and efficient subscriptions by consumers.
But this kind of design is impossible in Event Hubs.
As a result, we're now forced to redesign - either by doing our own topic sharding across multiple EH Namespaces, or worse, to now do stay within the 10 hub limit, we mingle heterogeneuos events together and then force consumers to then do filtering and polymorphic deserialization, ruining the performance and cleanliness of the design.
-
Adriaan
commented
We hit this issue yesterday with our microservices - and ended up having to do a re-design.
Disappointing that this isn't mentioned on EventHub Pricing page - we didn't know about this limit until we got the error!
But as per comments below - this is too low - I would have expected this to be tied to TUs at the very minimum; it just doesn't make sense for us to use dedicated event hub as pricing is rediculous.
In our scenario we essentially have 15 event hub namespaces (5 for each test environments, 3 per APAC/EMEA/AMERICA regions) total. Our event hubs individually doesn't have much activity but, from a test environment and business segregation perspective it makes sense to have many event hubs per namespace.
So atm we reduced/reworked our architecture to instead combine/pump data from multiple business domain areas into same event hub as this will be more cost effective long term solution. -
Vivek
commented
Please increase # Azure Event hubs per namespace to at least 100. We are using Event hub for our Microservice applications and have 100 of services and generating many events. 10 per namespace is way too low.
-
James Newsome
commented
Makes no sense when you consider using Azure Event Hubs with Kafka and Kafka Connnect for CDC on SQL tables. Where each hub is a topic which is just a SQL table.
-
Brecht
commented
This limit is way to low.
-
Trent Telfer
commented
This limitation has actually caused us to do a proof of concept with Confluent Kafka. The lack of feature enhancement on Azure's Event Hub product is disappointing especially consider their inability to listen to the clients.
-
Ramki Palle
commented
If a public cloud provider is offering a dedicated tier for a PaaS service, it only means they did not do their job properly and are still trying to figure out the architecture about how to build this in a truly PaaS model. They are not asking us to use "dedicated" option for other PaaS services such as Azure SQL Data warehouse, Azure Blob/Data Lake Store etc. In any case, if the goal of giving a Kafka for event hubs is to ask the customers to move the existing kafka based applications to event hubs, that goal will not be achieved with this limitation. A single tenant (dedicated) option is too cost prohibitive.
-
Jose Antonio
commented
Completely agree. We have exactly the same problem, the 10 topic limit is ridiculously small and artificial, and makes Event Hubs for Kafka pretty cumbersome to use in a real use case, where it is very likely that you have more than 10 topics.