Event Hubs

Azure Event Hubs is a hyper-scale data ingestion service, fully-managed by Microsoft, that enables you to collect, store and process trillions of events from websites, apps, IoT devices, and any stream of data.

  1. Data Lake Gen2 Capture Support

    Currently you can capture to Data Lake Gen1 or Blob but not Data Lake Gen 2. Sad times.

    29 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  2. Allow more than 10 hubs per event hub namespace

    Event hub namespaces have a hard limit of 10 hubs. It would be great to be able to raise that limit to 100 or 1000 or whatever via a support request.

    We use Azure Event Hubs with Kafka. A namespace for us is a kafka server and a hub is a kafka topic. Now we have to split topics per kafka server based on Azure's 10 hubs hard limit, which is counterintuitive and complicates our setup.

    27 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. emulator

    An Azure Event Hub emulator (preferably in a container) to do local testing

    18 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. I want to see number of outgoing messages by consumer group

    I can load up graphs showing outgoing messages by the event hub, but I want to see more granularly down to outgoing messages by the consumer group.

    15 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Provide Event Hub viewer / explorer

    It would be good if some sort of viewer/explore was provided for looking at the events within an Event Hub. This could be done either via the portal or Azure Storage Explorer.

    When developing I currently use https://marketplace.visualstudio.com/items?itemName=Summer.azure-event-hub-explorer to view events as they are produced.

    However, it would be good if a tool existed that let you browse events within the retention period. With the possibility to query by timestamp & partition key. As well as the event body, it would be good to be able to see the properties and other metadata.

    We use Event Hub as the integration…

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Diagnostics Logging should also log successful/unsuccessful authentication/authorization requests

    Same as in Azure Search, Cosmos DB and other components, diagnostics logging should also log 401/403 errors when e.g. trying to access Event Hubs with a wrong Token/Key.

    This is important for security reasons to setup e.g. Alerting in Log Analytics to monitor "unsuccessfull authentication/authorization" attempts

    7 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Remove the EventHub Message Size limit of 1MB

    Currently we have a size limit on the incoming messages on an EventHub as 1MB. Some processes batch the messages before sending them, to avoid making several POST requests. Even though the batch size is small as 10, the size is unpredictable when the message is sent, as it hits the 1MB mark. Please could we remove the size limit or at least increase the limit form 1MB to 5MB/10MB?

    Kafka does not have a size limit. Google has a limit of 10MB.

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. event hubs firewall

    Provide an option to allow azure services access through the firewall for Azure Event Hubs.

    It is common practice to utilize services such as Azure Stream Analytics to consume data from Event Hubs. When enabling the firewall for the event hub service this blocks ASA from connecting. Other services such as ADLS and Azure SQL Database already have this simple option to toggle for their firewall settings. It would be great to have this common option for firewall options across all services.

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Add more fine-grained ACLs for message listeners

    Currently, a listener of an EventHub has access to all the messages within that EventHub. In the case of untrusted consumers (e.g. in a multi-tenant system), it's sometimes useful to restrict this permission to only read some of the messages in the EventHub.

    A sample scenario is for using a shared Event Hub as a multi-tenant distributed log for a service. As it stands, each customer can already have their own publisher key to the shared Event Hub. It would also be nice if each customer can get a SAS key that only allows them to listen to messages from…

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Kafka Streaming connection with Event Hub

    In trying to be more efficient with read/writes into CosmosDB my customer is requesting that having a Kafka Stream connect to Event Hub directly will provide some signficant savings. Also Kafka is very commonly used for Streaming across various enterprises and so hopefully supporting this connectivity for Event Hub might mean a good thing.

    Some more details with an example below, there are workarounds I am sure but hope this helps:

    o We currently have several read services and several write services. With Kafka Streaming the reads are consolidated in a Kafka queue and published to Event Hub as a…

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  11. Please add VNET & SE configuration

    We have VNETs configured with express route for on-prem connectivity. Can we have VNETs and SE configuration to connect EVentHub securely from VNET to EventHubs.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. please add a group of ip at one time in IP filter

    We try to add 30+ IPs of specified region in azure https://www.microsoft.com/en-sg/download/details.aspx?id=41653. But we find that we have to add it one by one. It is brute force.

    Please help adding a function to add a group of ip/range. We can separate different ip/range with ; or ,

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Show Event Hub usage and quota in Usage + Quotas.

    Please consider showing the Event Hub usage and quota in Usage + Quotas just like it is shown for Storage Accounts.

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Support of compression when using Kafka connector

    We are streaming a lot of JSON to Event Hub, unfortunately it does not support payload compression. We really would like to enable that to not hit any limits and save on throughput units.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Add information about update on the backend server of namespace

    We received lot of error in
    Metric Namespace : Event Hub standard metrics
    Metric : User Errors. (Preview)

    The support Microsoft Azure told me :

    There is an ongoing update on the backend server for your namespace, during these updates it is expected that you are going to see User Error and Server Errors on a few requests.
    For all these errors the detailed error message should surface into your application code, if you are not seeing any exception on your application probably your client application SDK is retrying the operation successfully so no error is thrown.
    These service/OS updates…

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Add reporting for metrics that impact billing, and for cost/budget forecasting.

    - TU count
    - Event.Ingress: % of actual versus total capacity
    - Events.Egress: % of actual versus total capacity
    - Throughput.Incoming: % of actual versus total capacity
    - Throughput.Outgoing: % of actual versus total capacity

    These metrics would help to answer questions:
    - Based upon our current usage and current billing structure, how can I forecast future costs as our scale increases? Similar to existing Azure Stream Analytics "SU %": this metric allowed me to roughly guage our current usage (versus capacity) and budget for future growth.
    - What was the metric that triggered auto-inflate of TU count: Events? Throughput?…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Please increase the number of event hubs per namespace

    Please increase the number of event hubs per namespace. 10 hubs are not enough.
    Please increase the current limit from 10 to 100 on standard tier.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Emit EventGrid messages when the EventHub is scaled up/down

    Downstream services that consume EventHubs sometimes need to know when an EventHub is scaled up or down so they can react and scale accordingly. It would be really useful if an EventHub emitted this into EventGrid so that downstream services can listen for it and react to it.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Injecting Dynamic Parameters in Event Hubs Capture File Path

    Currently Event Hubs Capture automatically stores events into Data Lake based on predefined paths with placeholders. It would be great if additional parameters could be injected in the path based on event properties dynamically.

    Example: {Namespace}/{EventHub}/{TenantId}/{PartitionId}/{Year}/{Month}/{Day}/{Hour}/{Minute}/{Second}

    TenantId is a property found in every event denoting the originating tenant of the event. This ensures that the Data Lake directory structure can reflect dynamic business requirements.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Please provide message and events replication for disaster recovery in the Event hub .Currently It supports metadata replication only.

    Please provide message and events replication for disaster recovery in the Event hub .Currently It supports metadata replication only.We have to poll stuck messages from old primary event hub.Event hub Disaster recovery involves manual steps .Please provide feature to replicated the messages also .So that this process become automatic.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1
  • Don't see your idea?

Event Hubs

Categories

Feedback and Knowledge Base