Event Hubs

Azure Event Hubs is a hyper-scale data ingestion service, fully-managed by Microsoft, that enables you to collect, store and process trillions of events from websites, apps, IoT devices, and any stream of data.

  1. Built-in Capture feature of Event Hub should preserve milliseconds in EnqueuedTimeUTC system property

    When activating the built-in capture feature in Event Hub, messages are stored in Blobs using AVRO format. In AVRO, system prop EnqueuedTimeUTC is serialized as a string, and the odd part is that milliseconds are not there. Basically, capture feature is truncating the original information, which it should not do

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Kafka protocol prevents connections from closing after a GeoDR failover

    Event Hub Disaster Recovery needs manual intervention of restarting the application to start pushing messages to secondary namespace as kafka protocol prevents connections from closing after triggering failover.

    It would be nice if the kafka can automatically have a new connection at the time of failover.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Diagnostic setting should allow EventHub SAS policy instead of Namespace SAS policy

    In Diagnostic setting for any resource when sending events to EventHub it allows only namespace SAS policy. It will be good if it allows EventhHub SAS Policy.

    Also today the option should say “EventHub Namespace policy name” instead of “Event hub policy name” as it confuses many users.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Allow Secondary EventHub's tags to be updated via ARM/PS/Policies

    When setting up Azure policy for tag inheritance, you cannot exclude secondary namespaces, even though by definition the secondary namespaces cannot be updated under any circumstances. So when you do an automated deployment the policy kicks in and deployment fails wit the error "Cannot update a namespace that is secondary".

    The only suggested way to get around this from Microsoft is to break the pairing (10 minutes) delete secondary namespace's event hubs (2-5 minutes) and set up tags enable pairing gain (10 minutes). I am setting this as the baseline, and it turns out it will be at least 25…

    36 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  5. Add aliasPrimaryConnectionString to Event Hubs ARM reference() object

    Use case: Output from ARM template the primary connection string of an event hub geo-DR alias. Similar to the 'primaryConnectionString' property, but using the alias primary connection string.

    For example, "[reference(....).primaryConnectionString]" works for the non-alias connection string, but "[reference(....).aliasPrimaryConnectionString]" doesn't exist.

    Here's the error if you try it:

    The language expression property 'aliasPrimaryConnectionString' doesn't exist, available properties are 'primaryConnectionString, secondaryConnectionString, primaryKey, secondaryKey, keyName'..

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  6. ORC format for Event Capture

    Please support ORC format for event capture. Research suggests it could be much denser for sparse data.

    FYI: It is already supported by Data Factory, SQL DW/Synapse, ADLA, and Hive (via HDInsight).

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Docker image for Event Hub in local/dev/test environments

    A container that acts like an event hub so that in local/dev/test/ci environments it's easy to spin up and tear down application stacks that are dependent on event hub.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. In azure portal Eventhub how to find AMQP links are idle, expired, active etc.

    In azure portal Eventhub how to find AMQP links are idle, expired, active etc.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. add a description free text field inside envent hub IP firewall list

    add a description free text field inside envent hub IP firewall list

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Allow Event Capture to output in different file formats

    Currently event capture only supports writing in avro. It would be nice to support other file formats for processing e.g JSON and parquet.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Please provide message and events replication for disaster recovery in the Event hub .Currently It supports metadata replication only.

    Please provide message and events replication for disaster recovery in the Event hub .Currently It supports metadata replication only.We have to poll stuck messages from old primary event hub.Event hub Disaster recovery involves manual steps .Please provide feature to replicated the messages also .So that this process become automatic.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. how to add customized EnqueuedTimeUtc to event hub message?

    if we want to delay the eventhub message execution in eventhub receiver for some dependency .

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Add an automatic scale down to better handle fluctuating workloads.

    The current auto-inflate feature is great to handle gradual increase in workload without downtime. For peak loads, however, it means billing is done on the highest throughput of a given period if no action is taken. This is detrimental to a number of use-cases.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Add reporting for metrics that impact billing, and for cost/budget forecasting.


    • TU count

    • Event.Ingress: % of actual versus total capacity

    • Events.Egress: % of actual versus total capacity

    • Throughput.Incoming: % of actual versus total capacity

    • Throughput.Outgoing: % of actual versus total capacity

    These metrics would help to answer questions:
    - Based upon our current usage and current billing structure, how can I forecast future costs as our scale increases? Similar to existing Azure Stream Analytics "SU %": this metric allowed me to roughly guage our current usage (versus capacity) and budget for future growth.
    - What was the metric that triggered auto-inflate of TU count: Events? Throughput? Incoming? Outgoing?
    - "TU count":…

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Remove the EventHub Message Size limit of 1MB

    Currently we have a size limit on the incoming messages on an EventHub as 1MB. Some processes batch the messages before sending them, to avoid making several POST requests. Even though the batch size is small as 10, the size is unpredictable when the message is sent, as it hits the 1MB mark. Please could we remove the size limit or at least increase the limit form 1MB to 5MB/10MB?

    Kafka does not have a size limit. Google has a limit of 10MB.

    16 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Update Eventhub documentation with correct aliases

    Event hub team has recently made some updates in their aliases. It would be really great if we can have proper documentation regarding event hub aliases.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Provide Event Hub viewer / explorer

    It would be good if some sort of viewer/explore was provided for looking at the events within an Event Hub. This could be done either via the portal or Azure Storage Explorer.

    When developing I currently use https://marketplace.visualstudio.com/items?itemName=Summer.azure-event-hub-explorer to view events as they are produced.

    However, it would be good if a tool existed that let you browse events within the retention period. With the possibility to query by timestamp & partition key. As well as the event body, it would be good to be able to see the properties and other metadata.

    We use Event Hub as the integration…

    19 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Let EventHub REST api for batch event sending take "Body" as a json object instead of just a string.

    The two options for batch sending right now are to use the managed client or the REST apis. The managed client lets you just slam in a bytesegment created on a buffer that holds valid json, but then you have to manually set name/value pairs in the Properties collection for any items that you want set in that collection. The REST api lets you just provide the whole UserProperties collection as a single json object, but then requires that you escape all your json to put it in the "Body" field as a string.

    If I've already got the ability…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Kafka Streaming connection with Event Hub

    In trying to be more efficient with read/writes into CosmosDB my customer is requesting that having a Kafka Stream connect to Event Hub directly will provide some signficant savings. Also Kafka is very commonly used for Streaming across various enterprises and so hopefully supporting this connectivity for Event Hub might mean a good thing.

    Some more details with an example below, there are workarounds I am sure but hope this helps:

    o We currently have several read services and several write services. With Kafka Streaming the reads are consolidated in a Kafka queue and published to Event Hub as a…

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  20. Diagnostics Logging should also log successful/unsuccessful authentication/authorization requests

    Same as in Azure Search, Cosmos DB and other components, diagnostics logging should also log 401/403 errors when e.g. trying to access Event Hubs with a wrong Token/Key.

    This is important for security reasons to setup e.g. Alerting in Log Analytics to monitor "unsuccessfull authentication/authorization" attempts

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1
  • Don't see your idea?

Event Hubs

Categories

Feedback and Knowledge Base