How can we improve Azure Event Grid?

Advanced filtering scenarios, Subject Wildcard, Regex & Data Filters

The subscription filters are very limited, I can only filter the events based on subject and Event Type, and the current filter options does not help much for advanced scenarios.

It would be more flexible if we could define filters by regex or wildcard on EventTypes and Subjects, and also have filters on data attributes

- EventType using WildCards.
- -> Use: [ "Microsoft.Storage.*", "Microsoft.EventHub.*"]
- -> Instead of: [ "Microsoft.Storage.BlobCreated", "Microsoft.Storage.BlobDeleted" ]

- Subject filter by WildCard, Contains or Regex:
- -> Use > subject : match("blobServices/default/containers/mycontainer/log/*.jpg")
- -> Or > subject : match("blobServices/default/containers/mycontainer/log/*.[jpg|gif|png]")
- -> Or > subject : contains([.jpg|.gif|.png])
- -> Or > subject : endsWith([.jpg|.gif|.png])

- -> Instead of:
-----{ "subjectBeginsWith": "blobServices/default/containers/mycontainer/log",
-----{ "subjectEndsWith": ".jpg",

The current options we can't have for example a filter by images, so we should create multiple similar subscriptions for different extensions or multiple folders.

I also would like to filter by the data content, i.e:
- - > Use: "data.sizeInBytes" : lessThan(10000)
- - > Or: "data.supportPlan" : contains("Enterprise","Business")
- - > Instead of: Currently there is no way to filter data attributes and does not have multiple conditional filters on subject

We cannot filter for attributes in the data payload, so we could avoid for example processing files too big or too small, redirecting it to right services.
Another example we could have different priorities inside the payload and each priority could be handled by a different service\queue

17 votes
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)

We’ll send you updates on this idea

Diego shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

With our new advanced filters, we’ve added data attribute filters and StringContains as an operator (among many others). Take a look and let us know what you think!

We will likely not add wildcards and regex for quite some time due to the compute and latency costs it would incur, however, we think, most of the use-cases described (and more) can be addressed with our new filters: https://docs.microsoft.com/en-us/azure/event-grid/event-filtering#advanced-filtering

2 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...
  • Anonymous commented  ·   ·  Flag as inappropriate

    how do I check if something is null using advanced filters?

    Like Data.Whatever is null ?

  • Richard commented  ·   ·  Flag as inappropriate

    Great suggestions. In my case, I have multiple publishers that generate similar events (e.g., basic CRUD and other custom events) and I'd like to be able to exclude some events (or subjects) from going into a subscription. My solution right now is to enumerate each unique combination of event sources and types and manually enter each event type into each subscription. So cumbersome and unmaintainable.

Feedback and Knowledge Base