Azure Data Explorer

How can we improve Azure Data Explorer?

(thinking…)

Enter your idea and we'll search to see if someone has already suggested it.

If a similar idea already exists, you can support and comment on it.

If it doesn't exist, you can post your idea so others can support it.

Enter your idea and we'll search to see if someone has already suggested it.

  1. Cross-region follower support

    Our main way of doing resource governance is to rely on follower databases. We have a locked down production cluster which doesn't accept any non-production workload and then if anyone else needs access to the data, we have them set up a follower database. It sounds like ADX has a single-region restriction for those follower databases and this is going to put a pretty major kink in our arch. Please consider adding this feature.

    21 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    6 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Provide out of the box resource governance for Kusto clusters

    Our Kusto cluster is used for production site and tools and needs to have 99.99% availability. However, as our cluster is open to tens of thousands of users across the company for ad-hoc queries, we often suffer from unexpected cluster wide outages due to misbehaved customers. We have tried both cluster-wide query limits as well as customized approaches, however, neither is helping us much. We would like to see Kusto to provide the resource governance solution out of box.

    26 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    4 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Add ".update" management command to allow correct wrong ingested data

    When data gets wrongly ingested into ADX, it very hard to correct this data afterwards. -> I know ADX is a append-only store.

    Let's say a sending machine had issues with daylight saving time and all data is shifted and after some days, it gets recognized and the machine is sending correct values. There is small slice of data, which needs to be aligned.

    Or some properties where wrongly set over some time. Same as above.

    Currently, we use the ".purge" command and re-ingest the data, which leads to a wrong "ingestion-time". Another option is completely re-ingest the data into…

    11 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. 6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Autoscale form 1 VM on Azure Data Explorer's Data Management cluster

    Autoscale form 1 VM on Azure Data Explorer's Data Management cluster

    6 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Permissions by views

    I would be useful to be able to restrict specific users to only be able to access data to a specific view
    (without direct access underlying tables). This could enable use to limit specific users access to time series data for a specific period or specific equipment.

    8 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  7. Allow creation of Tables & Data Maps in ARM template

    It doesn't make much sense that ARM template deploy of AzDEx allows for creation of Clusters, Databases, and DataConnections when DataConnections expliclity depend on a Table and DataMap to exist. In fact the error message is simply "Bad Input" with an "Internal Server Error" if somebody mistakenly tries to do this..

    DEx should be able to be fully-configurable via ARM including Tables and Datamaps so the existing capability of creating DataConnections via ARM is able to actually be used.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Follow at the table level

    We love the database follower feature but it would be fantastic if we could follow at a table level.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  9. Integration with Azure portal dashboard. Create a tile in portal.azure.com that queries a kusto cluster.

    I'd like to be able to create a dashboard on the Azure portal that executes kusto queries against any kusto cluster. Today, you can pin any azure monitor kusto query to the dashboard. I'd like to do the same for my own kusto cluster.

    There is a feature to export to Power BI, but it would be great if there was integration Azure portal as well

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Ignore (cold) cached data

    When perf testing a query, it would be useful to wipe out any cold data that has been temporarily cached on the local disk so that I can get apples-to-apples comparisons between subsequent runs. I can imagine it either being something where I just set a flag to ignore any of the cold cached data or a command that would completely wipe out any of that type of cached data. Note that I would not expect any of this to affect the standard hot cache data which always lives on the disk.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Create New Management Function to Create Data Connections

    You can script deployment of a Data Explorer cluster using an ARM template. Everything works, except data connections. This is because the validation requires target tables to exists before creating a data connection.

    I suggest a new management function is created that works something like this:

    .create data connection with (name="", kind="EventGrid", storageAccountResourceId="", eventHubResourceId="", consumerGroup="", tableName="", dataFormat="", mappingRuleName="")

    This way I can create data connection in a .csl file along with tables, functions, policies etc.

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Support drop all functions in a db.

    Please provide an easy way to drop all functions in a db.

    this does not work:
    ".drop function <| .show functions"

    The currently proposed workaround from the Kusto Team does not appear user friendly:

    a. Run the following command

    .show functions
    | summarize run_me_with_caution_after_carefully_reviewing = strcat_array(make_list(strcat(".drop function ['", Name, "']")), "\n")

    b. Copy the result and save as a script (text file)
    c. Use Kusto.Cli to run the `script`
    d. [hope your team members don’t get mad at you. or me. If anyone asks – a) I had nothing to do with functions getting unintentionally dropped; b) they can be…

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Mapping EventData properties during Event Hub ingestion

    When configuring a JSON Ingestion Mappings pre-created on a table, we couldn't map and access Properties which are set by Event Hub service in the EventData.SystemPropertiesCollection structure of each received message. These Properties are:
    • PartitionId
    • SequenceNumber
    • Offset
    • EnqueuedTimeUtc

    But also we could have to map any custom properties that we want to store in the same collection when sending messages to event hub.

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Invoke a function from another database

    Currently I can only invoke a function if it is in the same database. I would like to invoke functions in other databases (and ideally in other clusters.) There are two main use cases:
    1) We have some useful helper functions that use the invoke pattern. We have to copy them to all of our databases.
    2) When doing .set-or-append commands, I'm stuck if my query involves a an invoked function on database A but I want the results written to database B.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Ability to view and change data management endpoint SKU from Azure Portal

    Currently it's required to create a support ticket in order to change the data management endpoint SKU when there is a need for scaling the ingestion through event hub to a database table in ADX. The ability to view and change the endpoint scaling from the portal would be a great feature.

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. LINQ support / Fluent Query Builder APIs for .NET and Python

    The Kusto SDKs are very good, but it would be even better if instead of handling Kusto queries as strings, we could write them using a fluent query builder library. It would be awesome if Kusto supported LINQ for .NET, meanwhile for Python, a simple query builder DSL should be possible.

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Supply syntax to use Service-specific schemas no general the Top-level diagnostic logs schema

    Can we supply syntax to use the [Service-specific schemas](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-logs-schema#service-specific-schemas-for-resource-diagnostic-logs) template for specific Azure services, not general the [Top-level diagnostic logs schema](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-logs-schema#top-level-diagnostic-logs-schema). I don't want to type each column property with `project operator` all the time. Is it possible?

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  18. project wildcard support

    It would be great if we could provide wildcard support for the project statement. This would be helpful in prioritizing certain columns visually (i.e. column 'x' first, column 'y' second) while still accommodating for the dynamic nature of Kusto function output.
    When creating functions in KQL you typically end up with dependencies which cause a version of one function to return more \ different columns than before. Right now, for me to reorder columns I would need to project them all individually - and when versioning functions this can have a huge trickle-down effect that causes a bunch of unnecessary…

    4 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  19. Row level security

    Add read and write access at row level within a table

    13 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  20. Azure data explorer ingesion: json mapping only in single line

    I’m ingesting data to Azure Data Explorer from Logstash and figured out a feature that makes uncomfortable to use.
    If I put ‘enter’ between query, it occurs syntax error.
    (e.g. .If I write [create table sampletable
    ingestion json mapping ‘samplemapping’ (…)], it doesn’t work. If I write with one line, it works with no error)

    If it fixed, it will be very helpful to introduce ADX to customers :)

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3
  • Don't see your idea?

Azure Data Explorer

Feedback and Knowledge Base