Azure Data Explorer

  1. todatetime() does not comply to ISO8601 when converting milisecs

    ISO8601 states that BOTH , and . can be used as milisec separator
    todatetime only understands "."

    "2020-06-30 08:19:59.775" currently parses ok, but
    "2020-06-30 08:19:59,775" fails.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Add AllDatabasesMonitor role

    I’d like to be able to give a user or app the Monitor role for any database created on a cluster that I own. In the portal, I see that I can give AllDatabasesAdmin or AllDatabasesViewer. Can we please get a similar way to provide Monitor to all databases that get created on the cluster?

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Rename Azure Data Explorer to Kusto

    Searching the Internet for Azure Data Explorer is hard because the name matches too many othe things. Pleasu just take the old name Kusto to make searching for answers easier

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Column retention policy

    Feature Suggestion:
    Add support for retention policy in a column level.

    Why?
    As part of our work on keeping privacy and GDPR compliances we are making sure that we are storing only data that we are allow to and only to a specific period.
    I faced a weird situation where I got a an approval to keep all my table’s data for two years period but I had to exclude ONE column as the allow retention period for this column is only 3 months.

    All today solutions are not good enough (remove the column, set the entire table retention for…

    3 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Allow accessing other databases within a RLS let statement

    New RLS functionality has allowed our team to create RLS functions that are data driven; identity to permission associations come from a table.

    However, tables and objects in other databases may not be accessed from within the let statement of the RLS function. In our case, that means we need to copy two tables to each of up to 10+ databases in our two clusters.

    Can the restrictions against the "database" notation be relaxed within the let statement of RLS functions?

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Please support Delta Lake as a source format

    Currently ADX supports parquet as a data source, but not Delta Lake. Delta Lake is an open-source storage layer that brings ACID
    transactions to Apache Sparks (on top of parquet, see delta.io for more details). Delta Lake is supported in Azure Databricks and Azure Synapse Spark, but currently not supported as data source (e. g. external tables) in ADX.
    Currently Delta Lake data has to be exported to parquet to be consistently consumable by ADX.

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Set metadata when creating files in Azure Storage

    It would be nice if the commands which create files in Azure Storage - .create external table and .export to - had some additional 'with' parameters which allowed metadata to be set in the resulting blobs. For example, a compressed JSON file could have content-encoding set to 'gzip' and content-type set to 'text/javascript'.

    This would help us with an application where we use ADX to down-sample and export data to Storage, which is then directly consumed by client web browsers. Browsers will automatically un-compress files with the appropriate content encoding, and respect cache headers.

    If we could have options to…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Custom Code Ingest

    When ingesting files only the supported formats are possible. In order to ingest other files you have to convert them first. It would be great to get the ingest data as byte array or stream and then process it inside C# or Python. The result would be a DataTable / DataFrame that gets ingested in the end.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. totimespan() should parse ISO8601 formatted timespans

    Strings in ISO8601 timespan format are not being converted.
    For example:
    totimespan("PT0.517S") returns null.

    We are storing duration timespans with that format and now have to do our own string parsing.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Add shortcuts for panel management

    I use kusto with accessibility software and shortcuts are important to be able to navigate the ui without needing to use a mouse. You have a lot of wonderful shortcuts already, but I would like to request that you add some for:
    1) Closing an active panel (potentially one that you opened like the filter panel)
    2) Resizing a panel (Both up down and left right). I believe Visual studio code has shortcuts like this.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Add filename metadata to be added Event Grid Subscription from Storage options

    Currently I use Event Grid subscription from Storage to ingest into Kusto. I can manually use Python SDK and orchestrate the ingestion and get the filename added as a column. However, if I want to use the native ingestion capabilities of the Event Hub, I cannot add the filename as metadata to the ingestion. It currently only supports filesize but not filename. By allowing filename, it allows us to tie the data ingested to file that contains the issue. I would rather let the Event Hub/Grid orchestrate the ingestion of thousands/millions of files without having to rely on a script…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Cancelling in-flight queued ingestions

    There can be cases in which i would like to discard messages posted for ingestion, after figuring out that something went wrong, or is no longer relevant.

    this is especially relevant when the size of the queue is really large.

    at the moment, the "best solution" is to drop or rename the target table, and patiently wait for all ingestions to fail (which could take a long while)

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. React

    Create a React module which would directly embed this Explorer into the custom web application

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Provide support to render timepivot / timeline view

    The Timepivot visualization is one of the best visualizations we have come across when analyzing data across hundreds of different tables. It tells the story of “what happened” in a beautiful manner.

    It helps us generates multiple different views of the “same data” by allowing us to pick multiple levels of slicing options views (in this case I have 10 slice options, so for two levels I can get about 90 different views). In addition, clicking on the cells shows the actual activity in detail.

    Given that Kusto is time-optimized and practically every log has timestamps associated with them, we…

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Azure Data Explorer setting to display datetime values in local timezone

    Can Azure Data Explorer support displaying datetime values in local timezone with a setting?
    I'm a bit tired of extending all queries by doing time arithmetic myself (e.g. | extend localT = timestamp - 8h) which is not daylight saving proof as well :)

    23 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Pump data to Kusto

    Create log exporter for Azure Monitor Application Insights SDK to enable double pumping of logs to ADX and Application Insights

    5 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Automatically merge small parquet files (with same table) that are put in ingestion queue

    We have a lot of parallel processes that each output small parquet files. The tables are the same for the different processes. We want to batch ingest all the data in these files, but have found that enqueueing them all (using KustoIngestClient.ingestfromblob) takes a long time (up to 30k blobs). Note that this is not the ingestion time, but just the time it takes to queue the files for ingestion. It would be convenient to have a function that could take a list of blob files that all contain data destined for the same table, merge them into…

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Add ADX as Trusted Microsoft service on firewall

    Please consider to add ADX as a trusted Microsoft service on Azure Event Hubs' (or any other data source service's) firewall or publish IP address range of ingest service.
    For now, we have to deploy the cluster with VNet in order to restrict access of Event Hubs but it seems unnecessary since the traffic between Azure service is completed inside Azure network.
    This option makes it simple/easier for the users to create secure environment.

    14 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Queued Export operation

    We can use QueuedIngestClient or ingest from Blobstore to let Kusto ingest data based on cluster's capacity.

    The number of export slots slots is also limited to number of core, so we should have the same capability to queue export operation.

    1 vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. ingest

    Add a way to schedule ingestion (queued ingestion) with a kusto command on external table (i.e. ingesting a huge external table reliably).

    2 votes
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)

    We’ll send you updates on this idea

    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3 4 5 6
  • Don't see your idea?

Azure Data Explorer

Categories

Feedback and Knowledge Base