Stream Analytics

Azure Stream Analytics is a fully managed real-time stream computation service hosted in Microsoft Azure, which provides highly resilient, low latency, and scalable complex event processing of streaming data. Azure Stream Analytics enables developers to easily combine streams of data with historic records or reference data to derive business insights easily and quickly.

  • Hot ideas
  • Top ideas
  • New ideas
  • My feedback
  1. Rename Inputs and Outputs

    Please can you let us rename the input and outputs of a stream. Thanks

    19 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. 19 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  5 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Ability to check for EXISTS when peforming a query over JSON with arrays

    My data has a data field that can have no value, a single value or an array of values. I need to be able to parse that field into single column with either a single row (for no value and single value) and into multiple rows (using, say GetElements for array) dynamically based on whether that data field is an array or not.
    Currently I get an error - "The expression outputStep experienced an error: Unable to cast type 'string' to 'Record'.. It will be disabled"

    18 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  4 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Add Cassandra to output

    Add Cassandra as an option to output.
    Cassandra (DataStaxEnterprise) offer a high performance queries on Azure

    18 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. limit output event data batch size

    Allow user to configure the number of output events that can go in a single event data. As of now, for EVent Hub as output, format of JSON output (new line or array) can be configured but there is no way to configure how many events can be batched. This would allow the user to control the throughput based on needs and the consumer of event hub can have consistent behavior on the max events (or one event) per message.

    17 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Support datetimeoffset datatype

    I'd like for Stream Analytics query language to support datetimeoffset datatype. This is a must in an IoT world.

    17 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Array or Record To String built in function

    The portal displays arrays and records as a json string but if you try to save the entire array as a json string to a field in an SQL table it is not possible. A built in function to convert and array or record data type to a json string would be useful.

    16 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Stream Analytics on Azure Stack

    Stream Analytics should be on Azure Stack so that we can implement the IoT Reference Architecture https://aka.ms/iotrefarchitecture on-premises. IoT Hub coming to Azure Stack was already announced, now the Stream Analytics is the missing piece of the puzzle.

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Optional Columns in Query

    Currently, when processing events from an Event Hub and using a 'CREATE TABLE' statement in the Stream Analytics Query, all specified fields must be present in the event. Otherwise, a 'FieldsMismatchError' is encountered.

    "Message:Missing fields specified in query or in create table.
    Fields expected: {list of fields in CREATE TABLE statement}
    Fields found: {list of fields in event}
    Missing fields: {list of missing fields} Type:FieldsMismatchError Correlation ID:"{GUID}"

    I would like to be able to use a 'CREATE TABLE' statement specifying optional fields, f.e. as following;

    CREATE TABLE
    table_name
    ( column_name <data_type> [ NULL | NOT NULL ] [ ,...n ]…

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Ensure reasonable error codes while deploying SA and violating a policy

    - While trying to deploy a Stream Analytics Job, the deployment is prohibited due to an existing policy.
    - This was done on purpose, because I am responsible for the according policies in this subscription.
    - Thus, the problem is NOT, that I can’t deploy the SA Job.
    - The problem is, that the error code during the deployment is totally useless because it does not state the name or the ID of the policy which has been violated. (Please see the attached image)
    - In this particular case, I know which policy was responsible. But in other cases me…

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Provide a better experience for updating queries without start/stop

    Currently, if I want to edit my query, I need to stop my stream, wait for it to go from stopping... to stopped, edit query, hit start, waiting for starting... to go to running.

    Instead, I would like to go to my running query, edit my query, like save, and have a prompt, would i like to restart my query NOW, (now and last stop time will be the same in this case), or in the past, and then you go and stop and start query without me having to wait.

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Usean Azure Service Bus Topic as Stream Analytics Input Source

    We have a number of Services which are Ingressed into Azure using Service Bus Topics, the use of topics is due to a the data stream being used in a number of different parallel processing paths, It would be great to be able to use use a topic as an input to a stream analytics job, (currently to achieve the same effect we have to use an API APP web role to receive the data from the topic and then immediately send the message onwards to a separate event hub which is then setup to be an input to a…

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  13. Provide mechanism to cache output state or values in a fast storage like redis.

    this would help in cases for comparing preceding values with out self joins.

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. More secure connection options between Stream Analytics and Azure SQL Database

    Currently, Stream Analytics only supports Username and Password. Several customers need to support more secure connections to realize higher security model such as supporting MFA (certificate) model and Key Vault.

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Allow TIMESTAMP BY with CROSS APPLY

    I have someincoming events where each event contains an array with the actual data events.

    So i do APPLY. But i want to define the TIMESTAMP BY for the resulting rows, as the timestamp property is nested inside this array.

    This would result in the following query:

    SELECT *
    FROM
    [input] AS i
    CROSS APPLY
    GetArrayElements(i.records) AS c TIMESTAMP BY c.ArrayValue.time

    but i cannot do this (syntax error)

    Alternative idea was to wrap the above query in a view, then define TIMESTAMP BY on the usage of the view:

    WITH view1
    AS (
    SELECT
    c.ArrayValue.*
    FROM
    [myi-vm-logs] As i
    CROSS…

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Support MySQL databases as an output option.

    Azure supports MySQL databases (via ClearDB). It'd be nice if, along with SQL Server, MySQL databases were available as an output option. (Although, please do not couple it directly to ClearDB's offering -- there are many people running MySQL clusters on other technology in Azure.)

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Add Output to Azure Storage Queue

    Allow output to be sent to an azure storage queue. The use case would be to trigger an azure function/signalr from the azure storage queue when a certain condition is found in the stream. Currently we are writing data to azure table storage and if a specific condition is found, we would like to trigger another action or further processing of the row. Right now we have to write to a service bus topic/queue, develop a listener/trigger function application, and then process the record. Enabling storage queue allows us to use only 1 storage account to hold the raw data,…

    14 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  18. Allow JSON Object as serialised string in Azure Table output

    I want to store a JSON object as serialised string in an Azure Table column. When outputting a JSON object it gives "Array" in de Azure Table. How can I output the JSON object as string?

    13 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  19. Output Error Rows to a Queue or Blob

    Create a stream for unsuccessful rows that can be sent to an output preferably a queue or blob. Currently I don't know which rows errored

    13 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. mongo

    Add an output to a MongoDb collection. So Stream Analytics can store information into a NoSql Document based DB, such as MongoDB.

    13 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Stream Analytics

Categories

Feedback and Knowledge Base