How can we improve Microsoft Azure Stream Analytics?

Ability to output data to dynamically generated tables.

We want incomming data for multiple of our customers input through one event hub and then output to a table per customer. The tables don't need to be created by stream analytics, we can do that from code when the customer creates an account with us. I just need to be able to direct the output to the correct table from my Stream Analytics query.

40 votes
Vote
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
You have left! (?) (thinking…)
Vossekop shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

5 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...
  • Fabio Maulo commented  ·   ·  Flag as inappropriate

    Using stream analytics for second project and again my idea is the same... I need an 'on the fly' table name.
    I have some measuring station each with some sensor. Measures are sent to IoTHub.
    The stream analytic works, in my case, as a reducer calculating the avg of each sensor each 5'; I need to store the result to {station}{sensor} table (partionKey=Day, rowKey=minuteOfTheDay).
    To workaround the problem I would use the QueueStorage (don't worry I take care about idempotent) but again the output is not available.
    The final workaround is the ServiceBus/EventHub.

  • Martin H. Normark commented  ·   ·  Flag as inappropriate

    Man this is really something I need. Being able to use the table name to group data will add an extra fast filter to data when querying.

    Storing data per customer into their own separate tables is vital.

    Any news on this?

  • Vossekop commented  ·   ·  Flag as inappropriate

    The most simple solution I would see that would work for us, is where we could specify the table name for the output from a column in the input event, or for instance as a constructed string. So for instance this kind of syntax:

    CREATE TABLE input_source (
    OrganizationId NVARCHAR(MAX),
    DeviceId BIGINT,
    Moment DATETIME,
    Value FLOAT,
    )

    SELECT DeviceId as PartitionKey, Moment as RowKey, Value
    FROM input_source
    INTO 'Organization' + OrganizationId

  • AdminAzure Stream Analytics Team on UserVoice (Product Manager, Microsoft Azure) commented  ·   ·  Flag as inappropriate

    Could you elaborate on how you would like to see this enabled? Would generating DDL for the CREATE TABLE meet your requirements? From there, you could create the required tables and reference them in your query, which supports multiple outputs. Please let me know if I'm understanding your scenario correctly.

  • Axel Evers commented  ·   ·  Flag as inappropriate

    Another Use Case is to be able to mass delete old data. If i am able to create one table per day (or event better per day & customer) i can just drop old tables without having to mass delete records within the table. Therefore providing a table name as it is already possible for partition & row key would be really helpful. Auto create of those tables would also be great!

Feedback and Knowledge Base