How can we improve Azure Log Analytics ?

Support conversion and formatting functions in the search language

There should be option in the search language to convert metrics. For example If I want to convert Bytes to Gigabytes that should be possible in the search language. Other examples are in converting time to specific format (shorter time format, adding timezone and etc.)

15 votes
Vote
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
You have left! (?) (thinking…)
Stanislav Zhelyazkov shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

8 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...
  • Austin McCollum commented  ·   ·  Flag as inappropriate

    Any consideration of porting the conversion, formatting functions from LogParser 2.2? I use Log Parser Studio heavily, and it would be great to have certain functions like TO_LOWERCASE(), EXTRACT_PREFIX, EXTRACT_TOKEN.

    Some examples of data that is difficult to read without these formatting functions:
    IIS log "cs-uri-stem" field

    /OWA/14.2.328.9/ClientBin/OwaSl.xap
    /owa/14.2.328.9/scripts/premium/blank.htm
    /owa/14.2.328.9/scripts/premium/cdayvw.js
    /owa/14.2.328.9/scripts/premium/fedmtinv.js
    /owa/14.2.328.9/scripts/premium/fedtcali.js
    ...

    I really would love results that just tally results between the first two "/", for example in LogParser I can use the following formatting functions
    Select TOP 10 TO_LOWERCASE(EXTRACT_TOKEN( cs-uri-stem ,1, '/' )) AS Path

    for results like this:

    Path Total
    rpc 6021529
    ews 119456
    powershell 13795
    autodiscover 3247
    mapi 2304
    owa 1907
    ecp 891
    microsoft-server-activesync 834
    favicon.ico 11
    oab 6

    Then of course, like OMS, Log Parser Studio makes it easy to create a bar chart for visualization.

  • Stanislav Zhelyazkov commented  ·   ·  Flag as inappropriate

    At this point probably that kind of format changes/math for time are not needed for aggregated results. The count() by TImeGenerated function is limited to certain scenarios and those scenarios can live without such format changes. May be in the future when there are more advanced function that you can do with TimeGenerated it will be needed.

  • AdminOMS Log Analytics Team (Product Manager, Microsoft Azure) commented  ·   ·  Flag as inappropriate

    Thank you.

    Do you ALSO need to do this type of format changes/math against AGGREGATED results (from Measure) ?

    With datetime you have today some form of that with measure count() by TImeGenerated interval 1DAY/6HOURS/whatevevr interval - which allows to somehow 'bucketize' times without trimming them down as strings.

    But besides that, supporting the same with other data types might be very expensive initially and we were so far considering it more of a P2, where the P1 would be the support in 'SELECT' that I described in the previous comment.

    Let us know your point of view, it is always appreciated!

  • Stanislav Zhelyazkov commented  ·   ·  Flag as inappropriate

    The suggestion below seems nice and also flexible. May be the Time can follow the same approach like setting offset. Which will add/substract hours/minutes to the time and that way you can viewed as it is in another time zone. I am not sure about what math function can be used to make the time in different format like showing only hours and minutes and not seconds.

  • AdminOMS Log Analytics Team (Product Manager, Microsoft Azure) commented  ·   ·  Flag as inappropriate

    In the case of conversion/formatting function, we were thinking they would belong in the 'SELECT' command by introducing a 'AS' option to it (like you can assign a name to 'AggregatedResult' column with 'AS' in a MEASURE command). A bunch of these formatting capabilities are also present in Select-Object in powershell, so it could sound similar (as usual, cannot be identical).

    I.e. could be something similar to this -

    Type=Event | Select Computer as MachineName
    csUriStem="/foo/bar" | select TimeTaken/1000 as Seconds
    CounterName="Available MBytes" | Select Average/1024 as Giga

    would that work?

    Just gathering feedback on whether the current thinking makes sense.
    Thanks,

Feedback and Knowledge Base