How can we improve Azure Diagnostics and Monitoring?

Provide functionality to solve character limit api requests.

Hi,
We are using the new Azure SDK 2.5 Diagnostics for our Cloud Service.
Since our Performance Counters are dynamic and rely on process ID and process Identifier (the thing with the #) we cannot set the configuration in the wadcfg files but have to create the configuration during runtime and upload it via the REST API.

E.g. ChangeDeploymentConfiguration:
https://management.core.windows.net/{0}/services/hostedservices/{1}/deploymentslots/{2}/?comp=config "POST"

Today we received an error that the maximum length for the public configuration cannot exceed 20480 characters.

That is not acceptable.
First of all we have so many performance counters that we can easily reach this limit. And secondly we do not have any alternative to extend the public configuration e.g with a PUT request (as someone would expect from a RESTful API).

Luckily i was able to work around this by removing some counters from our configuration which are not essential, but in the long run this in just plain unacceptable, since we only postponed the inevetible.

Request:
Add a PUT functionality to extend the public configuration of the cloud service or remove the character limit in the POST method. (at least increase it by the factor 10).

As an example i have added a list of performancecounters we are currently using when running a cloud service with 4 web, 4 worker and 4 cache roles..
those are 6300 characters.
Scaling up to 10 roles each and adding a few counters will already reach the limit.

35 votes
Vote
Sign in
(thinking…)
Sign in with: oidc
Signed in as (Sign out)
You have left! (?) (thinking…)
David Terbeek shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

3 comments

Sign in
(thinking…)
Sign in with: oidc
Signed in as (Sign out)
Submitting...
  • David Terbeek commented  ·   ·  Flag as inappropriate

    We had a nice idea to include wildcards , thus reducing the number of counterspecifiers required by the configuration. e:.g
    <PerformanceCounterConfiguration counterSpecifier="\Process(w3wp_*)% Processor Time" sampleRate="PT10S" />
    But the api denies the request with the error that the config is not valid against the required schema.

    So this would also be a possiblity to solve the problem. Let your API accept wildcard characters

  • David Terbeek commented  ·   ·  Flag as inappropriate

    And the other half of the counters. because apparently there is also a character limit on this comment text box..

    <CounterSpecifier>\tvcustom(_Total)\Active Calls</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Total Calls</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Process Time per Request</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Requests Per Second</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Active SignOut Processes</CounterSpecifier>
    <CounterSpecifier>\tvcustom(*)\Commands received per second</CounterSpecifier>
    <CounterSpecifier>\tvcustom(*)\Commands sent per second</CounterSpecifier>
    <CounterSpecifier>\Network Interface(*)\Bytes Received/sec</CounterSpecifier>
    <CounterSpecifier>\Network Interface(*)\Bytes Sent/sec</CounterSpecifier>
    <CounterSpecifier>\.NET CLR Exceptions(WaWorkerHost)\# of Exceps Thrown</CounterSpecifier>
    <CounterSpecifier>\.NET CLR Exceptions(WaWorkerHost)\# of Exceps Thrown / sec</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_2376)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_2376)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_2376)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_804)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_804)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_804)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1996)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1996)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1996)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1656)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1656)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1656)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Windows Azure Caching:Host\Total Data Size Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1956)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1956)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1956)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1984)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1984)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1984)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1912)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1912)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_1912)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_2652)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_2652)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(WaWorkerHost_2652)\Private Bytes</CounterSpecifier>

  • David Terbeek commented  ·   ·  Flag as inappropriate

    Since i cannot see the file i uploaded, here is the list of counters for everyone to inspect:

    <CounterSpecifier>\Processor(_Total)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Retry Count - Active Logins Tracker</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Retry Count - Frontend Notification Queue</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Retry Count - Error Mail Queue</CounterSpecifier>
    <CounterSpecifier>\tvcustom(_Total)\Retry Count - Session Reporting Queue</CounterSpecifier>
    <CounterSpecifier>\tvcustom\ExceptionsPerSecondw3wp</CounterSpecifier>
    <CounterSpecifier>\tvcustom\ExceptionsPerSecondw3wp1</CounterSpecifier>
    <CounterSpecifier>\tvcustom\ExceptionsPerSecondw3wp2</CounterSpecifier>
    <CounterSpecifier>\tvcustom\ExceptionsTotalw3wp</CounterSpecifier>
    <CounterSpecifier>\tvcustom\ExceptionsTotalw3wp1</CounterSpecifier>
    <CounterSpecifier>\tvcustom\ExceptionsTotalw3wp2</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3624)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3624)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3624)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3664)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3664)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3664)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2336)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2336)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2336)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2804)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2804)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2804)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2204)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2204)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2204)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3992)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3992)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3992)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2652)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2652)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2652)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_4008)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_4008)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_4008)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2412)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2412)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2412)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2744)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2744)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_2744)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3352)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3352)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3352)\Private Bytes</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3392)\Thread Count</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3392)\% Processor Time</CounterSpecifier>
    <CounterSpecifier>\Process(w3wp_3392)\Private Bytes</CounterSpecifier>

Feedback and Knowledge Base