Batch

How can we improve Azure Batch? Do you have an idea, suggestion or feedback based on your experience? We would love to hear it!

  1. Make it possible to set memory usage limitation for a batch task

    it would be nice if we can set the maximum memory usage for a specific task.
    This could help us protecting concurrent tasks from failing when one task is consuming a lot of memory.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  2. Add Job based metrics in autoscale formula to facilitate the scaling of Dedicated nodes based on the number of JobManagerTasks

    Now autoscaling only supports task-based metrics: $ActiveTasks, $RunningTasks, $PendingTasks, etc. It would be nice to have similar metrics for jobs to be able to scale the TargetDedicatedNodes count to the number of JobManagerTasks with AllowLowPriorityNode = false.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Make JobPreparationTask and JobReleaseTasks independent from each other

    I only need the JobReleaseTaks, but I MUST create also a JobPreparationTask, otherwise I get the following error:

    "A job release task cannot be specified without job preparation task being specified"

    For me this is quiet odd. So I added a JobPreparationTasks which just "echos" something.
    I am using the C# SDK (Azure.Batch, 8.0.0)

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Task should be started immediately whenever its possible

    Starting a task is delayed by 10 seconds even the pool is idle and there is no task run on the pool. It should be started immediately. The clock skew between server is not an excuse to not able to start the task asap.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Add support for ceil, round, floor functions in auto-scaling formula

    Auto-scaling formulas should add support for these rounding functions:

    round()
    ceil()
    floor()

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  6. Custom job/node statistic/metrics

    There's a vote for per-node resource statistics for general metrics that are always relevant - CPU, RAM usage, etc.

    The ability to push custom statistics, probably through some kind of node SDK, would be great too. For example, if you're doing database work, you might want the rows p/s or if it was render work, pixels p/s or ETA for that section.

    Ideally, these metrics would also be available for the entire job. In the first example above, the entire job would show the sum of all rows p/s to get the total rows p/s. For ETA, it would be…

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. Provide a way to increase default quotas at the subscription level

    It would be nice if we could increase the Max cores, max jobs etc at a subscription level so that each new batch account we create will have those settings.

    I understand you need to know how many cores we can use at any given time but you also have another quota for maximum number of batch accounts we can have.

    I think you can determine our core potential just by multiplying the default limit by the limit on how many batch accounts we can have.

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  8. Improved error / fail sooner when pool removal is gated with a delete lock

    We recently had a case where a pool removal was 'stuck' and it appeared that the completion of the command was gated by the timeout value (which we inadvertently made very long).

    It turned out that the removal of the pool needed to delete a configuration on a subnet which was part of a VNet/RG that had a deletelock applied.

    It seems unreasonable to hang a process for a configuration change and would propose that in this case the resize should've failed once it determined it was blocked due to a delete lock on a 'sub resource'. Even better if…

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Automatically docker pull on node reboot

    It would be EXTREMELY helpful if Azure Batch automatically did a docker pull on reboot or at least added the credentials to docker so a pull works from a start task without needing to login to the private docker registry with credentials. The password for the registry seems impossible to find via rest or environment variables.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Set Hybrid Benefit license cap for auto-scaling on pools

    When using Hybrid Benefit for Windows Server on a batch pool, it would be great if it was possible to register a max number of licenses to use under Hybrid Benefit, so that the auto-scaling feature will use only nodes under Hybrid Benefit up to the registered number, and beyond that use nodes at full cost.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Allow using custom data to calculate scale formula

    I want to be able to grow/shrink number of nodes in anticipation to external data - e.g. I know that based on an event I'll need a few nodes up and running an hour from now. I wish the formula could allow making a call to Azure function/REST/take a value from a Storage blob etc.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Allow more than 20 applications in a Batch account

    Currently, a Batch account can have no more than 20 applications. This limit is too constraining and should preferrably be removed or at least raised considerably.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. BatchJob Tasks must pick the latest build automatically from ApplicationPackages

    Hi,
    On test environments, we can not change the DefaultVersion for every new build or we can not change the code. It would be very useful if the JobManagerTask and Tasks can take the latest build uploaded into Application Packages AUTOMATICALLY.

    And also, Restarting the Pool everytime we add a Package to the Pool is very tiresome.

    Regards,
    Srinivas

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Add larger VMs

    Add support for high priority VMs in the M series. We require over 1TB of ram for a regular task at times, you currently have no VM (that cam't be preempted) that had this capability.

    Please add one, RAM requirements are only going up as the data grows and our problem is not embarrassingly parallel, so adding more machines adds both a time and financial cost (inter-node communication) that we do not want to incur.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Add features to the Azure BAtch mobile app

    The mobile app is currently not very helpful. Without the ability to remove jobs/tasks/pools, it's just a way to look at things without being able to impact any change.

    Please add the ability to look at multiple subscriptions at one, and the ability to manage Azure Batch resources.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Let's use Open Source as the runtime used in Node of Azure Batch.

    Although Azure Batch is excellent, it is difficult to understand the detailed operation of Rumtime.
    Please make it OSS of Azure Batch.

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Raise the number of concurrent tasks per node core

    The idea I have in mind is to use Azure Batch to export a >100 TB Azure Table, at a much faster rate than Azure Data Factory can manage.

    Azure Tables is quoted to cope with 10,000s of simultaneous requests. To maximize my dev velocity, I would create millions of lightweight tasks and let Azure Batch runtime handle all the scheduling details.

    But there is a limit of 4 concurrent tasks per core so this would make for extremely underutilized nodes and long runtime.

    https://docs.microsoft.com/en-us/azure/batch/batch-quota-limit

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. Misleading error message when using a custom image

    I tried to create a pool for Azure Batch (from python) with a custom image. I got this error message:

    "AuthenticationErrorDetail: The specified type of authentication SharedKey is not allowed when external resources of type Compute are linked."

    The real problem seems to be using a custom image, which makes SharedKey auth not usable. The "external resources of type Compute" didn't have any meaning for me; something like "using a custom image with Azure Batch requires AAD/ServicePrincipal auth" would have been more helpful.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. Implement tags into Batch Pools, Jobs and Tasks

    It would be great to be able to create tags into batch resources like Pools, Jobs and Tasks, as we can do it with Batch Accounts.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Support Resource Files In Azure Data Lake

    When setting up an Azure Batch Service for the first time I tried to use files in an Azure Data Lake as a resource files in a task. I received the following errors and when following up with Azure Support I was trying to do something that isn't currently supported.

    I think supporting adl files as an option would be a valuable addition to the service

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Batch

Categories

Feedback and Knowledge Base