Batch

How can we improve Azure Batch? Do you have an idea, suggestion or feedback based on your experience? We would love to hear it!

  1. Azure batch node disk encryption

    It would be helpful for azure batch to have encryption at rest for a layer of protection for task temporary files.

    The discussion here illustrates it's overly tricky:

    https://social.msdn.microsoft.com/Forums/azure/en-US/71bc46a7-593a-447c-912b-b95754198c6b/does-batch-use-encrypted-local-storage?forum=azurebatch

    11 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Static public IP on Batch Pool

    Make it possible to assign a static public virtual IP (VIP, both basic and standard) to a pool in Batch, just like it is possible to assign a VIP to a cloud service deployment.
    In our use case we need all nodes in the pool to use the same ip on outgoing traffic.

    8 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Raise the number of concurrent tasks per node core

    The idea I have in mind is to use Azure Batch to export a >100 TB Azure Table, at a much faster rate than Azure Data Factory can manage.

    Azure Tables is quoted to cope with 10,000s of simultaneous requests. To maximize my dev velocity, I would create millions of lightweight tasks and let Azure Batch runtime handle all the scheduling details.

    But there is a limit of 4 concurrent tasks per core so this would make for extremely underutilized nodes and long runtime.

    https://docs.microsoft.com/en-us/azure/batch/batch-quota-limit

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  4. Add larger VMs

    Add support for high priority VMs in the M series. We require over 1TB of ram for a regular task at times, you currently have no VM (that cam't be preempted) that had this capability.

    Please add one, RAM requirements are only going up as the data grows and our problem is not embarrassingly parallel, so adding more machines adds both a time and financial cost (inter-node communication) that we do not want to incur.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Expose compute resource provider level logs via batch diagnostic

    We currently ran into the situation where our batch pool did not start due to the error:

    Offer with PublisherId: microsoft-azure-batch, OfferId: ubuntu-server-container cannot be purchased due to validation errors. See details for more information.[{\"Legal terms have not been accepted for this item on this subscription. To accept legal terms using PowerShell, please use Get-AzureRmMarketplaceTerms and Set-AzureRmMarketplaceTerms API(https://go.microsoft.com/fwlink/?linkid=862451) or deploy via the Azure portal to accept the terms.

    However, there is no way to see this error via Batch diagnostics. The only error we could see was:

    Desired number of dedicated VMs could not be allocated due to…

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Better error messages if pool is unable to add nodes

    If the managed resources for a vm image are missing or the node type doesnt support a requested feature (e.g. managed disk on A series vm) then the pool keeps trying to add nodes eventually hitting resize timeout, after timeout it starts again. Its not easy to debug those situations and a better error report would be very useful.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Batch

Categories

Feedback and Knowledge Base