Batch

How can we improve Azure Batch? Do you have an idea, suggestion or feedback based on your experience? We would love to hear it!

  1. Azure batch node disk encryption

    It would be helpful for azure batch to have encryption at rest for a layer of protection for task temporary files.

    The discussion here illustrates it's overly tricky:

    https://social.msdn.microsoft.com/Forums/azure/en-US/71bc46a7-593a-447c-912b-b95754198c6b/does-batch-use-encrypted-local-storage?forum=azurebatch

    14 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Add larger VMs

    Add support for high priority VMs in the M series. We require over 1TB of ram for a regular task at times, you currently have no VM (that cam't be preempted) that had this capability.

    Please add one, RAM requirements are only going up as the data grows and our problem is not embarrassingly parallel, so adding more machines adds both a time and financial cost (inter-node communication) that we do not want to incur.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Raise the number of concurrent tasks per node core

    The idea I have in mind is to use Azure Batch to export a >100 TB Azure Table, at a much faster rate than Azure Data Factory can manage.

    Azure Tables is quoted to cope with 10,000s of simultaneous requests. To maximize my dev velocity, I would create millions of lightweight tasks and let Azure Batch runtime handle all the scheduling details.

    But there is a limit of 4 concurrent tasks per core so this would make for extremely underutilized nodes and long runtime.

    https://docs.microsoft.com/en-us/azure/batch/batch-quota-limit

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Better error messages if pool is unable to add nodes

    If the managed resources for a vm image are missing or the node type doesnt support a requested feature (e.g. managed disk on A series vm) then the pool keeps trying to add nodes eventually hitting resize timeout, after timeout it starts again. Its not easy to debug those situations and a better error report would be very useful.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Batch

Categories

Feedback and Knowledge Base