Batch

How can we improve Azure Batch? Do you have an idea, suggestion or feedback based on your experience? We would love to hear it!

  1. Raise the number of concurrent tasks per node core

    The idea I have in mind is to use Azure Batch to export a >100 TB Azure Table, at a much faster rate than Azure Data Factory can manage.

    Azure Tables is quoted to cope with 10,000s of simultaneous requests. To maximize my dev velocity, I would create millions of lightweight tasks and let Azure Batch runtime handle all the scheduling details.

    But there is a limit of 4 concurrent tasks per core so this would make for extremely underutilized nodes and long runtime.

    https://docs.microsoft.com/en-us/azure/batch/batch-quota-limit

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  2 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Need reimage support on VirtualMachineConfiguration-based pool

    Hi Batch Team, we would like to have reimage support on VirtualMachineConfiguration-based pools as well. We know that currently it is supported on CloudServiceConfiguration-based pools. This would greatly reduce the effort to maintain the nodes in the pool as we want to regularly reimage the nodes to keep them healthy without deleting the pool and recreating again. Thanks so much!

    45 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Azure batch node disk encryption

    It would be helpful for azure batch to have encryption at rest for a layer of protection for task temporary files.

    The discussion here illustrates it's overly tricky:

    https://social.msdn.microsoft.com/Forums/azure/en-US/71bc46a7-593a-447c-912b-b95754198c6b/does-batch-use-encrypted-local-storage?forum=azurebatch

    14 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. 22 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Implement local Batch emulation

    Please allow full emulation of Batch jobs to facilitate local development and testing prior to deployment.

    231 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    unplanned  ·  3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Add diagnostics support for Azure Batch

    Currently Azure Batch does not support diagnostics/telemetry/performance counters. It would be really useful to have something like a New relic or an App insights integration.

    56 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  7. Allow initial value for TargetDedicated when autoscale is enabled for a pool.

    Otherwise it takes at least 5 minutes to start the tasks!

    26 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Support on Premise Batch Nodes

    It would be great to have support for on-premise nodes to support hybrid workloads. This could help with connectivity issues for specific types of specialized data sources. Having a "Batch Agent" that could be installed on any server so it can register in a pool and be sent work would be really valuable.

    61 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Deploy certificate from Azure Key Vault to compute node

    Rather than having to associate a certificate with a pool, being able to associate a certificate in a Key Vault that's been created with the -EnabledForDeployment flag would help centralize management of certificates being deployed to both VM's and Batch nodes.

    32 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    unplanned  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. Jobs of equal priority should be executed in submission order

    If I have a fixed number of dedicated VMs, then it is important that queued jobs run in submission order. Sure I could use the priority to achieve this, but then I can't use priority for real business use case prioritisation. Sure I could reserve priority ranges, but this is getting complicated when it should just process them in submission order.

    25 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    under review  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. <Request>Please be sure to be able to delete batch trigger like other connectors

    Currently it is impossible to delete batch triggers by design and it is necessary to recreate Logic Apps once. Please be sure to be able to delete batch trigger like other connectors.

    <Japanese>
    現在、設計上、バッチトリガーを削除することはできません、一度Logic Appsの再作成が必要です。他のコネクターと同様にバッチトリガーも削除できますようにしてください

    14 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. Better error messages if pool is unable to add nodes

    If the managed resources for a vm image are missing or the node type doesnt support a requested feature (e.g. managed disk on A series vm) then the pool keeps trying to add nodes eventually hitting resize timeout, after timeout it starts again. Its not easy to debug those situations and a better error report would be very useful.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    started  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Let's use Open Source as the runtime used in Node of Azure Batch.

    Although Azure Batch is excellent, it is difficult to understand the detailed operation of Rumtime.
    Please make it OSS of Azure Batch.

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
2 Next →
  • Don't see your idea?

Batch

Categories

Feedback and Knowledge Base