How can we improve HDInsight?

Start/Stop cluster HDInsight

The possibility to start and stop a cluster. Now is only available delete the cluster and I do not want any charge unnecessarily if I don't use the cluster for several days.

1,132 votes
Sign in
Sign in with: oidc
Signed in as (Sign out)
You have left! (?) (thinking…)
Yolanda shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

[Update] Thanks for your continued feedback on this capability! Rest assured that we are tracking this request closely along with several other platform capabilities our customers have requested. In the meanwhile, you can leverage cluster scaling capability to adjust HDInsight cluster size according to your varying compute needs. Azure Data Factory is another option you can explore for scheduling jobs with automatic creation and deletion of clusters:

Adnan Ijaz
Program Manager
Microsoft Azure HDInsight


Sign in
Sign in with: oidc
Signed in as (Sign out)
  • Paul David Lassiter commented  ·   ·  Flag as inappropriate

    Qubole, a Microsoft ISV Gold Cloud Platform partner provides a serverless "multi-engine" solution for Spark, Hive, Hadoop, with Presto and Airflow. It has workload aware autoscaling so clusters automatically start and stop and scale up and down as required to support actual demand at any given point of time. Thus, you only use the compute you need when it is absolutely required to execute jobs. Having a serverless Azure solution reduces data admin support time, job failures, and frees data admin's up to support new business initiatives and ensures business SLAs are met. Qubole has free test drive at:
    and you can learn more about Qubole on Azure at: Qubole also offers free two week pilot in your Azure subscription. Contact if interested. Qubole Presto has a PowerBI connector that allows direct query of the Azure Data Lake making data lake analytics simple and accessible for business analysts.

  • Anonymous commented  ·   ·  Flag as inappropriate

    It would also be nice if I could set the cluster to pause/delete after a certain amount of inactivity. Databricks does this and it prevents you from forgetting to turn off a developer cluster when not in use.

  • Anonymous commented  ·   ·  Flag as inappropriate

    Hi Guys when I choose stop all from Ambari, the credit stop decreasing. maybe you should try that.

  • Ron DeFreitas commented  ·   ·  Flag as inappropriate

    Do you guys even read this forum? Nearly three years still with no response other than "we're tracking this".

  • BA commented  ·   ·  Flag as inappropriate

    Hi, as I can see "STOP" or "PAUSE" option for HDInsight Spark cluster has not yet been implemented.
    I know the solution for create and delete an hdinsight cluster, but I would like to ask information about another possibility.

    From the hdinsight cluster dashboard (Ambari home), in the left panel there is the button "Actions" and then "Stop All", in this way all the services are stopped.
    I wanted to ask, is this a solution to pause the cluster when I do not use it and then do not pay excessive additional costs for the service?
    Has anyone tried this solution and knows if it works or not?

  • Anonymous commented  ·   ·  Flag as inappropriate

    This is an essential feature to sell to stake holders and add flexibility to a cloud solution

  • Anonymous commented  ·   ·  Flag as inappropriate

    This is an essential feature. The value on cloud is basically you are meant to reduce costs and this is a key feature missing.
    I see people is asking for this since 2 years ago. I guess this is not simple to solve for MSFT. But definitely is a key point. Hope you are working on it.


  • Ron DeFreitas commented  ·   ·  Flag as inappropriate

    You continue to get votes for this feature with no response in over 2 years... this is creating a hardship for us, as well, and it does not paint Azure in a positive light when I'm trying to sell stakeholders on HDInsight as our technology target.

    Any updates?

  • Anonymous commented  ·   ·  Flag as inappropriate


    I also vote for "STOP" or "PAUSE" option for HDInsight cluster. It is currently available for VMs.

    I would also use this to stop a spark cluster.

  • Anonymous commented  ·   ·  Flag as inappropriate

    I also vote for "STOP" or "PAUSE" option for HDInsight cluster. It is currently available for VMs.
    Is there a technical reason why it can not be made available on spark cluster?

  • Anonymous commented  ·   ·  Flag as inappropriate

    This is a serious pain-in-the-hole and definitely a dealbreaker coming from AWS. Would be a shame to leave Azure.

    Please let us know when this feature will be implemented.

  • myusrn commented  ·   ·  Flag as inappropriate

    Btw the ibm data science experience [dsx] provides free spark cluster environment to facilitate development prototyping w/o you having to run up a bunch of charges or burn through ones monthly azure $200 developer allowance. So i'd highly recommend you folks at least provide ability to stop/restart a minimally configured hdinsight spark cluster or even better provide some limited performance capability version that a dev can leave running all the time to access for prototyping work. Having to delete and recreate your setup in between uses is not practical.

  • Ankit Sharma commented  ·   ·  Flag as inappropriate

    As a student, we have create a cluster and delete it everyday which is creating a burden for us. We are studying for the Certification named "Implementing Predictive Analytics with Spark in Azure HDInsight" as part of Data Science curriculum.
    Please add the start/stop functionality so that our efforts can be saved. If its not possible then at least share the reason with us.

    Ankit Sharma

  • Matthew Townsend commented  ·   ·  Flag as inappropriate

    We are evaluating between AWS, Azure, and Bluemix for Big Data platforms and will probably pick 2. This is definitely not a positive for Azure.

  • Luca Zavarella commented  ·   ·  Flag as inappropriate

    Any news on this request? It's simply a shame you're forced to pay since there isn't a way to turn off the service without delete it.
    Thank you.

← Previous 1 3

Feedback and Knowledge Base