How can we improve HDInsight?

Start/Stop cluster HDInsight

The possibility to start and stop a cluster. Now is only available delete the cluster and I do not want any charge unnecessarily if I don't use the cluster for several days.

686 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    Yolanda Yolanda shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    [Update] Thanks for your continued feedback on this capability! Rest assured that we are tracking this request closely along with several other platform capabilities our customers have requested. In the meanwhile, you can leverage cluster scaling capability to adjust HDInsight cluster size according to your varying compute needs. Azure Data Factory is another option you can explore for scheduling jobs with automatic creation and deletion of clusters: https://azure.microsoft.com/en-us/documentation/articles/data-factory-data-transformation-activities/

    Adnan Ijaz
    Program Manager
    Microsoft Azure HDInsight

    22 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • RZRZ commented  ·   ·  Flag as inappropriate

        As of now I am deleting the cluster and moving it to some other option...
        Anyway Hope to see this soon !

      • Asa BaileyAsa Bailey commented  ·   ·  Flag as inappropriate

        Our cluster gets used about a week to two weeks a month and runs about $10k a month if it is up all month. We really would like to be able to stop the cluster instead of building it from scratch everytime.

      • James HoferJames Hofer commented  ·   ·  Flag as inappropriate

        Considering it has been over a year since Microsoft has commented on this issue, should we take that to mean that you have no interest in customers that refuse to keep their hadoop clusters up 24/7? Or is this significantly harder to accomplish than it appears on the surface?

      • Raphael NashRaphael Nash commented  ·   ·  Flag as inappropriate

        Just as and FYI ... AWS Elastic Map Reduce allows cluster to be started and stop. Also, Amazon has setting that as soon as an MR job is completed the cluster will stop.

      • Anonymous commented  ·   ·  Flag as inappropriate

        yeah that's pretty pathetic, spending half the time creating the cluster then have to delete it everytime.

      • Richard WeeksRichard Weeks commented  ·   ·  Flag as inappropriate

        Wow, my class was given an educational subscription so we could learn Azure. Here's what I've learned: Azure (or at least HDInsight) is not the platform I want to use for Hadoop. I only have 2-3 hour blocks in my part-time student, full-time job schedule to work on my homework. Now I have to take 30-45 minutes of each block to create a Hadoop cluster -- not to mention importing the data? This is going to burn through the credit they gave us in the subscription faster than I could have imagined.

        Microsoft, how can you leave out such a critical feature from your Hadoop platform?

      • Lukasz TracewskiLukasz Tracewski commented  ·   ·  Flag as inappropriate

        Can't believe that this "feature" is still not there. Have a look at the pretty "stop" functionality AWS has, surely you can make similar. Unless that's how you want to differentiate from the competitor...

      • Brandon BassBrandon Bass commented  ·   ·  Flag as inappropriate

        This is a necessary feature for my team, we'd love to use the platform but starting and deleting a cluster is not a way forward.

      • DevonDevon commented  ·   ·  Flag as inappropriate

        April 2016. New interface redesign, but still not this feature? Please, this is necessary. Azure is otherwise the best platform right now for HDP.

      • PaulPaul commented  ·   ·  Flag as inappropriate

        Wow. Hard to believe. This issue alone is enough to make me walk away from Azure.

      • BenBen commented  ·   ·  Flag as inappropriate

        We too stayed away from HDInsight due to not being able to start and stop a cluster, versus having to delete the whole thing and recreate it.

      ← Previous 1

      Feedback and Knowledge Base