HDInsight

Welcome! You can use this site to tell the Microsoft HDInsight team what features you would like to see.

Remember that this site is for feature suggestions and ideas…

If you have technical questions, please visit our forums.
If you are looking for tutorials and documentation, please visit our getting started page.

  1. Support changing timezone settings on cluster nodes

    The customer is planning to monitor the logs on each nodes in case of failure and would like to change the time zone settings on host operating system of each nodes.
    This is not supported on HDI cluster now.
    Appreciate if you could consider supporting this.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  2. Enable tagging network interfaces at the HDI ARM template level

    Having multiple clusters in the same Resource Group it is impossible to easily determine which network interfaces belong to which cluster. It could be solved by assigning tags to network interfaces.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  3. Ability to resize HDInsight nodes without deleting cluster

    Would like the ability to resize the head, worker, zookeeper, and edge nodes of an HDInsight cluster without having to delete and recreate the cluster.

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  4. How to create quickstart cluster using AZ commands

    Please update the article to include the AZ command necessary to create the cluster rather than just using the UI.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  5. Change VM size of Worker, Head and Zookeeper

    Change VM size of Worker, Head and Zookeeper from Azure Portal. Today this option are locked, if I need change the vm size, I need to deploy a new cluster.

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  6. Vastly improve debugging return messages when creating HDInsight Kafka Clusters. The current "Unexpected error" is really hard to debug.

    Over the last 2 months, I've created multiple HDInsight based Kafka clusters. The ability to debug the creation process - On a scale of 1-10, I would give HDInsight a solid 0.

    The most seen error messages are:
    - Unexpected error
    - Conflict
    - Error

    When looking into the details, You find the detailed JSON response body. With the awesome added event timestamps. Sorry to say - but that does not help me figure out the cause.

    1 vote
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  7. Need premium Kafka Clusters (S series) in Japan-west region.

    For better Kafka throughput, in Japan West region, we need premium clusters but it is not offered in japan west region. We have checked, all the other regions offer Premium clusters for worker nodes for Kafka. To improve performance it is necessary and hope Premium VMs will be made available in Japan West region asap.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  8. Support Selecting a Certain Node When Scaling In

    The existing scaling out/in feature in HDInsight has a bad implication when it comes to scaling in, which is the inevitable failure of any pending or running jobs. It would be nice to have the ability to select certain nodes when scaling in, in order to safely shrink the cluster without loosing active jobs.

    37 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    3 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  9. Why there is no option to restart HDInsight Cluster?

    Why there is no option to restart HDInsight Cluster?

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    2 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  10. Option to create On demand cluster with Data Lake store as storage

    Hi Team,
    I could not find the option of selecting Azure data lake store as the storage medium while creating the on demand HD Insight cluster. I browsed the internet and learnt that On demand cluster + ADLS option is currently not present.
    Would be helpful if that is made available

    Thanks,
    Surya

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  11. Need HDInsight attach/detach edge node capability

    Need to be able to attach/detach edge nodes from an HDInsight cluster. This is not currently supported.

    201 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    19 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  12. Add Cluster Performance alerts

    We often struggle with lack of alerts when CPU and Memory reaches certain threshold. When this happens our jobs suffers and process hangs etc. In view of that we would like to have alters that are configurable and set thresholds on key resources such as CPU/Memory/Disk etc. System should able to generate emails to the group once resources reach certain threshold. I am sure every one will love this feature.

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  13. Provide a Unified Cluster Experience for HDInsight

    Instead of HDInsight seperating the core functionality into disparate packages, provide the ability to run multiple roles on the same cluster.

    This would enable multi-step processing use cases to be performed using a single cluster (instead of multiple).

    For example: Kafka Stream -> Spark -> HBase

    In the current topology, this scenario requires 3 clusters which is expensive and complicated.

    4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  14. Show only supported VM sizes when we create a HDInsight cluster on Azure portal

    When we create a HDInsight cluster on Azure portal, we can choose any VM sizes even if it isn't supported at the specified region. For example, Japan West doesn't support v2 size but we can choose it for worker nodes. Then, the deployment failed with the error "DeploymentDocument 'CsmDocuemtn_2_0' failed the validation. Error: 'VM size xxxxx_V2' provided in the CSM document is invalid or not supported for role 'workdernode''". Moreover, we can see HDInsight resource on Azure portal although the deployment failed. So, users have to delete it manually.

    It would be better if unsuppoted VM sizes wouldn't be shown…

    36 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  15. Cluster provisioning takes too long

    It takes at least 20 minutes to provision an HDInsight cluster with Standard A3 VMs. Repeated feedback from multiple customers, that it takes longer to provision it on Azure than on AWS or Google.

    9 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  16. Please create all resources belonging to an HDInsight cluster in the same Resource Group.

    Please create all resources belonging to an HDInsight cluster in one Resource Group.
    When you create an HDInsight cluster into a Vnet, all network resources such as NIC's, Load Balancers and Public IP addresses are created within the resource group that contains the Vnet. Only the HDInsight object itself is created in the original resource group. This is fine for testing but in multi-subscription environments with shared Vnets across resource groups, this is not ideal.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  17. Suggestion regarding the HDInsight

    While creating a HDInsight cluster, if we select the container, which is being used by other cluster it should alert the user then and there, it however raises a flag at the summery page in the end.

    It will be lot simpler for the users if they have a drop-down list of available containers in the storage account which are not in use.

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  18. Ability to add Azure Data Lake store after the HDI cluster has been created

    Currently the only way to add ADLS as an additional storage is during the cluster creation time. However, if there is way to add ADLS after the HDI cluster is created, customer can files on ADLS for further analysis

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  19. HDInsight AutoScale

    Please provide auto scale option in HDInsight for scaling down and scale up of Cluster based on usage/query running.

    102 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    5 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
  20. Capture Log Information for all queries

    HDInsight portal should provide a way to capture all log associated with query like Tez log, hive Log and Templeton log

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Microsoft
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Platform  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1
  • Don't see your idea?

HDInsight

Categories

Feedback and Knowledge Base