Enable Polybase on SQL Managed Instance
Please enable Polybase on SQL Managed Instance. This will allow us to integrate data from our other IoT Analytic environments (Databrick, and Snowflake) with our corporate data warehouse (SQL MI). Polybase will improve data availability between systems by setting up external tables over Azure Storage Blobs that are shared between systems.
Brian Walker
shared this idea
Thank you for using Azure Feedback and sharing your thoughts and feedback on enabling Polybase functionalities in Azure SQL Managed Instance.
We started working on it, and happy to announce our first milestone:
Private preview of querying external data (Parquet files) in Azure Data Lake Service (ADLS) Gen2 from Managed Instance. You can learn more about the currently available functionalities from the onboarding guide:
If you are interested in trying it, or would like to share more detailed feedback on the Polybase support, contact us by following instructions from the onboarding guide.
Thanks,
Azure SQL Polybase team
16 comments
-
Srini Velamakanti
commented
When we can expect this feature to be available in the Azure SQL Database and SQL Managed Instance?
-
SimonS
commented
Funny you get a very old error:
Msg 46923, Level 16 State 1, Procedure ***, Line *** [Bach Start Line X] Polybase feature not installed. Please consult Books Online for more information
Books online! Seems this is more a miss on MI since documentation on this feature (in the context of MI differences) does not cover it:
https://docs.microsoft.com/en-us/azure/azure-sql/managed-instance/transact-sql-tsql-differences-sql-server#polybase -
Aleksandr Panchuk
commented
We need to call WEB API or Azure App Functions directly from Managed Instance.
The possible scenario is:
1) Create External data source to Blob Storage or Cosmos DB.
2) Create External Table
3) Save data to external table
4) Call App Function by trigger on Blob Storage or Cosmos DB. -
Jiayi Yang
commented
Please implement it.There is a requirement to implement PolyBase in the matter of data platform.
-
Mark
commented
Possible Scenarios in some of our Project:
Injest JSON or CSV files to Azure Blob Storage
Then read files into Azure SQL Managed Instance via polybaseThis could replace the staging area in data warehouse scenarios
-
Anthony Genovese
commented
Connect to SQL Server first, then Azure SQL and other PASS, then other Azure things like data lake
-
Anonymous
commented
This is, undoutedly, useful. Please enable PolyBase for SQL Managed Instance, which is designed to feature match fully functional SQL Server.
-
Eitan Blumin
commented
We have a need to off-load cold data to a separate server (i.e. archive).
Both StretchDB and PolyBase are unavailable in SQL Managed Instances, so we're pretty much left without options. -
Ron Tucker
commented
Polybase on MI would allow data migration from CSV files stored in data lake gen 2 without using Data Factory.
-
Anonymous
commented
We would like to load a Data Lake Gen2 with parquet and csv files, then use PolyBase within Managed Instance to read this data.
-
Matthew Mellifont
commented
Polybase support for ADLS Gen2 parquet files would be awesome. This would allow us to implement a modern data warehousing approach to customers utilizing MI as a DW via the use of DataBricks for source ingestion etc.
-
Sune Boesgaard Krog-Meyer
commented
Polybase support for Data lake Gen2 with delimited files would be very useful.
-
Christopher K. Fenwick
commented
We have data in oracle, sybase, and azure sql db, our main warehouse is in SQL Server on prem. Polybase would make the data easily accessible to our analysts without having them log in to multiple systems and export data.
-
Troy Holland
commented
We would like to load a Data Lake Gen2 with parquet files, then use PolyBase within Managed Instance to read this data.
-
S M Atiqur Rahman
commented
We want to use Azure Blob as our cold storage area. While we can use bulk load to copy data from blob to Azure Managed Instance, but we just want to reference the data from blob without making any copy.
-
hemanth konduru
commented
100 Likes.. Microsoft , Please enable Polybase on SQL Managed Instance