Enable CDM support for Azure Synapse Analytics SQL on-demand
The "Common Data Model" (CDM) format is becoming increasingly popular. Therefore it would be important that this connector not only exists in the ADF, but that it is also possible to read and write (via CETAS) in CDM directly from SQL on-demand.
SELECT * FROM
BULK STORAGE ACCOUNT,
FORMAT = CDM
) AS [r];
CREATE EXTERNAL TABLE cdm.cdm_table
LOCATION = path_to_folder,
DATA_SOURCE = external_data_source_name
FILE_FORMAT = CDM
) AS SELECT * FROM source
Michael Hammons commented
I thought the one of the main points of Synapse was to run SQL over Azure Data Lake Gen 2 which has a key feature of CDM folders and the json meta data description?!
I should not have to convert the CDM data to parquet in order to read it from Synapse.
NAGATA Ryoma(永田 亮磨) commented
This feature is very important.
The ability to use Serverless Query on CDM folders created by the Common Data Service's ADLS integration allows for HTAP for Power Platform apps and Dynamics (like Synapse Link in Cosmos DB).