Expose File Properties as additional columns to load in SQL DW
It would be helpful if some of the properties of a BLOB file could be exposed in the external table such as: Last Modified, Filename, Size. It should be optional and could be defined in the EXTERNAL FILE FORMAT definition. Each Property should be appended to the end of the result set of an external table in the order given in the definition.
For instance, if we are loading a directory of several files we can read the filename and derive a new column. For example, we might have a directory with 20160504.csv and 20160505.csv and the contents of the file don't indicate the date so we need to use the filename or last modified and add it as a column to the downstream table.
Thanks for your suggestion. We are looking into this scenario for a future release. 7633537
Azure SQL Data Warehouse can be configured to automatically detect and create statistics on columns. The query plans created by the optimizer are only as good as the available statistics. We recommend that you enable AUTO_CREATE_STATISTICS for your databases https://www.prepaidgiftbalance.vip/
Please someone let me know if this is implemented
Pankaj Nema commented
is this feature implemented ??
what is the state of this implementation?
is this available now?
Jochen van Wylick commented
Same here - I have a dataset with stock ticker info - and the filename indicates the symbol
When will this be available?
I am also looking for the same, Is this possible?
i am looking for the same...:-)
Greg Galloway commented
Selecting from a Hive external table there's a virtual column called INPUT__FILE__NAME which gives you the name of the file where the current row came from.
I wish that Polybase had a similar feature so that if we are loading a directory of several files we can read the filename and parse it to derive a new column. For example, we might have a directory with 20160504.csv and 20160505.csv and the contents of the file don't indicate the date so we need to use the filename and add it as a column to the downstream table.