How can we improve Microsoft Azure Data Factory?

Use partition in filefilter and filename

At the moment you can only use * and ? in the file filter. It would be very helpful if you could use the partitionedBy section which you can use for the folderpath in the filefilter or the filename as well.

This would allow scenarios where you need files like myName-2015-07-01.txt where the slice date and time is part of the filename.

268 votes
Vote
Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
You have left! (?) (thinking…)
Benny shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

Thank you for your feedback. This can be accomplished in ADF V2 by passing in the value of a trigger variable using pipeline parameter and dataset parameter. Then you can construct a parameterized folder path name and/or filename, e.g. myName-yyyy-MM-dd.txt in your example.

Please refer to this article for more details: https://docs.microsoft.com/en-us/azure/data-factory/how-to-read-write-partitioned-data

7 comments

Sign in
(thinking…)
Sign in with: Microsoft
Signed in as (Sign out)
Submitting...

Feedback and Knowledge Base