Please add function that "get Metadata" Activity could read files and folders recursively
I want to move files recursively with "Move files by chaining the Copy activity and the Delete activity".
But "get Metadata" Activity" dose not work recursively.
I hope to add a function that "get Metadata" Activity could read files and folders recursively
■Move files by chaining the Copy activity and the Delete activity
Marko Oja commented
I wan't to load files based on their last modified time in a blob folder structure like yyyy/MM/dd.
I need to save latest modified time so I can load from the last point in time I actually managed to load all the files successfully. As of now I am unable to do this efficiently.
If my files were in a same folder (maintenance nightmare) I would be able to use get child items of the get metadata activity. As of now this is not possible.
As Paulo commented before option to choose recursively with object type and in my case using last modified information as filter (as it is currently already working) would be great.
Paulo Correia commented
What about getting metadata recursively on a data lake because I want to perform schema validation on each file.
From my finding there's no way to setup a recursive pipeline, so this feature would be great.
a simple solution would be giving an option return a full list of child items of a specific type, either file or folder, along with their "itemPath" property