Activity that copies and then deletes.
Thanks for the feedback. We are looking at this.
Keith Patton commented
this is blocking our adoption of azure data factory. We are happy to do a custom activity but have no working examples around how to delete a file using a file system linked service tied to on prem data source. I am coompletely bamboozled as to how a triggered copy activity could be of use to anyone without the ability to delete or archive the files as part of the operation, what am i missing!?
Please add at least a delete activity or a move operation
Jason Kohlhoff commented
Azure Data Factory definitely needs at least a Delete Activity.
Christopher Matthew Firenze Duizendstad commented
2 years under revision, really? Some one could give an update?
is there an ETA on this ?? It's been stagnant for way too long...
This definitely needs to be there...it just makes sense and fits the need
We strongly hope that Move (Copy & Deleat) will be implemented
A must have feature - please add the Move and Delete feature to ADF
Definitely need Move and Delete features added in ADF. The entire point of using such tools is for its simplicity while focusing on more important logic, If such basic features are missed out, then it is so disappointing especially when you take so much efforts in learning them.
Derek Adams commented
I've had nothing but praises for the Data Factory until I realized that I couldn't delete a file after copying it. So disappointing! A ton of work wasted unless this feature is added.
Hi there, Whilst the ADF 'Move/Delete/Rename' base function is being considered as an imporvement, can we use a Web activity to call the REST API provide by Data Lake to move or delete a file?
BUT, what are the instruction specific to calling Data Lake's REST API, and what are the url points etc..?
I've looked at this page, but it's fairly generic - NOT specfic to Data Lake:
Any help would be great.
IT can't be so complex "rm" ftp instruction ..... ADF base technology is "copy" files....I can imagine Microsoft IT engineer had this idea during implementation.
Ron DeFreitas commented
This is an absolute must have. Either as a "Move" Activity or a "Delete" Activity. It's absurd that this platform doesn't provide this capability.
I'm not fussed whether this is implemented as delete after copy or rename the file, or even move the file to another directory. The fact is copy it's useless without this functionality. It's almost the most basic integration pattern there is. I realise I can write a custom activity in .Net, but that shouldn't be necessary for such a simple use-case.
Do you have any information about this feature?
Move and delete source file is a common and important scenario..
James Devlin commented
We really need this feature. It's not great learning ADF and then finding out you have to also learn how to implement Custom Activity and then also find you have to learn Azure Batch.
Learning is a part of being an IT professional but at some point you just want to deliver a feature :/
Do you have any information when this feature will be available?
Craig R commented
I'm split on this one. I recommend "renaming" the file to achieve what you're looking for "and" avoid deleting. Below are my thoughts, and please correct me if I'm wrong.
Part of me says deleting files should be avoided because data loss is inevitable.
No matter how many checks you put in place, someone at sometime will introduce some code that does something wrong, and it may not even be your code (e.g. downstream consumer that deletes the output data so carefully protected).
And the other side of me realizes that in many cases moving data is a necessity, whether due to requirements of a system you don't control that outputs files of the same name, or even to deal with space concerns.
My recommendation is to avoid deletion and instead rename files. Rename them and it actually "can" move them on many of these technologies like ADLS. You may find you can achieve what you want "and" not delete the data. After all, the path is really just a label applied to a set of related data presented abstractly as a "file"... the folders don't exist in many of these systems anymore (e.g. you can vi a folder on linux because a folder on linux is just a special file)
Would like to know when this feature of Move.
Just absolutely nuts that this does not exist already. Really shows me that the technology is nowhere near mature.