XML file type in copy activity.. along with XML schema validation
can we have a copy activity for XML files, along with validating schema of an XML file against XSD.. this would be helpful.. if schema validation is success then copy else fail the activity.. this will be useful for below scenarios..
1. Blob to Blob
2. Blob to SQL
3. SQL to Blob
if all above can work with specified schema that would be great...
Thank you for your feedback. We will evaluate this ask as part of product roadmap.
ADF is badly missing SSIS Source Adapter functionality, non-existent SSIS Destination (Sink in ADF parlance), as well as XML Task functionality for XSLT transformation and XSD validation.
Validate XML with the XML Task:
Jeroen H commented
Data tool that does not support XML... Is this 1995? (And I appreciate that JSON support is there, but XML is not going away soon in the big wide world)
Workarounds such as using string data type in copy activity add too much an overhead in my scenario given the size of xml files.
I've been following this thread for a while now and it's getting very awkward with my clients... please let us know if you're at least considering the implementation of this functionalities. If you're not we might consider moving to another cloud service.
Any update on this? Not much in ADF is moving...
Culshaw Stephen (Azure AD) commented
Amazing ... still no support XML data type ???
Crazy there is no support for XML data type. Someone below asked for a work around. My workaround was to use string data type in the copy activity, in you destination table use data type varchar(max) and then convert to XML in the DB. Obviously it adds overhead but was a simple solution for me
Brent Leslie commented
Not cool being unable to handle XML data. Pretty common activity for ETL. Until Data Factory supports common data formats and at least basic transformations, I can't large scale uptake.
It appears to be impossible to use XML file as BLOB source for the Copy activity in ADF V2 (as of this post).
Is there a work around in ADF pipeline to bring XML content to the SQL side for further shredding?
E.g. some Custom .NET activity to connect to BLOB source file, read the XML content as a String and then pass its value to the Sink connected to Azure SQL DB? Thanks.
Does ADF V2 have anything like XML Source in SSIS to do extract of XML as BLOB to Azure SQL DB at this time?
I have try https://blogs.msdn.microsoft.com/shwetasblogs/2015/05/06/solving-the-big-legacy-data-problem-using-azure-data-factory/
But i run code, it error at metadata ???
You can help me?
Suneet Nangia commented
This is coming from the scenario where we need to load xml data from blob storage and convert it to avro for Hadoop.