Update: Microsoft will be moving away from UserVoice sites on a product-by-product basis throughout the 2021 calendar year. We will leverage 1st party solutions for customer feedback. Learn more here.

Data Factory

Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources. You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems. The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.

Do you have an idea, suggestion or feedback based on your experience with Azure Data Factory? We’d love to hear your thoughts.

  1. refreshing Azure Analysis Cube

    Azure Data Factory pipeline activity to refresh Azure analysis services cube partitions.

    647 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    15 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Change Data Capture feature for RDBMS (Oracle, SQL Server, SAP HANA, etc)

    Is it possible to have CDC features on ADF please ?

    248 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  13 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. Allow Data Factory Managed identity to run Databricks notebooks

    Integrate Azure Data Factory Managed Identity in Databricks service.. like you did for Keyvault, storage, etc.

    128 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  7 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Support more complex types in Avro format, like Dictionaries and Arrays

    When trying to integrate a more complex scenario using Event Hub archive feature, I wasn't able to process this messages because Data Factory copy activity didn't support Dictionaries. When trying to use Stream Analytics writing to Avro format it didn't work because of the Arrays. More complex end-to-end scenarios should be supported.

    114 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  6 comments  ·  Flag idea as inappropriate…  ·  Admin →
  5. Elasticsearch

    source and sink.

    100 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    3 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Allow linking one factory to another

    I have been using the Walkthrough sample and successfully completed the exercise. This seems fairly straightforward and the entire experience of building a network of dependency between pipelines is great. This is very similar to SSIS but allows me to perform data integration @ scale with hybrid capabilities. My scenario is that we have few different teams within our organization and we need to have separate billing for each of these teams. I believe separating the subscription is the only option currently in Azure for separate billing. But we would like to allow one department to use the data of…

    97 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  5 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. 72 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  9 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Implement native error handling on ADF Data Flows

    Currently, ADF Data Flows don't handle errors natively. For example, if source columns in a file is a String and sink column in an Azure SQL Database is a Int, an error doesn't occur and a NULL value is loaded instead which is misleading.

    The only way to deal with that is to implement our own logic to detect inconsistent values which adds to the complexity of the data flow and development time.

    The Copy Activity in ADF itself has this fault tolerance feature where we can skip and log incompatible rows. It would be good to see ADF Data…

    24 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. A better debugging UI that will give more details about the job Id. Where it failed? What file it failed on?

    I'm new to Hive and I'm trying to figure out what I did wrong. But the only detail I get is below. It doesn't help much.

    Failed to submit Hive job: 1d625b0c-8e69-44ce-a0dc-21ba6b53db27. Error: An error occurred while sending the request..

    8 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  10. Source: Azure Blob; Target: IaaS SQL Server VM / Azure DB - Enable detailed mapping of SQL server types.

    We have a SQL Server (2014) table with data types like nvarchar(5), datetime, etc. At the moment loading data into an table with these data types fails. nvarchar(5) results into truncation errors. Datetime results into conversion errors.

    6 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. DataFlow Split Column should be supported

    Whuen trying to Split a column over a delimiter we are getting the error that at this moment this is not yet supported.
    Would be a great feature as frequently used in PowerQuery

    6 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. 3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    planned  ·  0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Data Factory

Categories

Feedback and Knowledge Base