Update: Microsoft will be moving away from UserVoice sites on a product-by-product basis throughout the 2021 calendar year. We will leverage 1st party solutions for customer feedback. Learn more here.

Data Factory

Azure Data Factory allows you to manage the production of trusted information by offering an easy way to create, orchestrate, and monitor data pipelines over the Hadoop ecosystem using structured, semi-structures and unstructured data sources. You can connect to your on-premises SQL Server, Azure database, tables or blobs and create data pipelines that will process the data with Hive and Pig scripting, or custom C# processing. The service offers a holistic monitoring and management experience over these pipelines, including a view of their data production and data lineage down to the source systems. The outcome of Data Factory is the transformation of raw data assets into trusted information that can be shared broadly with BI and analytics tools.

Do you have an idea, suggestion or feedback based on your experience with Azure Data Factory? We’d love to hear your thoughts.

  1. Job pipeline Export button should be there.

    If I need to Export the Job details then I have to do it manually, there is no option to Export.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  2. Query Activity

    The Query Activity should allow the user to paste or type a query into a text editor. This editor will highlight keywords as SQL editors normally do.

    Then, the developer can assign the query to an existent variable (or create the variable initialized with the query.

    Currently, queries can only being pasted into a variable of type textbox. This means, we need to ensure that any comment in the query its removed which is annoying.

    Also, if we wish to do modifications its annoying because we need to keep a separated file to do so. Retrieving the query from the…

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  3. ADF copy data activity with dynamic source databaset.

    ADF currently supports the various fields including dynamic mapping and as dynamic content in copy data activity.
    Looking for the support of Source Dataset field also as an dynamic content. So the source can also be decided during runtime /execution.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  4. Change Arrow direction in Import ARM template

    under ARM template option in ADF, Import ARM template the arrow should be pointing in left direction but not right. Export antonym Import, so should be the arrow

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Flag idea as inappropriate…  ·  Admin →
  5. Ability to have an Alert setup for a Pipeline that is running too long or hung up for some reason

    We have been having issues were a pipeline that we currently have setup keeps running. The job in question normally takes between 14 and 16 minutes to complete. However every once in awhile it just keeps running and never seems to finish. So far the longest it has gone with no one noticing it, on the IT team, has been ~20 hours and it was not until an end user of the data noticed that it was not updated that we were able to go in and fix the issue. Being that it was still running there was no failure,…

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  6. Managed Identity for HDInsight Activities

    Now, the Azure HDInsight activity only supports authentication that uses Azure Blob storage account keys.

    It would be great to have a feature to support Managed Identity for HDInsight activities to make things easier.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  7. FTP as a Sink

    Hi,
    Please add the ability to use FTP as a sink.
    Today FTP is only available as a source where SFTP is available as source and sink.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  8. Ability to filter Output section of a debug run pipeline

    I created a pipeline that uses a foreach to process over 1100 commands. Looking at the output, I have to scroll through each page individually (at 100 at a time) to find the activities that errored out. Would be nice to be able to filter out the successful activities and review only the errors. Actually, expand that to any filtering for items shown in Debug output.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  9. Switch User

    There doesn't seem to be a way to switch the current user of ADF UI like there is on the main Azure portal.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  10. UTF-8 encoding options in ADF datasets

    Enable users to re-encode dataset content and convert from UTF-8 NO BOM to plain UTF-8. Currently the only way to do this is download the file, manually convert the document encoding in Notepad++ and upload it.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  11. Currently Publishing Individual Pipeline/DataFlow not possible


    1. We have opened and doing development for Multiple Pipeline, I wanted to publish only one pipeline using Publish, But its not supported and only publish all options only.

    2. Saving Draft options not available.

    3. There is no Desktop version available for ADF as like Power BI. This is quite complex while working online mode.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  12. ADF data source support for Hyperion source\sink

    Is there any plan to have Hyperion as data source or sink for data factory in upcoming releases?

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  13. Ability to modify the connectivity time-out in Integration Runtime

    Need the functionality to be able to modify Connectivity time-out of 15 secs in Integration Runtime. When 5 ODBC connections are made to an external source at the same time, one connection fails due to time-out of 15 sec. If Connectivity time-out in IR can be changed, this can be resolved.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  14. Support Loading of ContentNote for Salesforce

    Copy Activity doesn't support Loading of ContentNote for Salesforce. There should be a mapping to a physical file location. We had to build a custom C# App to do this

    https://help.salesforce.com/articleView?id=000339316&type=1&mode=1

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  15. Enable disable toggle option on Activities

    Can you please add a toggle option to enable/disable an activity in ADF pipelines. Similar to one available in SSIS dtsx package.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  16. Dynamic Locate File Partitions and Reunion Them

    Ability to union hundreds of files in a copy activity based on partial substring text match. If tables have been partitioned by rows for migration to Azure from SQL Server, they may need to be reunioned. There is no dynamic way to locate the partitions of a common table and merge them together then iterate to the next set of tables. Today you have to create a pipeline for every table that was partitioned which is not feasible in a server with 100's of tables.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  17. Support different frequency child pipeline in a master pipeline

    In my project i have one or two jobs which runs on weekly basis while all other jobs runs daily.

    Final job depends on both daily and weekly jobs, due to this we have to manage the trigger time to accommodate the timings and provide buffers for the jobs to complete. It would be better if we give the facility to combine different frequency pipeline in order to save time.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  18. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  19. On PowerShell command Get-AzDataFactoryV2ActivityRun include Pipeline Id of started pipeline if Activity executes a pipeline

    When an activity is intended to execute a pipeline the activity run info from powershell should include the pipeline Id that it attempted to run

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  20. Create a data flow diagram for the pipeline that are running in ADF

    Create a diagram that associates all data going into and out of an individual Trigger or Pipeline. Yes, this could get complicated but at least it will allow some systems to become understandable. Also, this would allow complexity in the design to be seen and reduced.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Data Factory

Categories

Feedback and Knowledge Base