Allow pipeline schedule to skip if already it is running (ADF V2)
Please add a feature to skip the schedule if the current schedule is already running.
For example I have a pipeline schedule for every 1 minute, if the pipeline is still running, the next schedule will start which causes the overlap in pipeline execution.
Right now I'm updating some records in SQL table which takes time until then the next schedule is starting which is again updating the same records because the previous pipeline schedule execution is not completed.
demolition porg commented
Yes, definitely need this.. surprised there's not a tickbox to just allow this in the trigger. Currently we're using code to check but this has be put as an azure function in each pipeline - which is not good, it really should be an option in the trigger
Thomas Andersen commented
You can specify concurrency for tumbling window triggers, but that will queue the pipeline runs. It would be nice if you could, as this proposal suggests, skip if already running.