Need to allow job output to be used as input to downstream jobs
Today, if you want to have the output of one job be the input to another you must drop the output to blob storage or an event hub. The second job picks the data up from there. This adds 30-60s of latency per handoff.
Note that an ASA job can have up to five outputs. With that, it is often possible to compose what could be multiple jobs into a single one, gaining both resource elasticity between those parts of the composed job and improving end-to-end latency.
Please check this out https://www.youtube.com/watch?v=3xW-hUDsRIQ as to why this feature is needed, this is common in other stream processing engines
Amir Bahmanyari commented
Any progress on this pls?
Sudhesh Suresh commented
Hi Bryan, Will love to chat with you about this offline. Can you please send me your detailed scenario at firstname.lastname@example.org ? Thanks- Sudhesh