Read one queue item at a time
When calling a change token-based external API (get records since last change) from Azure Functions queue it is very important that queue items are run in order, one by one.
Explored several solutions to this, such as durable functions, but without luck.
Currently using queues.batchSize = 1 to ensure concurrency, but this will affect all queues.
If using consumption plan WEBSITEMAXDYNAMICAPPLICATIONSCALE_OUT must be set to 1, or else there might be several instances of the same function, and the batchSize fix won't help.
My dream scenario is some "concurrency key" in function.json, ensuring items with the given key (string) will always run in sequence, no matter what.
This would allow atomic item execution not on a host-level, or queue-level, or item-level, but item-value-level.
For example to set a queue to be fully concurrent simply set the value to true, to make specific items of a "category" run in sequence insert "{category}".
Eirik Brandtzæg
shared this idea
We are currently not planning on enabling this at a function level. Will keep open to see if others require this level of granularity. Thanks for submitting.