Support data compression for telemetry
Supporting formats such as Protobuf and/or compressing in DeviceClient.SendEventBatchAsync
This is not in our plans for now but definitively an interesting scenario.
As of today you can already do so in different ways such as compressing data in your device code before sending and using an Azure Function on the back end that will be triggered each time a compressed message arrives. Not totally integrated into IoT Hub but gives you the choice of the compression format and flexibility in routing messages before decompressing content (using message headers for routing).
JSON+gzip works well, but it can't be used with direct messages, as an example. A native implementation would be better
Any update on Data Compression as currently we are using GZIP in application code and in Stream Analytics input.
Please let me know whether Azure functions can be used to de-serialize protobuf at receiving end in IoT. Would it bring any performance issue? What is the recommended solution from Azure side to de-serialize protobuf format while receiving from IoT ?
MURTY ERANKI commented
Wanted to check if Azure IoT supporting Protobuf. any updates?
Olivier Bloch (MSFT) commented
Today's infrastructure for IoT Hub messaging is letting you put whatever you want in the payload of the messages. Nothing prevents from serializing your data with Protobuf on the device, sending up to IoT Hub and have your backend solution (using Stream Analytics, Functions, or your own worker role) deserialize the payload.
What more would you expect from IoT Hub?
Robin Jones commented
json is quite a lousy format when bandwidth is limited but streaming analytics depends on it, please support a compression format such as MessagePack which is automatically decompressed when ingested into IoTHub
Jon Gallant commented
There is currently no way to compress IoT Hub message collections. Currently the SendEventBatchAsync method does not compress the payload. We'd like the DeviceClient.SendEventBatchAsync method to behave exactly as it does today, but just enable compression BEFORE sending to IoT Hub and decompression BEFORE sending to EventHubClient.SendBatchAsync.
We know that we can create a receiver and decompress ourselves, but we don't want to do that because it breaks all downstream services that use IoT Hub as an input.