Export logs to Azure storage
We are using Azure Automation to orchestrate a number of operations including (1) moving some files around in BLOB storage (2) provisioning an HDInsight cluster and (3) kicking off a job on that cluster to process the data files.
The HDInsight job stores its logs in BLOB storage and our Operational Support have requested that we store all logs from our system in the same place.
The Azure Automation output logs are available through the Azure Portal or via Powershell, however that does not meet the requirement to store all logs from our system in the same place. We would love to be able to specify that the Azure Automation output logs automatically get stored in some given location in Azure BLOB Storage. This would enable to consume them however we like (our current preference is to expose them as a Hive table within our HDInsight cluster).
In summary this request is ... give us the option to store the Azur Automation output in BLOB Storage so that we can consume those logs however we wish.
Sorry this item has not been updated. Forwarding logs from Azure Automation to Log Analytics is available. For more information please see the following documentation.
Charles M. commented
After such a long time - February 2016 - there is still nothing available... The logs seem to be stored in a DocumentDB which is always reaching thresholds. It's unusable... Can't we have an option to export these logs or store them in a blob as this was requested?
Jamie Thomson commented
For those that are interested I've written a runbook that can do this for you and made it available on the script gallery: https://gallery.technet.microsoft.com/scriptcenter/Archive-Azure-Automation-898a1aa8
Blogged about it here: http://sqlblog.com/blogs/jamie_thomson/archive/2014/11/25/archiving-azure-automation-logs-to-azure-blob-storage.aspx