An SDK to generate ARM templates programmatically
For complex topologies or higher level abstractions it is not possible to have a static template because it may be overly complicated and hard to maintain. For such scenarios it will be good to have an SDK to generate ARM templates programmatically.
Existing Client Libraries are not entirely fitting ARM template language and require workarounds to be implemented to use them for template generation.
Vladimir Lozhkin
shared this idea
This is something we’re thinking about, but would like to ensure that we’re providing a sufficiently robust set of capabilities within the template language syntax itself.
What scenarios are “overly complicated and hard to maintain”? This will help us understand where we need to invest.
8 comments
-
Greg Lloyd
commented
During one of the ignite demos someone presented a way to leverage event grid, function apps, and a cosmos db. Using this they should that they were exporting ARM Templates based on changes to a Resource Group directly to cosmos db. The github repo is called 'azure-arm-events'.
-
Richie Costleigh
commented
Hi! Are there any prototypes or has there been any progress on this feature?
-
Sean Davis
commented
You could start by expanding whats in VS2017 with all the addiitonal settings and update the json in JSON outline. Right now it's very basic and I have to scour Automation settings in Azure which is useless after 200 resources are in there to get the info about a specific item.
-
David Lean
commented
1. Be nice if the Templates were Idempotent. Currently it has issues with Disks. Which if they already exist causes it to crash.
2. I'd like greater interdependence between values. eg:
When I drop from a GS5 (prod env) to a GS3(dev2 env), I need to create significantly fewer disks per storage. Or fewer storage groups with same number of disks. So I'd like a rule, function or something that could use a value of one parameter to control the generation of another.
Perhaps something more Powershell DSC like, to create the template like a MOF file.3. I'd like some attribute on the Disks I create, so that I can know what they are within Windows.
eg: I create 10 * P10's for the Log, 16 * P30's for the Data, 16 * P30's for the Indexes & a few more for TempDB. & some normal HDD's for other stuff. In Windows when I RAID them together, I need to ensure I group the P10's with P10's & so on. Be nice if there was a Tag, property or something I could set when I create the disk So I could use it in Windows storage pools.4. Depending on which machines we enable in the specific environment build, we'd then like to have the appropriate rules enables in the NSG. eg: Only if we enable the DataMart VM should we open port 1433. If we don't add the WEB VMs in the TEST4 environment, then we don't open ports 80, 433 etc
In short we create complete environments of 8+ Machines ( some have scale-out). Then want to swap the machine sizes depending on if it is Production, Pre-Prod, UAT, Dev, Test. And tweak it based on the size of Dev/Test environment we need to spin up.
We need the disks & networking to intelligently change based on the limits of the chosen VM size.
It would also be nice to use the same script to tear it down again. If we spin up a large stress test, I'd like to know I got rid of it all correctly, often even with Tags we miss a subnet, a nic or disk. And it is time-consuming. We also delete AD accounts but that is more DSC related. -
James Jackson
commented
If you want a way to create Azure Templates you might be worth looking at SparkleFormation, which I use for AWS CloudFormation and now supports Azure Templates. http://www.heavywater.io/blog/2016/01/20/sparkle-clouds/
-
seb brochet
commented
As a workaround to generate ARM templates programmatically, I've started to create some jinja2 macros (project is here: https://github.com/sebbrochet/azure-jinja2).
And this is the kind of syntax you get:
{% import 'azure-jinja2/macros/storage_account.json.j2' as storage_account %}
{% import 'azure-jinja2/macros/virtual_network.json.j2' as virtual_network %}
{% import 'azure-jinja2/macros/public_ip_address.json.j2' as public_ip_address %}
{% import 'azure-jinja2/macros/network_interface.json.j2' as network_interface %}
{% import 'azure-jinja2/macros/quick_virtual_machine.json.j2' as quick_virtual_machine %}
{% import 'azure-jinja2/macros/security_rules.json.j2' as security_rules %}
{% import 'azure-jinja2/macros/network_security_group.json.j2' as network_security_group %}"resources" : [
{{ virtual_network.from_vars() }},
{{ public_ip_address.from_vars() }},{% set jumbox_security_rules %}
[ {{ security_rules.SSH() }}, {{ security_rules.HTTPS() }} ]
{% endset %}
{{ network_security_group.from_vars(securityRules = jumbox_security_rules) }},{{ quick_virtual_machine.from_vars('jumpbox', withPublicIP = "true", withSSHKey = "true", withNSG = "true") }},
{% for name in ['docker1', 'docker2', 'docker3'] %}
{% set vm_name %}{{ name }}{% endset %}
{{ quick_virtual_machine.from_vars(vm_name, withSSHKey = "true") }}
{% if not loop.last %},{% endif %}
{% endfor %}
] -
Christopher
commented
So the all the tools and Azure SDKs exist that will enable you to create a solution that will build JSON on the fly.
A solution built in VS 2015 using the currently released Azure SDKs and info sync'd from Azure API would allow you to build ARM Templates as a single file as well as nested RGTs.
Though you have to taken into account the variable limits. Which is an issue.
Good Luck.
-
Jonathan Bedrava
commented
One example of something that's probably more complicated than it ought to be is how Azure SQL server tiers are specified in an ARM template. You need to know a GUID representing the service objective tier, the edition name, and then the maxSize you specify must fall within the a compatible range. Another example might be that some of the naming is not uniform between the Azure portal and the ARM template language. E.g., "cloud services" are "domain names" and so on. Despite the Very few of the questions I've had about leveraging ARM templates have been answered by documentation
Moreover, the overall capabilities of the ARM templates are shrouded in mystery unless you want to dig into all the existing gallery templates. I can now add storage accounts, but can I add a storage queue to that account? I have no idea.