Migrating AWS CloudFormation templates to Azure Resource Manager templates

Infrastructure of Code is key to achieving consistent deployment of infrastructure in the cloud. Amazon Web Services enable infrastructure as code using CloudFormation templates ( https://aws.amazon.com/cloudformation/ ). In AWS’s own words:

“AWS CloudFormation gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.”

On Microsoft Azure we use Azure Resource Manager templates, commonly referred to as ARM templates. I can easily amend the above phrase to describe ARM templates:

Microsoft Azure Resource Manager templates give developers and systems administrators an easy way to create and manage a collection of related Microsoft Azure resources, provisioning and updating them in an orderly and predictable fashion.”

The similarities do not stop there. Both can use JSON, both define resources, both have parameters and the list goes on. This makes ARM templates relatively straightforward for AWS CF users to understand, however despite there being similarities there are also many differences, as ultimately the underlying services being provisioned are different. Over the last few months I have been involved in several projects where a customer has been deploying to AWS and are now wanting to deploy to Azure. They are currently using CloudFormation templates and want to deploy to Azure using ARM templates.

Although starting the design from scratch is often preferable, when using traditional IaaS services it is often relatively straightforward to work through the existing CloudFormation template and “migrate” it into an ARM template.

One important thing to note is given the underlying services are different in each cloud often architecture adjustments need to be made to enable the transition from AWS to Azure. Certain features available on AWS are not available on Azure, this would also be the case had a solution had been developed initially for Azure and then moved to AWS.

Here are a number of useful resources for users of AWS beginning their journey to Azure:

For the purpose of this blog, rather than using customer’s own CloudFormation templates which often contain sensitive information I will work through a simple example CloudFormation template, “Amazon EC2 instance in a security group”. The template is available here: https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/EC2InstanceWithSecurityGroupSample.template 

Although this template is very simple I follow the same process with more complex templates. The key steps I carry out are:

  • Analysis
    • Resources
    • Parameters
    • Mappings
    • Outputs
  • Template Creation

Analysis

The first step I take is to analyse the CloudFormation template to understand what is being deployed and how.

Resources

Using CloudFormation designer we can see the architecture is very simple:

Reading through the CloudFormation template we can map each AWS resource to an Azure resource. Azure has equivalent resources for all resources in the CloudFormation template hence we can map them directly. If there are no equivalent resources then architecure changes will need to be made.

Name AWS Resource Type Azure Resource Type
EC2Instance AWS::EC2::Instance Microsoft.Compute/virtualMachines
InstanceSecurityGroup AWS::EC2::SecurityGroup Microsoft.Network/networkSecurityGroups

When using Azure Resource Manager every virtual machine must have a network interface resource that is then connected to a virtual network. A public IP address must also be defined to allow external access to the virtual machine.

The resources required in the ARM template are:

Name AWS Resource Type Azure Resource Type
EC2Instance AWS::EC2::Instance Microsoft.Compute/virtualMachines
InstanceSecurityGroup AWS::EC2::SecurityGroup Microsoft.Network/networkSecurityGroups
VMNic   Microsoft.Network/networkInterfaces
VNet   Microsoft.Network/virtualNetworks
VMNicPIP   Microsoft.Network/publicIPAddresses

Parameters

The following parameters exist in the CloudFormation template:

Name AWS Data Type Resource(s) using parameter
KeyName AWS::EC2::KeyPair::KeyName EC2Instance
InstanceType String EC2Instance
SSHLocation String InstanceSecurityGroup

Mappings

Only mappings that are application specific, rather than being specific to AWS need migrating. The following mappings exist in the CloudFormation template:

Name AWS Specific Resource(s) using mapping
AWSInstanceType2Arch Yes EC2Instance
AWSInstanceType2NATArch Yes Not Used
AWSRegionArch2AMI Yes EC2Instance

As the mappings are AWS specific they do not need migrating.

Outputs

Again anything specific to AWS wil not be migrated. We will also note where outputs values need to be retrieved from.

Name AWS Specific AWS Resource retrieved from Azure resource retrieved from
InstanceId Yes    
AZ Yes    
PublicDNS   EC2Instance VMNicPIP
PublicIP   EC2Instance VMNicPIP

Now we have retrieved all the information we require we will begin creating the Azure Resource manager template.

Create ARM Template

As the VM depends on the network interface and the network interface depends on the virtual network, this is the first resource that needs to be created.

VNet

Details on how to add a virtual network can be found here: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-create-vnet-arm-template-click

I use Visual Studio to work with ARM templates. I add the virtual network, connect it to the VNic and finally remove a subnet that is not required along with it’s associated variables.

44942B33-2F9A-4ADD-A202-0029C45248ED

My template now looks like this:

VMNic

I then add a network interface and instruct Visual Studio to connect it to my VNet. Visual Studio automatically adds a parameter for the VMNic name. I convert this into a variable as it is not one of our identified parameters.

5D3CC532-22B4-40B6-AC95-49B4C1224651

My template now looks like this:

VMNicPIP

Details on how to create IP addresses can be found here: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-deploy-static-pip-arm-template , I use Visual Studio to add public IP address to VMNic and again convert a parameter to a variable:

497CBD2C-4040-4141-8651-454F71E0836C

My template now looks like:

I can now look at creating the Instance and Security Group. As the Instance definition depends on the Security Group I will create the Security Group, often referred to as an NSG first.

Security Group

In the CloudFormation template it exists as follows:

Parameters

First we need to create any input parameters the resource requires – we identified these earlier. Parameters in CloudFormation templates are very similar to ARM Template parameters. Both AWS and Azure have documentation explaining how parameters are defined.

Azure: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates#parameters

AWS: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html

SSHLocation

This is defined as follows in the CloudFormation template:

The is no concept of constraints and allowed patterns in an ARM template (although it would be useful if there was). The rest of the parameter definition is very similar, the changes being the case of the property names and removal of quotes around integers:

Resource

Details on how to create a NSG in an ARM template can be found here: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-create-nsg-arm-template

The only rule that needs to exist is to allow SSH. The NSG resource is therefore defined as follows:

EC2Instance

For obvious reasons I made the decision to rename the instance to AzureVM! The EC2 instance in the sample template is very simple:

Parameters

InstanceType

AWS instance types are listed here: https://aws.amazon.com/ec2/instance-types/ . A  t2.small has 1 cores  and 2GB RAM.

I visited https://docs.microsoft.com/en-us/azure/virtual-machines/virtual-machines-linux-sizes to find a machine with 1 cores and 2GB RAM. The closest I could find was a Standard_A1. In Azure we would usually refer to VM size, so I add a VM size parameter, with Standard_A1 as the default size:

KeyName

In the CloudFormation template the SSHKey is provides as a parameter with type AWS::EC2::KeyPair . For the Azure deployment we will use a KeyVault secret. I will add the parameter to the template to store the SSHKey, and describe how to deploy the template with a specific secret later.

Resource

An Azure VM must have 4 profiles defined, they are:

  • Hardware Profile
  • OS Profile
  • Storage Profile
  • Network Profile

In the EC2 definition we have the following properties:

  • InstanceType
  • SecurityGroups
  • KeyName
  • ImageId

Hardware Profile

The hardware profile sets the VM Size parameter as follows:

OS Profile

The OS profile defines settings within the image, such as computer name, admin username and SSH configuration. It will use the SSH key defined within the KeyName parameter.

Storage Profile

The storage profile specified the image the VM will be created from and the disk layout. The CloudFormation template uses an AMI which is pulled from a Mapping. In Azure base images from the marketplace are referenced using four properties: publisher, offer, sku and version. The VM only has a single OS disk so the storage profile look as follows:

Network Profile

The Network Profile associates VMNic with the VM.

Outputs

Both AWS and Azure have documentation explaining how outputs are defined.

AWS: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html.

Azure: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates#outputs

Earlier in the post I identified two outputs than need transferring to the ARM template, PublicDNS and PublicIP.

As the Public IP address is assigned dynamically we can not retrieve this until the VM is running and the address is assigned. The address is also likely to change should the VM ever be de-allocated. For this reason I will only create on output for the public DNS name. If we needed get the IP the allocation method would need to be changed to “static”.

The Public DNS output is defined in the CloudFormation template:

In the ARM template we define this as follows:

The template is now complete and can be found here:https://gist.github.com/marrobi/a4dfb8e5881ec71c30868f8119c5ecac

Deployment

Given the template uses an SSH key stored in KeyVault we need to provide a reference to the KeyVault secret when deploying the template. This is done within a parameters file, a description of how to do this can be found here: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-keyvault-parameter.. The parameters file I below contains the reference to a KeyVault secret. Any parameters not defined will be deployed with their default values.

To deploy the template I use the following command:

Conclusion

This post’s purpose is to give an overview of the process I go through when looking at CloudFormation templates and migrating them “like for like” to Azure. It is often advisable to start from scratch, but for basic IaaS templates this process works well. When moving beyond IaaS a more in depth understanding of the services that Azure offers is often required.

If anybody has any questions regarding migrating CloudFormation templates to Azure please don’t hesitate to post in the comments below.

Leave a Comment

Your email address will not be published. Required fields are marked *