Azure Resource Manager Templates: Template-Based Infrastructure Deployment
In the rapidly evolving cloud landscape, organizations face the challenge of efficiently deploying, managing, and scaling their infrastructure. Microsoft’s Azure Resource Manager (ARM) templates provide a robust solution for this challenge, enabling template-based infrastructure deployment that brings consistency, repeatability, and scalability to cloud resource management. By allowing infrastructure to be defined as code, ARM templates have transformed how organizations approach cloud deployments on Azure.
Azure Resource Manager templates are JSON files that define the infrastructure and configuration for your Azure deployments. These templates follow a declarative syntax pattern, where you specify the resources you want to create and their properties rather than programming the step-by-step process of creating them.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountName": {
"type": "string",
"metadata": {
"description": "Name of the storage account"
}
}
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[parameters('storageAccountName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2"
}
],
"outputs": {
"storageAccountId": {
"type": "string",
"value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
}
}
}
This simple template demonstrates the core components of an ARM template:
- Schema and content version: Defines the template structure and version
- Parameters: Values that can be provided during deployment
- Resources: The Azure resources to deploy
- Outputs: Values that are returned after deployment
ARM templates deliver several significant advantages for organizations managing Azure infrastructure:
The declarative approach lets you specify your desired end state without worrying about the sequence or complexity of deployment operations. Azure Resource Manager handles the orchestration of creating resources in the correct order based on dependencies.
Templates can be deployed repeatedly with consistent results. If a resource already exists in the specified state, no action is taken. This idempotency enables reliable redeployments and updates without fear of unintended consequences.
Well-designed templates can be reused across multiple environments and projects:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environmentName": {
"type": "string",
"allowedValues": [
"dev",
"test",
"prod"
],
"defaultValue": "dev"
},
"resourceNamePrefix": {
"type": "string",
"defaultValue": "contoso"
}
},
"variables": {
"storageName": "[concat(parameters('resourceNamePrefix'), parameters('environmentName'), 'storage')]"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[variables('storageName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "[if(equals(parameters('environmentName'), 'prod'), 'Standard_GRS', 'Standard_LRS')]"
},
"kind": "StorageV2"
}
]
}
This template can be deployed to different environments by simply changing the parameter values.
ARM templates support complete deployments, where resources not defined in the template are removed from the resource group. This ensures your Azure environment exactly matches your template definition and prevents configuration drift.
ARM templates integrate seamlessly with CI/CD pipelines in Azure DevOps, GitHub Actions, and other automation tools, enabling infrastructure-as-code practices:
# Azure DevOps YAML pipeline
trigger:
- main
pool:
vmImage: 'windows-latest'
steps:
- task: AzureResourceManagerTemplateDeployment@3
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: 'MyAzureConnection'
subscriptionId: '$(subscriptionId)'
action: 'Create Or Update Resource Group'
resourceGroupName: '$(resourceGroupName)'
location: 'East US'
templateLocation: 'Linked artifact'
csmFile: 'templates/main.json'
csmParametersFile: 'parameters/$(environmentName).parameters.json'
deploymentMode: 'Incremental'
For sophisticated infrastructure requirements, ARM templates provide several advanced capabilities:
Complex infrastructures can be modularized using nested or linked templates:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"networkTemplateUri": {
"type": "string"
},
"storageTemplateUri": {
"type": "string"
}
},
"resources": [
{
"name": "networkDeployment",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[parameters('networkTemplateUri')]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"virtualNetworkName": {
"value": "vnet-main"
}
}
}
},
{
"name": "storageDeployment",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"dependsOn": [
"networkDeployment"
],
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[parameters('storageTemplateUri')]",
"contentVersion": "1.0.0.0"
}
}
}
]
}
This modular approach improves maintainability and enables teams to work on different infrastructure components independently.
ARM templates include a rich set of functions for transforming, combining, and manipulating values:
"variables": {
"resourceName": "[concat(parameters('prefix'), '-', uniqueString(resourceGroup().id))]",
"location": "[resourceGroup().location]",
"isProduction": "[equals(parameters('environment'), 'production')]",
"skuName": "[if(variables('isProduction'), 'Standard', 'Basic')]",
"tags": {
"Environment": "[parameters('environment')]",
"DeploymentDate": "[utcNow('yyyy-MM-dd')]"
}
}
These functions enable dynamic values and conditional logic in your templates.
ARM templates can be deployed at different scopes, including subscription, management group, and tenant levels, enabling governance and organization-wide infrastructure patterns:
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2021-04-01",
"name": "data-processing-rg",
"location": "eastus",
"properties": {}
},
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"name": "dataProcessingDeployment",
"resourceGroup": "data-processing-rg",
"dependsOn": [
"data-processing-rg"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "dataprocessingstorage",
"location": "eastus",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2"
}
]
}
}
}
]
}
This subscription-level template creates both a resource group and resources within it.
For data engineering teams, ARM templates provide powerful capabilities for deploying and managing data infrastructure on Azure:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dataLakeName": {
"type": "string"
}
},
"variables": {
"storageAccountName": "[concat(parameters('dataLakeName'), 'store')]",
"dataFactoryName": "[concat(parameters('dataLakeName'), 'factory')]",
"databricksWorkspaceName": "[concat(parameters('dataLakeName'), 'dbricks')]"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[variables('storageAccountName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2",
"properties": {
"isHnsEnabled": true,
"networkAcls": {
"bypass": "AzureServices",
"defaultAction": "Deny"
}
}
},
{
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"name": "[variables('dataFactoryName')]",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
}
},
{
"type": "Microsoft.Databricks/workspaces",
"apiVersion": "2018-04-01",
"name": "[variables('databricksWorkspaceName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "standard"
}
}
],
"outputs": {
"storageAccountId": {
"type": "string",
"value": "[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]"
},
"dataFactoryId": {
"type": "string",
"value": "[resourceId('Microsoft.DataFactory/factories', variables('dataFactoryName'))]"
},
"databricksWorkspaceUrl": {
"type": "string",
"value": "[reference(variables('databricksWorkspaceName')).workspaceUrl]"
}
}
}
This template deploys a basic data lake architecture with Azure Data Lake Storage, Azure Data Factory for orchestration, and Azure Databricks for data processing.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"synapseWorkspaceName": {
"type": "string"
},
"sqlAdministratorLogin": {
"type": "string"
},
"sqlAdministratorPassword": {
"type": "securestring"
}
},
"variables": {
"dataLakeAccountName": "[concat(parameters('synapseWorkspaceName'), 'dls')]",
"sparkPoolName": "sparkanalysis",
"sqlPoolName": "datawarehouse"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[variables('dataLakeAccountName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2",
"properties": {
"isHnsEnabled": true
}
},
{
"type": "Microsoft.Synapse/workspaces",
"apiVersion": "2021-06-01",
"name": "[parameters('synapseWorkspaceName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('dataLakeAccountName'))]"
],
"properties": {
"defaultDataLakeStorage": {
"accountUrl": "[concat('https://', variables('dataLakeAccountName'), '.dfs.core.windows.net')]",
"filesystem": "synapse"
},
"sqlAdministratorLogin": "[parameters('sqlAdministratorLogin')]",
"sqlAdministratorLoginPassword": "[parameters('sqlAdministratorPassword')]"
},
"identity": {
"type": "SystemAssigned"
}
},
{
"type": "Microsoft.Synapse/workspaces/sqlPools",
"apiVersion": "2021-06-01",
"name": "[concat(parameters('synapseWorkspaceName'), '/', variables('sqlPoolName'))]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Synapse/workspaces', parameters('synapseWorkspaceName'))]"
],
"sku": {
"name": "DW100c"
},
"properties": {
"collation": "SQL_Latin1_General_CP1_CI_AS"
}
},
{
"type": "Microsoft.Synapse/workspaces/bigDataPools",
"apiVersion": "2021-06-01",
"name": "[concat(parameters('synapseWorkspaceName'), '/', variables('sparkPoolName'))]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Synapse/workspaces', parameters('synapseWorkspaceName'))]"
],
"properties": {
"nodeCount": 3,
"nodeSizeFamily": "MemoryOptimized",
"nodeSize": "Small",
"autoScale": {
"enabled": true,
"minNodeCount": 3,
"maxNodeCount": 10
},
"autoPause": {
"enabled": true,
"delayInMinutes": 15
},
"sparkVersion": "3.1"
}
}
]
}
This template creates a Synapse Analytics workspace with both SQL and Spark pools, providing a comprehensive analytics platform.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"projectName": {
"type": "string"
}
},
"variables": {
"eventHubNamespaceName": "[concat(parameters('projectName'), '-eventhub')]",
"eventHubName": "dataingest",
"storageAccountName": "[concat(parameters('projectName'), 'storage')]",
"functionAppName": "[concat(parameters('projectName'), '-func')]",
"hostingPlanName": "[concat(parameters('projectName'), '-plan')]"
},
"resources": [
{
"type": "Microsoft.EventHub/namespaces",
"apiVersion": "2021-11-01",
"name": "[variables('eventHubNamespaceName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard",
"tier": "Standard",
"capacity": 1
},
"properties": {}
},
{
"type": "Microsoft.EventHub/namespaces/eventhubs",
"apiVersion": "2021-11-01",
"name": "[concat(variables('eventHubNamespaceName'), '/', variables('eventHubName'))]",
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces', variables('eventHubNamespaceName'))]"
],
"properties": {
"messageRetentionInDays": 7,
"partitionCount": 4
}
},
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[variables('storageAccountName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_LRS"
},
"kind": "StorageV2"
},
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2021-02-01",
"name": "[variables('hostingPlanName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Y1",
"tier": "Dynamic"
}
},
{
"type": "Microsoft.Web/sites",
"apiVersion": "2021-02-01",
"name": "[variables('functionAppName')]",
"location": "[resourceGroup().location]",
"kind": "functionapp",
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', variables('hostingPlanName'))]",
"[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]",
"[resourceId('Microsoft.EventHub/namespaces/eventhubs', variables('eventHubNamespaceName'), variables('eventHubName'))]"
],
"properties": {
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('hostingPlanName'))]",
"siteConfig": {
"appSettings": [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('storageAccountName'), ';EndpointSuffix=', environment().suffixes.storage, ';AccountKey=', listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2021-04-01').keys[0].value)]"
},
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~4"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "dotnet"
},
{
"name": "EventHubConnection",
"value": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('eventHubNamespaceName'), variables('eventHubName'), 'RootManageSharedAccessKey'), '2021-11-01').primaryConnectionString]"
}
]
}
}
}
]
}
This template creates an event-driven data processing architecture with Azure Event Hubs for data ingestion and Azure Functions for processing.
Over years of implementation, several best practices have emerged for effectively using ARM templates:
Create separate parameter files for different environments to keep templates environment-agnostic:
// parameters.dev.json
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environmentName": {
"value": "dev"
},
"instanceSize": {
"value": "Standard_D2s_v3"
},
"deployHighAvailability": {
"value": false
}
}
}
// parameters.prod.json
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environmentName": {
"value": "prod"
},
"instanceSize": {
"value": "Standard_D4s_v3"
},
"deployHighAvailability": {
"value": true
}
}
}
This approach keeps environment-specific settings separate from your template logic.
Break complex deployments into modular templates:
templates/
├── main.json
├── modules/
│ ├── networking/
│ │ ├── vnet.json
│ │ └── nsg.json
│ ├── compute/
│ │ ├── vm.json
│ │ └── vmss.json
│ └── data/
│ ├── storage.json
│ └── database.json
└── parameters/
├── dev.parameters.json
└── prod.parameters.json
This structure improves maintainability and enables reuse of common components.
Implement consistent naming conventions using variables:
"variables": {
"resourcePrefix": "[concat(parameters('companyPrefix'), '-', parameters('environmentName'))]",
"storageAccountName": "[concat(variables('resourcePrefix'), 'stor', uniqueString(resourceGroup().id))]",
"keyVaultName": "[concat(variables('resourcePrefix'), '-kv')]",
"databricksWorkspaceName": "[concat(variables('resourcePrefix'), '-dbw')]"
}
Consistent naming makes resources easier to identify and manage.
Use tags to organize resources:
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-04-01",
"name": "[variables('storageAccountName')]",
"location": "[parameters('location')]",
"tags": {
"Environment": "[parameters('environmentName')]",
"Project": "[parameters('projectName')]",
"Department": "Data Engineering",
"CostCenter": "[parameters('costCenter')]",
"DeployedBy": "ARM Template",
"DeploymentDate": "[utcNow('yyyy-MM-dd')]"
},
// Other properties...
}
]
Comprehensive tagging improves governance, cost management, and resource organization.
Store and reference sensitive data using Azure Key Vault:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyVaultName": {
"type": "string"
},
"keyVaultResourceGroup": {
"type": "string"
},
"sqlAdminPasswordSecretName": {
"type": "string"
}
},
"resources": [
{
"type": "Microsoft.Sql/servers",
"apiVersion": "2021-02-01-preview",
"name": "[variables('sqlServerName')]",
"location": "[resourceGroup().location]",
"properties": {
"administratorLogin": "sqladmin",
"administratorLoginPassword": "[reference(resourceId(parameters('keyVaultResourceGroup'), 'Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), parameters('sqlAdminPasswordSecretName')), '2019-09-01').secretValue]"
}
}
]
}
This approach avoids storing secrets in templates or parameter files.
While ARM templates are powerful, their JSON syntax can be verbose and challenging to work with. Microsoft addressed this with Bicep, a domain-specific language that provides a more concise and readable syntax for Azure deployments:
// The same storage account in Bicep syntax
param storageAccountName string
param location string = resourceGroup().location
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' = {
name: storageAccountName
location: location
sku: {
name: 'Standard_LRS'
}
kind: 'StorageV2'
}
output storageId string = storageAccount.id
Bicep transpiles to ARM templates, offering the same capabilities with improved developer experience. For data engineers already familiar with ARM templates, Bicep represents a natural evolution rather than a replacement.
For robust deployment practices, integrate ARM templates into CI/CD pipelines:
# Azure DevOps pipeline for ARM template deployment
trigger:
branches:
include:
- main
paths:
include:
- templates/*
- parameters/*
variables:
- name: resourceGroupName
value: 'data-platform-rg'
- name: location
value: 'eastus'
stages:
- stage: Validate
jobs:
- job: ValidateTemplate
pool:
vmImage: 'ubuntu-latest'
steps:
- task: AzureCLI@2
inputs:
azureSubscription: 'Azure Subscription'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
az deployment group validate \
--resource-group $(resourceGroupName) \
--template-file templates/main.json \
--parameters @parameters/dev.parameters.json
- stage: Deploy_Dev
dependsOn: Validate
jobs:
- deployment: DeployDev
environment: Development
pool:
vmImage: 'ubuntu-latest'
strategy:
runOnce:
deploy:
steps:
- task: AzureCLI@2
inputs:
azureSubscription: 'Azure Subscription'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
az deployment group create \
--resource-group $(resourceGroupName) \
--template-file templates/main.json \
--parameters @parameters/dev.parameters.json
- stage: Deploy_Prod
dependsOn: Deploy_Dev
jobs:
- deployment: DeployProd
environment: Production
pool:
vmImage: 'ubuntu-latest'
strategy:
runOnce:
deploy:
steps:
- task: AzureCLI@2
inputs:
azureSubscription: 'Azure Subscription'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
az deployment group create \
--resource-group $(resourceGroupName) \
--template-file templates/main.json \
--parameters @parameters/prod.parameters.json
This pipeline validates templates, deploys to development first, and then to production after approval.
Azure Resource Manager templates represent a powerful approach to infrastructure deployment for organizations leveraging the Azure cloud. By enabling infrastructure as code, they bring consistency, repeatability, and automation to the deployment process, allowing teams to focus on creating value rather than manual configuration.
For data engineering teams, ARM templates provide a robust foundation for deploying complex data architectures—from data lakes and warehouses to processing pipelines and analytics workspaces. Their integration with Azure’s comprehensive data services enables end-to-end solutions that scale with growing data needs.
As organizations continue to embrace cloud-native approaches and DevOps practices, ARM templates and their evolution through Bicep will play an increasingly important role in enabling efficient, reliable infrastructure deployment on the Azure platform.
Keywords: Azure Resource Manager, ARM templates, infrastructure as code, template deployment, Azure, cloud automation, IaC, Bicep, deployment templates, resource provisioning, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, DevOps, CI/CD
#ARMTemplates #AzureResourceManager #InfrastructureAsCode #Azure #CloudAutomation #DevOps #IaC #Bicep #DataEngineering #DataOps #CloudDeployment #AzureDeployment #ResourceManagement #AzureSynapse #AzureDataFactory