Use the AzAPI provider to deploy Virtual Network Manager with Terraform

I believe most of us who works with Azure have felt the frustration of managing virtual networks as they grow in complexity. It’s easy to make mistakes when configuring peering and route tables and end up spending too much time running queries in Network Watcher to figure out what’s going on. Azure Virtual Network Manager aims to make this a lot easier and let us configure both Hub-Spoke and Mesh networks, as well as central management of security rules for all virtual networks.

As this new service is still in preview, there is no support for using the AzureRM provider in Terraform to configure it, but by using the AzAPI we can get up and running without diving into scripty shenanigans that breaks or otherwise declarative approach to infrastructure as code

Let’s start our example with defining our required providers, some basic configuration, and deploy a resource group, a hub network and two spoke networks. To save some typing (and make it a bit easier when creating a network group later on) we define the spoke networks in a local value and use for_each to create multiple networks.

 1terraform {
 2  required_providers {
 3    azapi = {
 4      source  = "azure/azapi"
 5      version = "= 0.4.0"
 6    }
 7    azurerm = {
 8      source  = "hashicorp/azurerm"
 9      version = "= 3.16.0"
10    }
11  }
12}
13
14provider "azurerm" {
15  features {}
16}
17
18data "azurerm_subscription" "current" {}
19
20locals {
21  spoke_subnets = {
22    "spoke1" = {
23      address_space = ["10.11.0.0/16"]
24    }
25    "spoke2" = {
26      address_space = ["10.12.0.0/16"]
27    }
28  }
29}
30
31resource "azurerm_resource_group" "test" {
32  name     = "NetworkManager"
33  location = "westeurope"
34}
35
36resource "azurerm_virtual_network" "hub" {
37  name                = "hub"
38  resource_group_name = azurerm_resource_group.test.name
39  location            = azurerm_resource_group.test.location
40  address_space       = ["10.10.0.0/16"]
41
42  subnet {
43    name           = "GatewaySubnet"
44    address_prefix = "10.10.0.0/24"
45  }
46}
47
48resource "azurerm_virtual_network" "spokes" {
49  for_each            = local.spoke_subnets
50  name                = each.key
51  resource_group_name = azurerm_resource_group.test.name
52  location            = azurerm_resource_group.test.location
53  address_space       = each.value.address_space
54}

Deploy Network Manager

We can then proceed with the more interesting part and deploy our Azure Virtual Network Manager. A common downside to using Terraform is that everyone will run into a case where the relevant provider not yet have support for the feature you wish to utilize. As preview-features in public cloud are subjected to many changes before they eventually become production ready (or deprecated and deleted) it makes sense that third party tools don’t focus on supporting these services, but even with features that have become generally available it can be annoying waiting periods as maintainers resolve dependencies, prioritize other improvements etc. As a work-around for this issue in Azure Microsoft has released the AzAPI provider.

All azapi_resource-blocks have the same set of required and optional parameters. As a minimum we have to supply body, name, parent_id, and type:

1resource "azapi_resource" "name" {
2    type = "Provider/resource-type@api-version"
3    name = ""
4    parent_id = 
5    body = 
6}

body: JSON object that contains the request body used to create and update azure resource
name: Specifies the name of the azure resource
parent_id: The ID of the azure resource in which this resource is created. Most common is using the either azurerm_resource_group.<name>.id or to the parent azapi_resource
type: Reference to the resource-type and api version for the resource created

As this provider is a very thin layer on top of the Azure Resource Manager REST APIs it also means that we need to get familiar with reading the API documentation to translate the API calls into Terraform code. For most of the resources a good tip is to start with the documentation for ARM Templates & Bicep.

To deploy a Network Manager we can go to Microsoft.Network networkManagers to get the API endpoint we will use as type and the structure of the body for the API endpoint. Using the jsonencode function in Teraform is very helpful when it comes to writing human readble code:

 1resource "azapi_resource" "network_manager" {
 2  type      = "Microsoft.Network/networkManagers@2022-04-01-preview"
 3  name      = "networkmanager"
 4  parent_id = azurerm_resource_group.test.id
 5  location  = azurerm_resource_group.test.location
 6
 7  body = jsonencode({
 8    properties = {
 9      networkManagerScopeAccesses = [
10        "Connectivity",
11        "SecurityAdmin"
12      ]
13      networkManagerScopes = {
14        subscriptions = [
15          data.azurerm_subscription.current.id
16        ]
17      }
18    }
19  })
20}

Network Groups and Hub-Spoke configuration

We can now proceed with adding configuration to the Network Manager. To set up a Hub-Spoke architecture we will need to complete the following steps:

 1resource "azapi_resource" "spoke_group" {
 2  type      = "Microsoft.Network/networkManagers/networkGroups@2022-04-01-preview"
 3  name      = "spokes"
 4  parent_id = azapi_resource.network_manager.id
 5
 6  body = jsonencode({
 7    properties = {
 8      memberType = "VirtualNetwork"
 9    }
10  })
11}
12
13resource "azapi_resource" "spoke_group_members" {
14  type      = "Microsoft.Network/networkManagers/networkGroups/staticMembers@2022-04-01-preview"
15  for_each  = azurerm_virtual_network.spokes
16  name      = each.value.name
17  parent_id = azapi_resource.spoke_group.id
18
19  body = jsonencode({
20    properties = {
21      resourceId = each.value.id
22    }
23  })
24}
25
26resource "azapi_resource" "hub_spoke_configuration" {
27  type      = "Microsoft.Network/networkManagers/connectivityConfigurations@2022-04-01-preview"
28  name      = "hub-spoke"
29  parent_id = azapi_resource.network_manager.id
30
31  body = jsonencode({
32    properties = {
33      appliesToGroups = [
34        {
35          groupConnectivity = "None"
36          isGlobal          = "False"
37          networkGroupId    = azapi_resource.spoke_group.id
38          useHubGateway     = "True"
39        }
40      ]
41      connectivityTopology  = "HubAndSpoke"
42      deleteExistingPeering = "True"
43      hubs = [
44        {
45          resourceId   = azurerm_virtual_network.hub.id
46          resourceType = "Microsoft.Network/virtualNetworks"
47        }
48      ]
49      isGlobal = "False"
50    }
51  })
52}

Worth mentioning here is that we can define members of a network group in two ways: Either static (like in this example), or dynamic (similar to dynamic memberships in Azure AD-groups). After creating a network group and adding members we create a Hub-Spoke configuration and apply it to the network group. Omitted from this post is the creation of a virtual network gateway used for transitive routing between spokes. The complete example can be copied from GitHub Gist

Commit configuration to Network Manager

Finally we have one last (annoying) hurdle to overcome before the Network Manager is operational. After creating the Hub-Spoke configuration and assigning it to a group of virtual networks we need to deploy the configuration. The bad news is that there is no good way of doing this with the AzAPI provider. Our choices are limited to:

Azure CLI - az network manager post-commit
Powershell - Deploy-AzNetworkManagerCommit
HTTP POST - Network Manager Commits - Post

In any way this means we fall back to our last resort and utilize a provisioner. In my opinion the lesser of three evils here is to use the third option and not introduce preview versions of Azure CLI/Azure Powershell as dependencies. To avoid having to replace the Hub-Spoke configuration we use a null_resource so we can easily run terraform apply -replace=null_resource.network_manager_commit to trigger the provisioner again if necessary

 1resource "null_resource" "network_manager_commit" {
 2  depends_on = [
 3    azapi_resource.hub_spoke_configuration
 4  ]
 5
 6  provisioner "local-exec" {
 7    command = <<CMD
 8            AccessToken=$(az account get-access-token --query accessToken --output tsv) && \
 9            curl -X POST https://management.azure.com${data.azurerm_subscription.current.id}/resourceGroups/${azurerm_resource_group.test.name}/providers/Microsoft.Network/networkManagers/${azapi_resource.network_manager.name}/commit?api-version=2021-02-01-preview \
10            -H "Content-Type: application/json" \
11            -H "Authorization: Bearer $AccessToken" \
12            -d '{"targetLocations": [ "${azurerm_resource_group.test.location}" ], "configurationIds": [ "${azapi_resource.hub_spoke_configuration.id}" ], "commitType": "Connectivity"}' \
13            -v
14        CMD
15  }
16}

If you prefer to use Azure CLI you can replace the null_resource and provisioner with:

 1resource "null_resource" "network_manager_commit" {
 2  depends_on = [
 3    azapi_resource.hub_spoke_configuration
 4  ]
 5
 6  provisioner "local-exec" {
 7    command = <<CMD
 8            az network manager post-commit \
 9                --commit-type Connectivity \
10                --network-manager-name ${azapi_resource.network_manager.name} \
11                --resource-group ${azurerm_resource_group.test.name} \
12                --target-locations ${azurerm_resource_group.test.location} \
13                --configuration-ids ${azapi_resource.hub_spoke_configuration.id}
14        CMD
15  }
16}

And there you have it. An easier way to manage complex networks in Azure. There is a lot more nice functionality in Azure Virtual Network Manager to check out. Instead of creating Hub-Spoke architectures you could create a mesh, you can create security rules that are evaluated before any network security groups (so you can let developers manage their NSG’s without being paranoid about them doing stupid things like opening RDP from the internet) +++