Azure subscription vending machine with Github and Terraform

This week we kicked off the first of six events in our Atea Community tour and I am lucky enough to be asked to host a session again this year. With more time, and a more technical fun stuff!

One of my demo's in this session is using issue templates and actions in GitHub to create a very basic "vending machine" where users can request a new landing zone subscription without any knowledge about Git, CI/CD, automation tools or Landing Zone Architectures, and just a tiny bit of admin/bureaucracy (basically having your pull request approved and merged by an adult).

Please note that the example described in this post is not exactly production ready. Because it is made in the context of fitting demo's into a 40 minute session shortcuts have been taken, some of the fields in the issue template form should use external lookups and not be user inputs, and the Terraform module being used have a lot of optional inputs to provide more flexibility and customization that are not used.

Issue Templates in GitHub

Issue and pull request templates gives us tools to make sure that relevant information is entered by users when they submit bugs, feature requests, security vulnerabilities, pull requests etc. to our repositories. All of this metadata can make it easier for us to solve bugs, understand why and how new features should be added, but if combined with GitHub Actions we can also create workflows to provide self service interfaces.

To start off my example for a "landing zone vending machine" I first create an issue template with four different inputs:

  1. The type of landing zone a user wants to request. this corresponds to the archetype definitions in terraform-azurerm-caf-enterprise-scale and subscriptions are placed in separate management groups under "landing-zones" in the Azure Landing Zones/Enterprise Scale-architecture.
  2. An Azure subscription ID. The only reason why this is a user input in the demo is because it was challenging to set up an identity with access to the the correct billing scopes and invoice sections. In a production ready implementation, you are more likely to add functionality to create new subscriptions on request and place them in the appropriate invoice sections, or have a pool of empty subscriptions in a separate management group that can be used.
    A challenging constraints here is that the issue templates does not give us interactivity where we can have a backend read relevant information about a user and provide personalized options in the inputs (e.g. to only display billing profiles and invoice sections a user have access to, get the user identity from Entra ID etc.). A better option could be to implement this interface in a Power App or creating your own frontend to provide more flexibility.
  3. The address space of the virtual network. My demo assumes that a virtual network is needed. Just like the bullet point above, this should probably not be a raw user input as it requires users to have information about which address space to use, not overlapping with other virtual networks, sizing etc. A better option would be to have a true/false and have the backend read information from an IP address management-system.
  4. If the virtual network should be peered with the central hub in the Connectivity Zone to be part of a Hub-Spoke architecture

./github/ISSUE_TEMPLATE/new-landing-zone.yml

name: Create new landing zone
description: Use this form to request a new landing zone in Azure
title: 'New landing zone: '
labels: ['landingzone']

body:
  - type: markdown
    attributes:
      value: |
        Deploy a new Azure Landing Zone with GitHub Actions
        
        ## Landing Zone Archetype
        
        * `corp`: Internal resources without access from the internet
        * `online`: For resources that will be accessible from the internet
        * `secure`: Resources that needs additional protection and security settings
        * `arc`: For managing on-premises resources with Azure Arc

  - type: dropdown
    id: archetype
    attributes:
      label: archetype
      description: Which type of Landing Zones do you want to deploy?
      options:
        - corp
        - online
        - secure
        - arc

  - type: input
    id: subscription_id
    attributes:
      label: Subscription ID
      description: Subscription ID for landing zone. Can be retrieved from the Azure Portal

  - type: input
    id: address_space
    attributes:
      label: Address Space
      description: IP address space for virtual network in CIDR format (e.g. 10.0.0.0/24)
      placeholder: 10.0.0.0/24

  - type: dropdown
    id: vnet_peering
    attributes:
      label: VNET peering
      description: Connect the virtual network to the hub
      options:
      - "true"
      - "false"

Create a landingzone configuration file

The second step of our vending machine process is to have a GitHub Action pick up on the newly created issue, parse the content, write a configuration file Terraform can use, and push this to our repository in a new branch along with a pull request.

./github/workflows/add-new-lz.yml

name: 'Create PR for new landing zone'

on:
  issues:
    types:
      - labeled

permissions:
  contents: write
  pull-requests: write
  issues: read

jobs:
  create_pr:
    if: github.event.label.name == 'landingzone'
    name: Create PR for new landing zone
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository from GitHub
      uses: actions/checkout@v4

    - name: Parse issue
      id: parse_issue
      uses: GrantBirki/issue-template-parser@v7.0.3
      with:
        body: ${{ github.event.issue.body }}

    - name: Format issue title for landing zone name
      id: issue_title
      run: |
        export name=$(echo "${{ github.event.issue.title }}" | cut -d: -f2 | awk '{$1=$1};1' | tr " " "-")
        echo title=$name >> $GITHUB_OUTPUT

    - name: Create config
      uses: 1arp/create-a-file-action@0.4.5
      with:
        path: 'infra/subscription-vending-machine/landing-zones'
        isAbsolutePath: false
        file: ${{ steps.issue_title.outputs.title }}.yaml
        content: |
          name: ${{ steps.issue_title.outputs.title }}
          subscription_id: ${{ fromJson(steps.parse_issue.outputs.json).subscription_id }}
          subscription_type: ${{ fromJson(steps.parse_issue.outputs.json).archetype }}
          virtual_networks:
            vnet1:
              address_space:
                - ${{ fromJson(steps.parse_issue.outputs.json).address_space }}
              hub_peering_enabled: ${{ fromJson(steps.parse_issue.outputs.json).vnet_peering }}

    - name: Create pull request
      id: pr
      uses: peter-evans/create-pull-request@v7
      with:
        title: 'Configure new landing zone: ${{ steps.issue_title.outputs.title }}'
        body: ${{ github.event.issue.body }}
        commit-message: 'Configure new landing zone: ${{ steps.issue_title.outputs.title }}'
        committer: LandingzoneBot <demo@de.mo>
        signoff: false
        branch: lz-${{ steps.issue_title.outputs.title }}
        draft: false

A few thing is worth mentioning in this workflow. The first is that the `issues` trigger does not have an option to specify a specific label. We do not want to run this workflow on bug reposts, feature requests etc, so instead we have a conditional on line 15 that will skip all the steps if the label "landingzone" is not set. It is not the best named label in the world, but then again: this is a demo ;)

The second part is that we want to sanitize the information from the Issue Title-field as we will use this to name our landing zone, configuration file and the feature branch created for the pull request. On lines 31-35 we use a collection of familiar linux tools to chop the issue title into two parts, get rid of the Issue prefix, trim whitespace at the start and end, and replace all whitespace between words with hyphens.

In the demo environment I also run multiple Terraform deployments from the same repository. The reason for this is to make it easier to share with attendees and provide a single URL at the end of my slide deck. The side-effect is that the path on line 38 should probably be updated if you want to test this on your own.

The end result of this action is that a new configuration file is created:

./infra/subscription/vending-machine/landing-zones/automation-testing.yaml

---
name: automation-testing
subscription_id: <uuid for subscription>
subscription_type: secure
virtual_networks:
  vnet1:
    address_space:
      - 10.42.69.0/24
    hub_peering_enabled: true

After approving and merging the pull request this will be merged into our main branch, and the final step of the process is triggered

Deploying the new landing zone

In our final step, we have another GitHub Action run `terraform apply` to deploy our changes

./github/workflows/deploy-new-lz.yml:

name: 'Deploy new landing zone'

on:
  push:
    branches:
      - main
    paths:
      - infra/subscription-vending-machine/**

  workflow_dispatch:

permissions:
  contents: read
  pull-requests: write

jobs:
  deploy:
    name: 'Deploy new landing zone'
    runs-on: ubuntu-latest
    environment: demo

    defaults:
      run:
        working-directory: ./infra/subscription-vending-machine
        shell: bash

    steps:
      - name: Checkout repository from GitHub
        uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_wrapper: false

      - name: Initialize Terraform
        run: terraform init -backend-config="subscription_id=${{ vars.MANAGEMENT_SUBSCRIPTION_ID}}" -backend-config="resource_group_name=${{ vars.BACKEND_RESOURCE_GROUP_NAME }}" -backend-config="storage_account_name=${{ vars.BACKEND_STORAGE_ACCOUNT_NAME }}" -backend-config="container_name=lz-vending" -backend-config="key=terraform.tfstate"
        env:
          ARM_CLIENT_ID: ${{ vars.ARM_CLIENT_ID }}
          ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
          ARM_SUBSCRIPTION_ID: ${{ vars.MANAGEMENT_SUBSCRIPTION_ID }}
          ARM_TENANT_ID: ${{ vars.ARM_TENANT_ID }}

      - name: Apply changes
        run: terraform apply -auto-approve -input=false
        env:
          ARM_CLIENT_ID: ${{ vars.ARM_CLIENT_ID }}
          ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
          ARM_SUBSCRIPTION_ID: ${{ vars.MANAGEMENT_SUBSCRIPTION_ID }}
          ARM_TENANT_ID: ${{ vars.ARM_TENANT_ID }}
          TF_VAR_connectivity_subscription_id: ${{ vars.CONNECTIVITY_SUBSCRIPTION_ID }}

This is a very simple workflow that provides no checks, approval gates or validation. All variables/secrets are read from GitHub and unlike the `issues` trigger in our previous step we can specify the `paths` where this workflow will be triggered to scope it only to our vending machine. In a more production ready setup you probably want to have the vending machine in a separate repo, have more complex workflows that runs `terraform validate` and `terraform fmt`, linting, security scans like checkov or trivy, has a `terraform plan` step to give you information to approve before `apply` and so on.

When triggered, the workflow will run this terraform deployment that essentially does two things:

  1. Scan a folder for configuration files. Each landing zone will have a separate file
  2. Call the terraform-azurerm-lz-vending-module and loop through the configuration files to apply it multiple times

./infra/subscription-vending-machine/main.tf

locals {
  landing_zone_data_dir = "${path.module}/landing-zones"
  landing_zone_files    = fileset(local.landing_zone_data_dir, "*.yaml")
  landing_zone_data_map = {
    for f in local.landing_zone_files :
    f => yamldecode(file("${local.landing_zone_data_dir}/${f}"))
  }
}

module "lz_vending" {
  source            = "Azure/lz-vending/azurerm"
  version           = "= 4.1.3"
  disable_telemetry = true

  for_each = local.landing_zone_data_map

  location = "westeurope"

  subscription_id                                   = each.value.subscription_id
  subscription_display_name                         = each.value.name
  subscription_management_group_association_enabled = true
  subscription_management_group_id                  = "${var.root_id}-${each.value.subscription_type}"

  virtual_network_enabled = true
  virtual_networks = {
    vnet1 = {
      name                    = "vnet-${each.value.name}"
      address_space           = each.value.virtual_networks.vnet1.address_space
      resource_group_name     = "rg-${each.value.name}-networking"
      hub_peering_enabled     = each.value.virtual_networks.vnet1.hub_peering_enabled
      hub_network_resource_id = data.azurerm_virtual_network.hub.id
    }
  }
}

This is a very minimal demo example. The module terraform-azurerm-lz-vending provides a lot of configuration options we can use to deploy managed identities, tag resource groups and resources, apply RBAC assignments +++

My complete demo environment from the session is available at github.com/atea/community-2024-clickops-til-kode if you want to see the full examples. Please don't run the examples in an environment you care about. It is a demo. Shortcuts have been made. There might be unexpected ghosts in the machines.