This is the fifth post in a multi-part post that describes using a GitHub Action and Terraform to deploy infrastructure in Azure. In this post I will create a GitHub Action that will run the Unicorn CI/CD pipeline.
Adding the GitHub Action
In GitHub, click on the Actions tab. GitHub has a lot of templates to get you started quickly. I figured I would use the Terraform template. However, after looking at the template, it didn’t do anything that I actually wanted. For example, it stores the .tfstate file in the Terraform Cloud and uses an API token to connect to it. I want to store my .tfstate file in Azure blob storage and connect to it with an Azure security principal using OAuth2. So I decided to start from scratch.
GitHub actions are a YAML file in a specific folder: ”{repo-root}/.github/workflows”. I created that folder structure with:
|
1 2 |
mkdir .github mkdir ./.github/workflows |
Next, I added a file to the workflows folder named main.yml and added some scaffolding for my GitHub Action.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
name: Deploy Unicorn on: push: branches: - main jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Run a command line script run: echo "Hello, World!" |
The name of the Action is “Deploy Unicorn”. It runs whenever someone pushes to the main branch. It has a single job (so it runs on a single runner). It has two steps. First, it checks out the repository. Second, it runs the Linux command echo and prints “Hello, world!” to the log.
Branching Strategy
My use case is to implement a GitHub Action that supports the GitHub Flow branching strategy. The GitHub Flow is a simple branching strategy designed for very small teams. Basically, the main branch contains the current production code. Developers create feature branches to implement features and commit to the published upstream branch as they develop their features. Perhaps they test locally using VS Code or Visual Studio. When their feature is complete, they create a pull-request to merge from their feature branch to the main branch. This gives the dev lead an opportunity to review and comment on their code. When the pull-request is approved/complete, a GitHub Action runs that deploys the code to the production environment. To achieve this, the following things are required:
- GitHub Action YAML is in ./.github/workflows directory
- GitHub Branch Protection Ruleset with the “Require a pull request before merging” rule is active (this prevents people from pushing directly to main)
- GitHub Action Trigger is:
|
1 2 3 4 |
on: push: branches: - main |
Branch Rule Sets
GitHub supports rules, called Rulesets, that protect branches from a variety of misbehavior and bad practices. I do not want developers by-passing the code-review (pull-request) process and pushing their code directly to main and by extension, production. So, I’m going to create a branch Ruleset to protect the main branch.
Click Settings, then under Code and Automation, click Rulesets. I gave my new ruleset a name, set it to Active, set the Target branch to default (the main branch) and checked the “Require a pull request before merging” checkbox. I require at least one approval. Finally, click Create.
Now, when I try to push directly to main, I get an error.
Pushing directly to the main branch is blocked by the Ruleset. Since I have created a Ruleset, I had to stash my changes, create a feature branch and pop the stashed files.
|
1 2 3 4 |
git add ./.github/workflows/main.yml git stash git branch feature/github-actions git stash pop |
Refactoring the Terraform template
In order to run the Terraform template from the GitHub Action, we need to refactor it. The refactor involves the following steps.
- Create the production GitHub Environment
- Create secrets (for the security principal) in the production environment
- Create variables (about where the state file is) in the production environment
- Refactor main.yml to read the variables and secrets and set environment variables on the runner
- Refactor unicorn.tf to remove the hard-coded state file information
Create the production GitHub environment
In the post about creating the security principal, I mentioned that the Unicorn project would use an GitHub Environment named “production”. This was related to the Subject of the Federated Credential. The Subject Identifier of the security principal is “repo:<github-organization-name>/unicorn:environment:production. So the environment name must be “production”. It’s time to create that environment. In GitHub, click Settings, then under Code and automation, click Environments and finally, the New environment button.
On the Environments/Add form, set the Name field to production and click the Configure environment button.
Create secrets (for the security principal) in the production environment
In the environment secrets, we need to create three secrets.
- AZURE_CLIENT_ID
- AZURE_SUBSCRIPTION_ID
- AZURE_TENANT_ID
The values for these secrets can be found in the Azure portal. To find the subscription id, click on the search bar at the top of the portal (G+/) and start typing subscription. It should come right up. Click the Subscriptions icon and select your subscription from the list.
The client id and tenant id can be found in Microsoft Entra Id in App Registrations. Search for the security principals name (unicorn) under All Applications.
Once you have collected all the values, set the secrets in GitHub. Click Add environment secret.
Enter the secret name, value and click the Add secret button.
After the three environment secrets have been added, we will need add Environment variables.
Create variables in the production environment
In the unicorn.tf template, there is a block named “backend” that defines where the state file is located. We don’t want those values hard-coded, we want them defined in Environment variables. Use the Add environment variable button to add four variables. The values should be copied from the unicorn.tf Terraform template.
- AZURE_STATEFILE_STORAGE_ACCOUNT_NAME
- AZURE_STATEFILE_CONTAINER_NAME
- AZURE_STATEFILE_STORAGE_ACCOUNT_RESOURCE_GROUP_NAME
- AZURE_STATEFILE_KEY
Refactor main.yml to read the variables and secrets and set environment variables on the runner
Now that we have the secrets and variables set up in GitHub, refactor main.yml. Information about how to access the .tfstate file will be set as environment variables on the GitHub Action Runner.
When the GitHub Action Runner authenticates, it sets an envirionment variable with the value of the authentication token. To do that, it has to be granted permission. In main.yml, below the on: statement, add a permissions block.
|
1 2 3 |
permissions: id-token: write contents: read |
Next, under the permissions block, use the env: keyword to set environment variables on the runner with values from the environment secrets and variables in GitHub.
|
1 2 3 4 5 6 7 8 9 10 |
nv: AZURE_STATEFILE_STORAGE_ACCOUNT_NAME=${{ env.AZURE_STATEFILE_STORAGE_ACCOUNT_NAME }} AZURE_STATEFILE_CONTAINER_NAME=${{ env.AZURE_STATEFILE_CONTAINER_NAME }} AZURE_STATEFILE_STORAGE_ACCOUNT_RESOURCE_GROUP_NAME=${{ env.AZURE_STATEFILE_STORAGE_ACCOUNT_RESOURCE_GROUP_NAME }} AZURE_STATEFILE_KEY=${{ env.AZURE_STATEFILE_KEY }} ARM_CLIENT_ID=${{ secrets.AZURE_CLIENT_ID }} ARM_SUBSCRIPTION_ID=${{ secrets.AZURE_SUBSCRIPTION_ID }} ARM_TENANT_ID=${{ secrets.AZURE_TENANT_ID }} ARM_USE_OIDC=true ARM_USE_AZUREAD=true |
What is going on here?
- The
env:statement is used to set environment variables on the GitHub Action Runner - The following lines set individual environment variables
${{ env.AZURE_STATEFILE_CONTAINER_NAME }}uses the GitHub Repository Variable value${{ secrets.AZURE_CLIENT_ID }}uses the GitHub Repository Secret
The end result is that environment variables will be set on the runner that will later be consumed by terraform init command.
Before we can run terraform init, we need to set up Terraform on the runner. This can be done with the hashicorp/setup-terraform@v1 module. Below the steps: statement, under the actions/checkout@v4 statement, add the following block to main.yml.
|
1 2 3 4 |
- name: Setup Terraform uses: hashicorp/setup-terraform@v1 with: terraform_version: 1.0.0 # Specify the Terraform version |
Finally, we can run the Terraform init command. Add the following command to initialize the Terraform environment on the runner.
|
1 2 3 4 5 6 |
- name: Terraform Init run: terraform init \ -backend-config="storage_account_name=${{ env.AZURE_STATEFILE_STORAGE_ACCOUNT_NAME }}" \ -backend-config="container_name=${{ env.AZURE_STATEFILE_CONTAINER_NAME }}" \ -backend-config="resource_group_name=${{ env.AZURE_STATEFILE_RESOURCE_GROUP_NAME }}" \ -backend-config="key=${{ env.AZURE_STATEFILE_KEY }}" |
Now, what is going on here?
- The
run:is used to run a command in the shell - Call
terraform init, the\is the GitHub Actions YAML line continuation character - The following lines set parameters in the “backend-config” for Terraform and describe where to find the statefile; backend-config may be called repeatedly and the configurations are added, not replaced.
What isn’t clear is how terraform init authenticates to Azure. terraform init uses ARM_USE_AZUREAD=true to know that it is going to authenticate to Azure AD (Microsoft Entra ID) and ARM_USE_OIDC=true to know to use the OpenID Connect protocol. OpenId Connect requires the client id, subscription id and tenant id which are set with the ARM_CLIENT_ID, ARM_SUBSCRIPTION_ID and ARM_TENANT_ID environment variables. The environment variables must have those names or the terraform init statement will not work.
The entire main.yml file now looks like this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
name: Deploy Unicorn on: push: branches: - main permissions: id-token: write contents: read env: AZURE_STATEFILE_STORAGE_ACCOUNT_NAME=${{ vars.AZURE_STATEFILE_STORAGE_ACCOUNT_NAME }} AZURE_STATEFILE_CONTAINER_NAME=${{ vars.AZURE_STATEFILE_CONTAINER_NAME }} AZURE_STATEFILE_STORAGE_ACCOUNT_RESOURCE_GROUP_NAME=${{ vars.AZURE_STATEFILE_STORAGE_ACCOUNT_RESOURCE_GROUP_NAME }} AZURE_STATEFILE_KEY=${{ vars.AZURE_STATEFILE_KEY }} ARM_CLIENT_ID=${{ secrets.AZURE_CLIENT_ID }} ARM_SUBSCRIPTION_ID=${{ secrets.AZURE_SUBSCRIPTION_ID }} ARM_TENANT_ID=${{ secrets.AZURE_TENANT_ID }} ARM_USE_OIDC=true ARM_USE_AZUREAD=true jobs: build: runs-on: ubuntu-latest environment: production steps: - uses: actions/checkout@v4 - name: Setup Terraform uses: hashicorp/setup-terraform@v1 with: terraform_version: 1.0.0 # Specify the Terraform version - name: Terraform Init run: terraform init \ -backend-config="storage_account_name=${{ env.AZURE_STATEFILE_STORAGE_ACCOUNT_NAME }}" \ -backend-config="container_name=${{ env.AZURE_STATEFILE_CONTAINER_NAME }}" \ -backend-config="resource_group_name=${{ env.AZURE_STATEFILE_RESOURCE_GROUP_NAME }}" \ -backend-config="key=${{ env.AZURE_STATEFILE_KEY }}" |
Refactor unicorn.tf to remove the hard-coded state file information
The elements of the backend block can be removed from unicorn.tf, but leave the empty block there or you will get an error. unicorn.tf now looks like this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
terraform { required_version = ">=1.2" required_providers { azurerm = { source = "hashicorp/azurerm" version = "~>3.7.0" } } backend "azurerm" { } } provider "azurerm" { features {} } provider "azurerm" { features {} } |
Finally the changes are committed to the feature branch and published to the repo (not shown).
Over in GitHub, on the Code tab, I clicked Compare & pull request.
On the Comparing changes page, click Create pull request.
On the Open a pull request page, click Create pull request.
On the pull request page, click Merge pull request and Confirm merge (not shown).
Finally, the pull request screen showing a successful merge.
At this point, the Action ran and connected to the state file successfully.
This was a long post, and it covered a lot of ground. It covered:
- Creating a barebones GitHub Action
- Described the branching strategy
- Creating the branch Ruleset
- Refactoring the Terraform template
- Creating the GitHub Environment
- Creating GitHub Secrets and Variables
- Refactoring main.yml to read the GitHub Secrets and Variables
- Refactoring the Terraform template to remove hard-coded state file information
- Running the GitHub Action
In the next post, we’ll look at adding the infrastructure that will host the service.