How to use Github OIDC and Terraform to assume roles in AWS using WebIdentity

When using Github Actions to deploy infrastructure to AWS with Terraform, you can use Open ID Connect (OIDC) to grant Github access to AWS without needing to provide access keys.

As a GitHub workflow, this would be somewhat like this:

And as a diagram, it would look something like this:

This makes deploying a solution more complex, since the code needs to be broken up in parts per account (stacks). This requires a separate Terraform state file for each stack, and will (most likely) result in multiple stacks with dependencies on each other.

Or, alternatively, we’ll be role chaining, which has some limitations, as well as administrative challenges.

When using web identities, e.g. setting up OIDC in every accounts, we can can set conditions per role, per account.

Using ‘AssumeRoleWithWebIdentity’
#

The AWS API supports assuming roles using a web identity.
We can use this in Terraform to logically group resources in stacks, regardless of the accounts in which they need to be deployed, and without the need to create AssumeRole-policies in every account.

To use this method, we have to set up GitHub OIDC on each target account with a corresponding role and create a web token file in the workflow.

On the Terraform-side, we need to use at least version 4.22.0 of the AWS provider, since a bug which prevented the assume_role_with_web_identity to work properly has been fixed in that version.

For the S3 backend, we also need to create a credentials file, since the backend doesn’t (yet) support web identities.

NOTE: To get this to work, no AWS-specific environment variables should be set in the workflow. If there are any, those can disrupt the process.

DISCLAIMER: The examples given in this post do not necessarily show the use of best practices; they are given purely to show how the mechanics work.

Setting up OIDC in an account
#

To set up OIDC, we have to add an identity provider, provide the Provider URL, get the thumbprint of the provider certificate,set an audience and assign a role. The procedure can be found on GitHub Docs.

Alternatively, a CloudFormation template can be used to accomplish the same, as well as create a role and assign it to the identity provider to use it (thanks Aidan Steele):

Parameters:
  GitHubOwner:
    Type: String
    Description: Owner of the repository/repositories
  GitHubRepositoryFilter:
    Type: String
    Description: Filter to determine the repositories the role can be assumed from
    Default: "*"
  GitHubActionsRoleName:
    Type: String
    Description: Name for the GitHub Actions OIDC Role
    Default: GitHubActionsRole

Resources:
  Role:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${GitHubActionsRoleName}
      ManagedPolicyArns: [arn:aws:iam::aws:policy/AdministratorAccess]
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Action: sts:AssumeRoleWithWebIdentity
            Principal:
              Federated: !Ref GitHubOidc
            Condition:
              StringLike:
                token.actions.githubusercontent.com:sub: !Sub repo:${GitHubOwner}/${GitHubRepositoryFilter}:*
              StringEquals:
                token.actions.githubusercontent.com:aud: "sts.amazonaws.com"
  GitHubOidc:
    Type: AWS::IAM::OIDCProvider
    Properties:
      Url: https://token.actions.githubusercontent.com
      ThumbprintList: [6938fd4d98bab03faadb97b34396831e3780aea1]
      ClientIdList: 
        - "sts.amazonaws.com"

Outputs:
  Role:
    Value: !GetAtt Role.Arn

*NOTE: In this CloudFormation template the role is given full administrator access to the account. Best practice mandates that you use the principles of least privilege, especially in a production environment!

Once the OIDC connection has been set up for an account, and a role has been assigned to the connection, we can use a web token to connect to the account and assume the role.

To get the web token, we need to create a web identity token file.

Creating a web identity token file
#

In the workflow, we add a step to create a web identity token file, as described on this GitHub Document (but adjusted for AWS):

      - name: Get OIDC Token
        id: get_oidc_token
        run: |
          curl -s -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=sts.amazonaws.com" | jq -r '.value' > /tmp/web_identity_token_file

Using the web identity token file in Terraform
#

We can use the web identity token file directly in the AWS provider configuration in Terraform. This ensures that all changes will have to go through a GitHub Workflow.

Using a web identity for the S3 Backend
#

The Terraform S3 backend doesn’t currently support the use of a web identity directly. It does, however, support the use of a profile.
AWS Documentation on how to configure profiles can be found here.

So we can define our backend as follows:

terraform {
  backend "s3" {
    bucket                  = "terraform-state-bucket-111111111111-eu-west-1"
    key                     = "eu-west-1/terraform-webidentity/terraform.tfstate"
    region                  = "eu-west-1"
    encrypt                 = true
    dynamodb_table          = "terraform-state-lock-table-111111111111"
    profile                 = "backend"
    shared_credentials_file = "~/.aws/credentials"
  }
}

With the corresponding profile entry in ~/.aws/credentials (we set up this file in the workflow later on in this write-up):

[backend]
region=eu-west-1
role_arn=arn:aws:iam::111111111111:role/GitHubActionsRole
web_identity_token_file=/tmp/web_identity_token_file

Configuring the AWS provider(s)
#

Terraform provides assume_role_with_web_identity in provider configurations, which we can use like this:

provider "aws" {
  # default profile set to 'staging' account
  region  = var.aws_region

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::222222222222:role/GitHubActionsRole"
    session_name            = "github_action_session"
    web_identity_token_file = "/tmp/web_identity_token_file"
  }
}

provider "aws" {
  region  = var.aws_region
  # Similar to the default provider, but this makes it easier to differentiate between providers in the code
  alias   = 'staging'

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::222222222222:role/GitHubActionsRole"
    session_name            = "github_action_session"
    web_identity_token_file = "/tmp/web_identity_token_file"
  }
}

When you want to deploy to an additional account, we just have to add another provider with an assume_role_with_web_identity block and an alias. For example:

provider "aws" {
  region  = var.aws_region
  alias   = "production"

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::333333333333:role/GitHubActionsRole"
    session_name            = "github_action_session"
    web_identity_token_file = "/tmp/web_identity_token_file"
  }
}

Using the provider(s)
#

Now we have configured the providers, we can start using them in our resource definitions.

For the sake of completeness, these are the files used in the project.

`backend.tf`
#

terraform {
  backend "s3" {
    bucket                  = "terraform-state-bucket-111111111111-eu-west-1"
    key                     = "eu-west-1/terraform-webidentity/terraform.tfstate"
    region                  = "eu-west-1"
    encrypt                 = true
    dynamodb_table          = "terraform-state-lock-table-111111111111"
    profile                 = "backend"
    shared_credentials_file = "~/.aws/credentials"
  }
}

`providers.tf`
#

# We're referring to variables for the account IDs and the GitHub role name to be used
provider "aws" {
  # Default profile set to 'staging' account
  region  = var.aws_region

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::${var.aws_account_id_staging}:role/${var.github_role_name}"
    session_name            = "github_action_session"
    web_identity_token_file = "/tmp/web_identity_token_file"
  }
}

provider "aws" {
  region  = var.aws_region
  # Similar to the default provider, but this makes it easier to differentiate between providers in the code
  alias   = 'staging'

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::${var.aws_account_id_staging}:role/${var.github_role_name}"
    session_name            = "github_action_session"
    web_identity_token_file = "/tmp/web_identity_token_file"
  }
}

provider "aws" {
  region  = var.aws_region
  alias   = "production"

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::${var.aws_account_id_production}:role/${var.github_role_name}"
    session_name            = "github_action_session"
    web_identity_token_file = "/tmp/web_identity_token_file"
  }
}

`datasources.tf`
#

data "aws_availability_zones" "staging_available" {
  provider = aws.staging
  state    = "available"
}

data "aws_vpc" "default_staging" {
  provider = aws.staging
  default  = true
}

data "aws_availability_zones" "production_available" {
  provider = aws.production
  state    = "available"
}

data "aws_vpc" "default_production" {
  provider = aws.production
  default  = true
}

`main.tf`
#

resource "aws_subnet" "staging_private" {
  provider          = aws.staging
  vpc_id            = data.aws_vpc.default_staging.id
  availability_zone = data.aws_availability_zones.staging_available.names[0]
  cidr_block        = cidrsubnet(data.aws_vpc.default_staging.cidr_block, 8, 1)
  tags = {
    Name = "staging-private-${data.aws_availability_zones.staging_available.names[0]}"
  }
}

resource "aws_subnet" "production_private" {
  provider          = aws
  vpc_id            = data.aws_vpc.default_production.id
  availability_zone = data.aws_availability_zones.production_available.names[0]
  cidr_block        = cidrsubnet(data.aws_vpc.default_production.cidr_block, 8, 1)
  tags = {
    Name = "production-private-${data.aws_availability_zones.production_available.names[0]}"
  }
}

`outputs.tf`
#

output "staging_identity" {
  value = data.aws_caller_identity.staging
}

output "staging_private_subnet_cidr" {
  value = aws_subnet.staging_private.cidr_block
}

output "production_identity" {
  value = data.aws_caller_identity.production
}

output "production_private_subnet_cidr" {
  value = aws_subnet.production_private.cidr_block
}

`variables.tf`
#

variable "aws_account_id_production" {
  type        = string
  description = "Account ID of the production account"
}

variable "aws_account_id_staging" {
  type        = string
  description = "Account ID of the staging account"
}

variable "aws_region" {
  type        = string
  description = "AWS region to use"
}

variable "github_role_name" {
  type        = string
  description = "Name of the GitHub role to use"
}

variable "web_identity_token_file" {
  type    = string
  default = "/tmp/web_identity_token_file"
}

`versions.tf` (not required but best practice)
#

terraform {
  required_version = ">= 1.2.0, < 1.3.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.22.0"
    }

  }
}

`terraform.tfvars` (not best practice to use a name that is automatically used, but for the sake of simplicity)
#

aws_account_id_staging    = "222222222222"
aws_account_id_production = "333333333333"
aws_region                = "eu-west-1"
github_role_name          = "GitHubActionsRole"
web_identity_token_file   = "/tmp/web_identity_token_file"

`credentials.aws` (will be copied to the default location in the workflow)
#

[backend]
region=eu-west-1
role_arn=arn:aws:iam::000000000000:role/GitHubActionsRole
web_identity_token_file=/tmp/web_identity_token_file

NOTE: The two blank lines at the end are intentional and need to be there. When there’s less blank lines, authentication can go wrong and throw an error.

The GitHub Workflow
#

To use all of this in a GitHub Workflow, we need to define the workflow.

The following is an example definition of such a workflow:

`/.github/workflows/terraform.yml`
#

name: "Terraform Actions - WebIdentity"
on:
  workflow_dispatch:
    inputs:
      terraform_version:
        description: "Version of Terraform to use"
        required: true
        default: "1.2.2" # 'latest'
        type: string
  push:
    branches:
      - main
env:
  WEB_IDENTITY_TOKEN_FILE: /tmp/web_identity_token_file
  PLAN_FILE: terraform.plan
jobs:
  terraform:
    name: "Terraform Apply Actions"
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read # This is required for actions/checkout
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Get OIDC Token
        id: aws_sts_creds
        run: |
          curl -s -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=sts.amazonaws.com" | jq -r '.value' > $WEB_IDENTITY_TOKEN_FILE          
      - name: Setup AWS credentials file
        run: |
          mkdir -p ~/.aws
          cp credentials.aws ~/.aws/credentials
          touch ~/.aws/config          
      - name: Check current AWS configuration
        run: |
          aws sts get-caller-identity || echo "[OK] - No AWS credentials found!"
          (env | grep AWS_) || echo "[OK] - No AWS environment variables present!"          
        continue-on-error: true
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: ${{ github.event.inputs.terraform_version }}
      - name: "Terraform Init"
        id: init
        run: terraform init
      - name: "Terraform Validate"
        id: validate
        run: terraform validate
      - name: "Terraform Plan"
        id: plan
        run: terraform plan -out=$PLAN_FILE
      - name: "Terraform Apply"
        id: apply
        run: terraform apply $PLAN_FILE

Using profiles with the web identity token file
#

When using the web identity token file directly in Terraform, you cannot use the same code and execute Terraform-commands both locally and in a GitHub Workflow, because we do not have a GitHub OIDC token locally.

To still be able to run commands locally, and use the web identity token file, we can use profiles, as we already do for the backend.

When configuring profiles, the actual configuration between what you use locally and what’s used in the workflow can be different, as long as the names of the profiles used are the same.

Do note, however, that this introduces a difference between local and workflow deployments, since local profiles might have different privileges set.

A setup using profiles, requires changes to the previous configuration.

Changes to Terraform files
#

`credentials.aws` (will be copied to the default location in the workflow)
#

We add the profiles for the other accounts here as well.

In this example the account IDs and role name(s) are ‘hardcoded’. These could be made variable when using an extra step to produce the credentials file, which builds the file dynamically. This is beyond the scope of this post, though.

[backend]
region=eu-west-1
role_arn=arn:aws:iam::000000000000:role/GitHubActionsRole
web_identity_token_file=/tmp/web_identity_token_file

[staging]
region=eu-west-1
role_arn=arn:aws:iam::111111111111:role/GitHubActionsRole
web_identity_token_file=/tmp/web_identity_token_file

[production]
region=eu-west-1
role_arn=arn:aws:iam::222222222222:role/GitHubActionsRole
web_identity_token_file=/tmp/web_identity_token_file

NOTE: The two blank lines at the end are intentional and need to be there. When there’s less blank lines, authentication can go wrong and throw an error.

`providers.tf`
#

Here we need to reference the profiles we’re using, instead of the assume_role_with_web_identity blocks which reference the web identity token file.

provider "aws" {
  region  = var.aws_region
  # Default profile set to 'staging' account
  profile = "staging"
}

provider "aws" {
  region  = var.aws_region
  # Similar to the default provider, but this makes it easier to differentiate between providers in the code
  alias   = "staging"
  profile = "staging"
}

provider "aws" {
  region  = var.aws_region
  alias   = "production"
  profile = "production"
}

`variables.tf`
#

Here we can leave out the variables for the account IDs.

variable "aws_region" {
  type        = string
  description = "AWS region to use"
}

variable "github_role_name" {
  type        = string
  description = "Name of the GitHub role to use"
}

variable "web_identity_token_file" {
  type    = string
  default = "/tmp/web_identity_token_file"
}

`terraform.tfvars` (not best practice to use a name that is automatically used, but for the sake of simplicity)
#

Here we can leave out the variables for the account IDs.

aws_region                = "eu-west-1"
github_role_name          = "GitHubActionsRole"
web_identity_token_file   = "/tmp/web_identity_token_file"

Changes to the GitHub Workflow
#

All changes are done within the Terraform code. The only change that might be needed in the workflow, is if the credentials file were to be created dynamically.
Since we’re not doing that for this post, no changes to the workflow are needed.

Caveats
#

No AWS environment variables can be set in the workflow; these will override the use of the web identity token file
A credentials file needs to be created in the workflow; this could be done dynamically in the same workflow with, for example, Terraform, or another language
Currently the Terraform S3 backend doesn’t support web identities directly, but requires a profile

Using ‘AssumeRoleWithWebIdentity’ #

Setting up OIDC in an account #

Creating a web identity token file #

Using the web identity token file in Terraform #

Using a web identity for the S3 Backend #

Configuring the AWS provider(s) #

Using the provider(s) #

backend.tf #

providers.tf #

datasources.tf #

main.tf #

outputs.tf #

variables.tf #

versions.tf (not required but best practice) #

terraform.tfvars (not best practice to use a name that is automatically used, but for the sake of simplicity) #

credentials.aws (will be copied to the default location in the workflow) #

The GitHub Workflow #

/.github/workflows/terraform.yml #

Using profiles with the web identity token file #

Changes to Terraform files #

credentials.aws (will be copied to the default location in the workflow) #

providers.tf #

variables.tf #

terraform.tfvars (not best practice to use a name that is automatically used, but for the sake of simplicity) #

Changes to the GitHub Workflow #

Caveats #

Using ‘AssumeRoleWithWebIdentity’
#

Setting up OIDC in an account
#

Creating a web identity token file
#

Using the web identity token file in Terraform
#

Using a web identity for the S3 Backend
#

Configuring the AWS provider(s)
#

Using the provider(s)
#

`backend.tf`
#

`providers.tf`
#

`datasources.tf`
#

`main.tf`
#

`outputs.tf`
#

`variables.tf`
#

`versions.tf` (not required but best practice)
#

`terraform.tfvars` (not best practice to use a name that is automatically used, but for the sake of simplicity)
#

`credentials.aws` (will be copied to the default location in the workflow)
#

The GitHub Workflow
#

`/.github/workflows/terraform.yml`
#

Using profiles with the web identity token file
#

Changes to Terraform files
#

`credentials.aws` (will be copied to the default location in the workflow)
#

`providers.tf`
#

`variables.tf`
#

`terraform.tfvars` (not best practice to use a name that is automatically used, but for the sake of simplicity)
#

Changes to the GitHub Workflow
#

Caveats
#