Introduction

I am currently preparing to recertify for the Google Professional Cloud Security Engineer Certification. I previously scheduled the HashiCorp Certified: Terraform Associate on March 29, 2021 at 3 PM. Maybe I will take both exams on the same day.

Date created: March 5, 2021
Last updated: March 7, 2021

This article is not complete. I am updating this article often as I prepare for the certification exams.

I have been using Terraform for about five years. There are areas I am weak with, and I feel I consult the documentation too often. In this article, I am combining Terraform, Google Cloud DNS, and IAM into a learning article. I just wrote Terraform code (link) to enable DNSSEC for delegated zones, so I thought I would dig deeper into Terraform and the DNS resources. I want to learn more about the error messages that Terraform displays when the credentials (service account) that Terraform is using do not have the correct roles assigned to them.

This article will start with a new service account that has no roles assigned. I will start with very basic Terraform code using the Google Provider for DNS and continue to more advanced code. Each time I see an error message, I will record it and then document the required IAM roles to solve the error.

For this article, I assume that you know how to create a service account, download the JSON key file, and add Google Cloud IAM roles to a service account. I am using Windows 10, Terraform version 0.14.7, Google Provider 3.58.0, and Google Cloud CLI (gcloud) version 330.0.0.

The CLI is authorized with my user account that has the role Editor. I will use the CLI to create the service account, download the JSON file, and then add/remove IAM roles as required. You can follow along using the CLI or the Google Cloud Console GUI.

Getting Started

Terraform uses a directory structure for projects. For each example in this article, create a new directory.

This Windows batch script deletes the Terraform project files except for the configuration files (*.tf). I use this to start over when developing Terraform projects. Normally you should use terraform destroy but sometimes you make mistakes and need to start over. In those cases, I delete the Google Cloud Resources in the GUI and then run this script to clean up the Terraform project.

Create a file tclean.bat and copy the following contents:

Download from GitHub.

IF exist .terraform (rd /s /q .terraform)
IF exist terraform.tfstate.d (rd /s /q terraform.tfstate.d)
IF exist .terraform.lock.hcl (del .terraform.lock.hcl)
IF exist terraform.tfstate (del terraform.tfstate)
IF exist terraform.tfstate.backup (del terraform.tfstate.backup)

Terraform Authorization

Terraform uses two types of authorization: Implicit and Explicit.

Implicit Authorization means referencing credentials in the environment. Explicit Authorization means specifying the service account JSON file in the Terraform configuration file in the Provider Resource.

Implicit Authorization means using one of these mechanisms:

  • Setting up the environment variable `set GOOGLE_APPLICATION_CREDENTIALS=fullpath.json`
  • Setting up authorization using the CLI command gcloud auth application-default-login. This method requires User Credentials (Gmail, Workspace, etc.) for authorization and does not support service accounts.
  • Fetching credentials from a Google compute service using the Google Metadata Server.

If explicit authorization is not used, Terraform will search the environment in the order listed above to find the authorization credentials to use. This also means if the environment variable is set (method #1), credentials setup using the CLI (method #2) is not used.

Implicit Authorization is also referred to as Application Default Credentials (ADC) in Google Cloud Platform.

In this article, I will use Explicit Authorization. The service account JSON file is specified as an argument in the Provider Resource:

variable "gcp_service_account" {
  description = "Service Account for Authorization"
  default = "c:/config/terraform-dns-test.json"
}
provider "google" {
  credentials = var.gcp_service_account
}

Terraform Commands

I use the following commands to develop Terraform Projects. Read the documentation to ensure that you understand each command.

  • terraform init
  • terraform validate
  • terraform plan
  • terraform apply
  • terraform destroy

Create the Terraform credentials

Step 1. Create the service account named terraform-dns-test.

gcloud iam service-accounts create terraform-dns-test ^
--description "Test service account for Cloud DNS and Terraform" ^
--display-name "terraform-dns-test

Step 2. List the service accounts

gcloud iam service-accounts list

Copy the email address for the new service account. The email address will look like this: terraform-dns-test@PROJECT_ID.iam.gserviceaccount.com. The email address is required for the next command.

Step 3. Create a service account key and download.

gcloud iam service-accounts keys create terraform-dns-test.json ^
--iam-account=terraform-dns-test@development-999999.iam.gserviceaccount.com

The service account JSON key is now stored in the file terraform-dns-test.json. I have a secure directory c:\config where I store my credentials. Move the JSON file to a secure location on your computer.

Create a Google Cloud DNS Managed-Zone

The zone name does not matter as this zone will not be published by a Domain Registrar. I do not recommend practicing with a real domain name at this point. For this zone, I will use myexample.com. Google Cloud does not allow some popular domains such as example.com.

gcloud dns managed-zones create myexample-com ^
--description="Managed Zone for Terraform testing" ^
--dns-name="myexample.com"

Terraform Data Sources

Data sources allow data to be fetched or computed for use elsewhere in Terraform configuration. Use of data sources allows a Terraform configuration to make use of information defined outside of Terraform, or defined by another separate Terraform configuration. [text source]

In other words to access an existing Google Cloud resource such as a Cloud DNS Managed Zone, use a Terraform Data Source. For Google Cloud, use google_dns_managed_zone.

This data source is used like this:

data "google_dns_managed_zone" "dns_zone" {
  name = "myexample-com"
}

Reference: google_dns_managed_zone

To display all the managed zone attributes:

output "gcp_dns_zone" {
  value = data.google_dns_managed_zone.dns_zone
}

Which outputs attributes in this format:

gcp_dns_zone = {
  "description" = "Managed Zone for Terraform testing"
  "dns_name" = "myexample.com."
  "id" = "projects/development-999999/managedZones/myexample-com"
  "name" = "myexample-com"
  "name_servers" = tolist([
    "ns-cloud-a1.googledomains.com.",
    "ns-cloud-a2.googledomains.com.",
    "ns-cloud-a3.googledomains.com.",
    "ns-cloud-a4.googledomains.com.",
  ])
  "project" = "development-999999"
  "visibility" = "public"
}

Example 1. Output the Terraform attributes for the Google Cloud DNS managed zone.

Create the file main.tf and copy the following contents:

Download from GitHub.

######################################################################
# Terraform
######################################################################

terraform {
  required_version = ">= 0.14.7"
}

######################################################################
# Google Cloud Provider
######################################################################

provider "google" {
  credentials = var.gcp_service_account
  project     = var.gcp_project
}

######################################################################
# variables for Google Cloud Provider
######################################################################

variable "gcp_project" {
  description = "Google Cloud Project ID"
  default = "development-999999"
}

variable "gcp_service_account" {
  description = "Service Account for Authorization"
  default = "c:/config/terraform-dns-test.json"
}

######################################################################
# variables for the Google Cloud DNS Managed Zone
######################################################################

variable "zonename" {
  description = "Google Cloud DNS Zone Name (not the domain name)"
  default = "myexample-com"
}

######################################################################
# access the Cloud DNS Zone attributes
######################################################################

data "google_dns_managed_zone" "dns_zone" {
  name = var.zonename
}

######################################################################
#
######################################################################

output "gcp_dns_zone" {
  value = data.google_dns_managed_zone.dns_zone
}

Update the variables section with values appropriate for your project:

  • gcp_project
  • gcp_service_acount
  • zonename

Initialize the Terraform project and validate the configuration files (code):

terraform init
terraform validate
terraform plan

The command terraform validate should succeed. If the command fails, find and correct the syntax/spelling error in main.tf.

The command terraform plan should fail with the message:

Error: googleapi: Error 403: Forbidden, forbidden

The reason for the command failure is the service account’s permissions. The service account does not have a role assigned to the project granting this service account rights to access Cloud DNS. This will now be corrected. The documentation for Google Cloud DNS Access Control lists the permissions for Cloud DNS. The Terraform configure file is not modifying Cloud DNS. This means only get/list/read types of permissions are required. Within the section on Roles, we find the role roles/dns.reader (DNS Reader). Grant that role to the service account.

Modify the Project ID (development-999999) and service account email address to match your project.

gcloud projects add-iam-policy-binding development-999999 ^
--member serviceAccount:terraform-dns-test@development-999999.iam.gserviceaccount.com ^
--role roles/dns.reader

Run terraform plan again and it should work. Sometimes you must wait for a couple of minutes for role assignments to complete.

Once the commands terraform validate and terraform plan are successful, execute the configuration. Nothing will be changed in the Google Cloud Project as this configuration only implements Terraform Data Sources which are read-only.

terraform apply

The output will look similar to this:

gcp_dns_zone = {
  "description" = "Managed Zone for Terraform testing"
  "dns_name" = "myexample.com."
  "id" = "projects/development-999999/managedZones/myexample-com"
  "name" = "myexample-com"
  "name_servers" = tolist([
    "ns-cloud-a1.googledomains.com.",
    "ns-cloud-a2.googledomains.com.",
    "ns-cloud-a3.googledomains.com.",
    "ns-cloud-a4.googledomains.com.",
  ])
  "project" = "development-999999"
  "visibility" = "public"
}

I notice that three attributes are not part of the Attributes Reference:

  • id
  • name
  • project

If you misspell the zone name argument:

Error: googleapi: Error 404: The 'parameters.managedZone' resource named 'xmyexample-com' does not exist., notFound

Example 1 Summary

In this example:

  • Created a Terraform project accessing a Google Cloud DNS Managed Zone.
  • Analyzed a permissions error.
  • Added the required Google Cloud IAM role.
  • Executed the Terraform project and output the managed zone attributes.

The above code is useful when your goal is Terraform code to add DNS resource records to an existing zone. For example, automating DNS setup for new Compute Engine instances.

Example 2. Creating a custom role

Often, users and service accounts are granted broad powerful permissions because a solid understanding of IAM roles and permissions is missing. When possible, the Principle of least privilege should be employed. This means using the minimum set of privileges required to perform a task or job function. This example repeats example 1, but this time with minimum permissions.

Roles can be created at the project level or the organization level. Custom role names include information about the project or organization.

Modify the Project ID (development-999999) and service account email address to match your project.

In this example:

  • Remove the roles/dns.reader from the service account. The service account now has no roles or permissions.
  • Create a custom role with one permission: dns.managedZones.get.
  • Assign the custom role to the service account.
  • Verify that the Terraform code can still output the managed zone attributes.

Step 1. Remove the role roles/dns.reader.

gcloud projects remove-iam-policy-binding development-999999 ^
--member serviceAccount:terraform-dns-test@development-999999.iam.gserviceaccount.com ^
--role roles/dns.reader

Step 2. Create a custom role with required permissions.

gcloud iam roles create TestTerraformDnsRole ^
--project development-999999 ^
--description "Test role for Terraform and DNS examples" ^
--permissions dns.managedZones.get ^
--title TestTerraformDnsRole

In this command’s output, note the custom role name. You need that for the next step.

Example output:

Created role [TestTerraformDnsRole].
description: Test role for Terraform and DNS examples
etag: BwW8_5BSPCY=
includedPermissions:
- dns.managedZones.get
name: projects/development-999999/roles/TestTerraformDnsRole
stage: ALPHA
title: TestTerraformDnsRole

Step 3. Grant the custom role to the service account.

When using custom roles, notice that the name format is different.

gcloud projects add-iam-policy-binding development-999999 ^
--member serviceAccount:terraform-dns-test@development-999999.iam.gserviceaccount.com ^
--role projects/development-999999/roles/TestTerraformDnsRole

Step 4. Repeat the Terraform commands using the same main.tf and variables.

terraform init
terraform validate
terraform plan
terraform apply

Example 2 Summary

This was a simple example to show how to create a custom role and then use it with a service account and Terraform.

[To be continued]

Photography Credits

I write free articles about technology. Recently, I learned about Pexels.com which provides free images. The image in this article is courtesy of Pixabay at Pexels.