Terraform-Operator

Terraform Backends

Terraform defines a backend this way.

A "backend" in Terraform determines how state is loaded and how an operation such as apply is executed.

That's nice... but what the heck does that mean? I think the above definition by Terraform is really referring to backend in the context of using Terraform Cloud since it's the only backend that handles remote execution.

A simpler way to think of a backend is "where Terraform stores state." Without any configuration, a Terraform module will save state on the local system running Terraform. However, since Terraform-Operator runs on pods which are ephemeral by nature, state must be stored elsewhere since data written to the pod's filesystem is lost when the pod terminates.

Terraform-Operator's Default Backend

Terraform-Operator uses Consul, a HashiCorp key/value store, as the default backend for any Terraform run. It does so by placing a backend_override.tf file in the module which points to the Consul service running on your cluster.

# backend_override.tf
terraform {
  backend "consul" {
    address = "hashicorp-consul-server.tf-system:8500"
    scheme  = "http"
    path    = "terraform/${NAMESPACE}/${RESOURCE_NAME}.tfstate"
  }
}

Where:

• RESOURCE_NAME is the name of the Kubernetes resource for your Terraform config.

See Terraform State with Consul from the first blog in this series.

Using a Different Backend

Terraform-Operator takes a rather simplistic approach to backends; either completely define one or completely don't. Let's do a simple exercise to see how using a different backend is accomplished.

Start by defining the Terraform-Operator resource. Let's name it hello-terraform-operator.yaml. Take close attention to the customBackend value in the spec. What you'll notice is a multi-line YAML string of a fully defined Terraform backend.

# hello-terraform-operator.yaml
---
apiVersion: tf.isaaguilar.com/v1alpha1
kind: Terraform
metadata:
  name: hello-terraform-operator
spec:

  terraformVersion: 0.12.23
  terraformModule:
    address: https://github.com/cloudposse/terraform-aws-test-module.git

  credentials:
  - secretNameRef:
      name: aws-session-credentials

  customBackend: |-
    terraform {
      backend "s3" {
        key            = "terraform-operator/my/awesome/test.tfstate"
        region         = "us-east-1"
        bucket         = "isaaguilar-terraform"
        dynamodb_table = "isaaguilar-terraform-lock"
      }
    }

  applyOnCreate: true
  applyOnUpdate: true
  applyOnDelete: true

In this example, I've got the remote state going to an AWS S3 bucket. The keyword in that statement being "AWS," because as with any cloud provider, credentials are required for access to AWS (or any providers') resources.

In the last post I showed how to create a secret with credentials, and then reference those credentials using credentials[0].secretNameRef.name in the resource spec. If that method is OK with you, skip to the Terraform run. For the adventurous type, I'd like to introduce you to:

The Kubectl AWS Session Plugin

The creatively named kubectl-aws-session-plugin is a kubectl plugin to uses AWS's Default Credential Provider Chain to get credentials and push them into a Kubernetes secret. Maybe it's overkill to do such a simple task, who's to judge? All I know is that it takes my current AWS session and dumps it into a Kubernetes Secret called "aws-session-credentials". And it has a native kubectl feel to it!
Install it by running:

Then run kubectl aws session -n default to see the Secret get created (or updated if it already existed).

secret aws-session-credentials created successfully

Luckily for us, the Secret that got created is named "aws-session-credentials", which just so happens to be the same value as credentials[0].secretNameRef.name in the resource spec. What a happy coincidence. 😊 Now we're ready to run the Terraform

Running Terraform

Simply apply the hello-terraform-operator.yaml to install your new resource.

terraform.tf.isaaguilar.com/hello-terraform-operator created

After a small while you'll see your pod

NAME                             READY   STATUS      RESTARTS   AGE
hello-terraform-operator-f5tw9   1/1     Running     0          7s

Run kubectl logs -f hello-terraform-operator-f5tw9 to see the Terraform run logs. Once the pod completes, a ConfigMap with the Terraform output content is saved. Let's check out what we've got:

{
  "example": "hello world 2401",
  "id": "",
  "random": "2401"
}

Finally, let's see what the state file looks like. Since I used AWS S3 as the backend for this example, I can simply go to the AWS Console and click my way to find the state file. But just for kicks, I'll try a neat trick and use an S3 pre-signed URL. This is a way to grant public access for a small duration of time to a an S3 object using an HTTP GET request.

{
  "version": 4,
  "terraform_version": "0.12.23",
  "serial": 2,
  "lineage": "46da0cb2-bac6-be2f-334f-fac25be96bec",
  "outputs": {
    "example": {
      "value": "hello world 2401",
      "type": "string"
    },
    "id": {
      "value": "",
      "type": "string"
    },
    "random": {
      "value": "2401",
      "type": "string"
    }
  },
  ...

Cleanup

To clean up the environment, run kubectl delete on the Terraform resource. This will trigger a new pod, this time running terraform destroy in the Terraform run.

terraform.tf.isaaguilar.com "hello-terraform-operator" deleted

All done. If you want to see that there are no more resources in the S3 state file, you can re-use the pre-signed URL and run curl "$PSURL".

Summary

Terraform-Operator provides an organized structure to define your infrastructure-as-code with a first-class Kubernetes experience. Using a different backend to store Terraform state is as easy as defining the backend in the resource manifest. Or leave backend definitions alone and let Terrform-Operator organize your tfstate in Consul automatically. To learn more check out the official docs at https://github.com/isaaguilar/terraform-operator/tree/master/docs.

I would love to hear your feedback and expand on the project!

Send bugs, questions, feature requests for the Terraform-Operator to isaaguilar/terraform-operator by posting a new issue!