State file location

Developing solo with a local state file is probably fine so long as you have backups of your local system, for example. But if you have n+1 people who need to use the same Terraform environment, you must elevate the state file location to something more sophisticated.

Terraform has a backend feature that by default is configured as a local backend and not shown in the Terraform configuration files. To change the state storage to a remote backend you add configuration data to your Terraform configuration files defining where and how the state is to be stored. In most organizations AWS S3, Azure Blob, or Terraform Cloud will be the likely solutions used by DevOps teams as these solutions solve the following problems:

Automated load Terraform handles loading and writing back the state file every time you run Terraform which eliminates the need for the operator to update their copy of the state file.

Locks Are handled by the storage solution so two operators can never change the same state file at once. If the state file is locked already then the 2nd operator will receive an error stating the fact.

Secrets Data is transported and stored in an encrypted state and access to the backend system can be better handled by IAM roles and other systems much better than in a VCS system.

Availability AWS S3 and other similar solutions are a managed service with built-in durability and availability meanng your important state file is quite secure

Versioning By adding versioning to the state file you can in effect take advantage of rolling back to previous configurations if an update creates problems

Don’t store state in the VCS

State files should not be saved to GitHub (or whichever VCS you use) as mistakes will happen. For example, you forget to pull down the state’s latest version and run a Terraform commmand. Suddenly you’re essentially rolling back the deployed infrastructure or making unintended changes. Most VCS systems do not support locks meaning two people could change the same state file leading to corruption. All secrets are written to state files in plain text, breaking a fundamental law of; DON’T SAVE SENSITIVE DATA IN A VCS.

Using AWS S3 as a backend

The following Terraform code examples will create an AWS S3 bucket specifically to support Terraform state and take into account the considerations explained above.

Typically you have a single S3 bucket to store multiple state files on a per AWS account or Azure subscrption basis as this keeps things relativley simple. When you create the new AWS account you’ll need to create the S3 bucket so you can store your VPC and other account wide Terraform state files. But there is a chicken before egg situation as you need to create the S3 bucket in Terraform and you want to store your state in an S3 bucket. So to store your state in S3 you need to go through two stages.

First, create the S3 bucket with a local backend defined in your Terraform configuration file that creates the S3 bucket. This will store the state file in the Terraform working directory.

Then you configure the remote backend in the same Terraform configuration file and define the new S3 bucket as the target. When you run terraform init it will ask you if you want to store the state in the newly described remote backend instead of locally and when you answer yes the state is then moved to the S3 bucket.

• S3 bucket
• Configuration - local
• Configuration - remote

resource "aws_s3_bucket" "terraform_state" {
  bucket = "terraform-state-grinntec-learn"

  # Prevent accidential deletion of this bucket
  lifecycle {
    prevent_destroy = true
  }
}

// Enabled bucket versioning
# Each update creates a new version of the state file
resource "aws_s3_bucket_versioning" "enabled" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

// Turn-on server side encryption
# Ensures data stored on S3 disk is encrypted
resource "aws_s3_bucket_server_side_encryption_configuration" "enabled" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

// Block all public access to the S3 bucket
# Prevents making the S3 bucket publicly accessible
resource "aws_s3_bucket_public_access_block" "public_access" {
  bucket                  = aws_s3_bucket.terraform_state.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

// Create a DynamoDB table for locking
# Key:vaue store
# Strongly consitent reads and conditional writes
# Primary key must be called 'LockID'
resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

provider "aws" {
  region = "us-east-2"
}

provider "aws" {
  region = "us-east-2"
}

// This section tell Terraform to store the state file remotley
# When you create the S3 bucket this section should be hashed out (chicken and egg)
# Once the S3 bucket exists then uncomment this block and run
#       terraform init
# Terraform will ask if you want to transfer the local state to the remote state
terraform {
  backend "s3" {
    bucket  = "terraform-state-grinntec-learn"
    key     = "global/s3/terraform.tfstate"
    region  = "us-east-2"

    dynamodb_table  = "terraform-locks"
    encrypt         = true
  }
}