How to easily manage and provision your infrastructure with Terraform

Codifying application infrastructure makes it easier to increase automation, create reproducible infrastructure and reduce human error.

In this article, we will look at how to manage and provision infrastructure using Terraform, an infrastructure provisioning tool from Hashicorp. To follow along, check that you have the prerequisites below.

Prerequisites

LEVEL - πŸ’» Intermediate - Knowledge of Docker, Docker Compose

Key terms & Concepts

  • Terraform - Tool for building and managing infrastructure in an organised way. You can use it to manage infrastructure from a number of cloud providers including Google, AWS and DigitalOcean.
  • Terraform terms - Terraform uses 5 steps to deploy infrastructure.
    • Scope - Identify the infrastructure of the project
    • Author - Write the config for your infrastructure
    • Initialize - Install the plugins required to manage the infrastructure
    • Plan - Preview changes that will be made to match your config
    • Apply - Make planned changes

Identifying the scope - We will deploy a small Go web app to a DigitalOcean droplet. We will use Terraform to provision our infrastructure.

Create personal access token on DigitalOcean

You will need a DigitalOcean access token to provision resources using Terraform. If you do not already have one, log in to your DigitalOcean account, find your applications and API page and generate a new token.

Click on 'Generate New Token'.

Terraform will use the access token to communicate with the DigitalOcean API. To enable this you will have to export your DO token to an environment variable named DO_TOKEN in the terminal where you will be running your terraform commands.

$ export DO_TOKEN="Your token here"

Set up the project and configure terraform

Our finished project folder structure will look like this. It is good practice to set up a basic folder structure at the beginning of your project. Go ahead and create a new project with this structure.

β”œβ”€proj # Main project folder
β”œβ”€β”€ ops
|     └─ terraform 
β”‚        β”œβ”€β”€ provider.tf
β”‚        β”œβ”€β”€ server.tf
β”‚        β”œβ”€β”€ .tfvars
β”œβ”€β”€ .gitignore
└── docker-compose.yml

Setup a terraform provider for digital ocean .

Author - In provider.tf add the following.

terraform {
  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
      version = "~> 2.0"
    }
  }
}


variable "do_token" {}
variable "pvt_key" {}

provider "digitalocean" {
  token = var.do_token
}

data "digitalocean_ssh_key" "terraform" {
  name = "terraform"
}

Notes

  • required_providers - tells Terraform which provider to use
  • variable "do_token" & variable "pvt_key" - set variables for "do_token" and "pvt_ket"
  • data "digitalocean_ssh_key" "terraform" {..} - instruct Terraform to add your SSH key to the droplet

Author - Create a file to store your variables in .tfvars

do_token ="YOUR DO TOKEN HERE" -> Add key from 
pvt_key="path-to-your-private-key/id_rsa" # -> 

Notes

  • pvt_key refers to the path of the private key file you created earlier.

The .tfvars file includes sensitive information. It should not be committed to a repository.

Update the .gitignore file to exclude all the variables.

# terraform
# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log

# Exclude all .tfvars files, which are likely to contain sentitive data, 
# password, private keys, and other secrets. These should not be part of 
# control as they are data points which are potentially sensitive and 
# subject to change depending on the environment.

*.tfvars

# Ignore override files as they are usually used to override resources 
# locally and so are not checked in
override.tf
override.tf.json
*_override.tf
*_override.tf.json

# Ignore CLI configuration files
.terraformrc
.idea/*
terraform.rc

Initialize - Install the plugins defined above by running Terraform

$ cd ops/terraform
$ terraform init


# you should see the following output.
Initializing the backend...

Initializing provider plugins...
- Finding digitalocean/digitalocean versions matching "~> 2.0"...
- Installing digitalocean/digitalocean v2.10.1...

Define the resources

Configure a 'resource' i.e. a DigitalOcean droplet to host our server.

server.tf

resource "digitalocean_droplet" "goapp" {
  image = "docker-20-04"
  name = "pypiserver"
  region = "ams3"
  size = "s-1vcpu-1gb"
  private_networking = true
  ssh_keys = [
    data.digitalocean_ssh_key.terraform.id
  ]
  connection {
    host = self.ipv4_address
    user = "root"
    type = "ssh"
    private_key = file(var.pvt_key)
    timeout = "2m"
  }
  provisioner "remote-exec" {
    inline = [
      "export PATH=$PATH:/usr/bin",
      "sudo apt-get update",
      "sudo apt install git",
      
      "git clone https://github.com/advantch/go-web-app",
      "docker-compose up -d --build",

      "cd go-web-app && docker-compose up -d --build", 
      "docker-container ls",

      "sudo ufw allow 8080",
      "curl ifconfig.me"
    ]
  }
}

Notes

  • "digitalocean_droplet" "goapp" {..} - instruct terraform to provide a digitalocean_droplet resource name "goapp".
  • image= "docker-20-04", region="ams3" - selects the correct image and region.
  • ssh-keys - SSH keys are retrieved from the data source that we defined in provider.tf. Data sources allow Terraform to use the information defined outside Terraform, defined by another separate config or modified by functions
  • provisioner "remote-exec" - Provisioners model specific actions on the machine. In this case, we are using the provisioner to pull the repository with our file and startup the docker container. See this link and note below on using provisioners.
  • git clone https://github.com/advantch/go-web-app - We are using the demo go app from Advantch as an example. You can replace this with your own private repo.
  • sudo ufw allow 8080 curl ifconfig.me - we added two commands here, one to expose the port to the outside world and another to show you the IP of your server.

In the root of your directory, add the docker-compose file for the go app. You can also replace this with your own docker-compose file.

docker-compose.yml

version: '3.7'

services:
  goapp:
    image: thembahank/go-web-app:latest
    restart: always
    ports:
      - "8080:8080"

Preview the updated changes

Preview the updated changes by running terraform plan.


$ cd ops/terraform 
$ terraform plan -var-file='.tfvars'

# Expected output

Terraform used the selected providers to generate the following execution 
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # digitalocean_droplet.goapp will be created
  + resource "digitalocean_droplet" "goapp" {
      + backups              = false
...
  • terraform plan -var-file='.tfvars' - Notice that we use the -var-file flag to refer to the variables stored in .tfvars file we added earlier.

Provision your resources

Finally lets provision the resources.

$ cd ops/terraform
$ terraform apply -var-file='.tfvars'

# Output
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.
   
  > Enter a value: yes # respond 'yes' when prompted.

# Output
 Enter a value: yes

digitalocean_droplet.goapp: Creating...
digitalocean_droplet.goapp: Still creating... [10s elapsed]
digitalocean_droplet.goapp: Still creating... [20s elapsed]

digitalocean_droplet.goapp (remote-exec): Rule added
digitalocean_droplet.goapp (remote-exec): Rule added (v6)
digitalocean_droplet.goapp (remote-exec): 206.189.10.53
digitalocean_droplet.goapp: Creation complete after 2m9s [id=255288750]

If everything has gone according to plan, you should see this page when you visit your IP.

Hope your tetris skills are better than mine!

Don't forget to remove your resources when you are done.

$ cd ops/terraform && terraform destroy -var-file='.tfvars'

Conclusion

In this article, we have explored how to use Terraform to provision infrastructure. In future, we will be looking at improving how we manage secrets and how to incorporate Terraform into our DevOps pipeline.

Further reading & Acknowledgements

Terraform Get Started tutorials - Best place to get started with Terraform.

Terraform Provider Registry docs - Docs on the different providers, includes a list of available resources and data sources for each provider.

Copyright Β© 2022 www.advantch.com