Table of contents
Terraform with Google Cloud
Introduction to Infrastructure as code
Infrastructure as code (IaC) tools allow us to manage infrastructure with configuration files rather than through a graphical user interface. IaC allows us to build, change, and manage our infrastructure in a safe, consistent, and repeatable way by defining resource configurations that we can version, reuse, and share.
Infrastructure as code (IAC) Tools
- Terraform
- AWS CloudFormation
- Azure Resource Manager
- Google Cloud Deployment Manager
- Oracle Resource Manager
Introduction to Terraform
Terraform is an open-source infrastructure as code (IaC) software tool that was introduced to the market by HashiCorp in 2014 as IaC software that provides a consistent CLI workflow to manage hundreds of cloud services.
Terraform is a state management tool that performs CRUD operations (create, read, update, delete) on managed resources. Terraform codifies cloud APIs into declarative configuration files. It lets us define resources and infrastructure in human-readable, declarative configuration files, and manages our infrastructure's lifecycle.
It includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc. Terraform can manage both existing service providers and custom in-house solutions.
Terraform configuration files are written in HashiCorp Configuration Language (HCL). The HCL syntax is very powerful and allows us to reference variables, attributes, and so on.
Terraform mainly uses Terraform files ending with .tf or .tf.json that hold detailed information about what infrastructure components are required in order to run a single application or our entire data center. Terraform generates an execution plan, which describes what it is going to do to reach the desired state, and then executes it to build the infrastructure described.
Terraform has several advantages over manually managing our infrastructure:
- Terraform can manage infrastructure on multiple cloud platforms.
- The human-readable configuration language helps us write infrastructure code quickly.
- Terraform's state allows us to track resource changes throughout your deployments.
- We can commit our configurations to version control to safely collaborate on infrastructure.
Multi-Cloud Deployment
By using a single region or cloud provider, fault tolerance is limited by the availability of that provider. Multi-cloud deployment allows for more graceful recovery of the loss of a region or entire provider.
Terraform is cloud-agnostic and allows a single configuration to be used to manage multiple providers, and to even handle cross-cloud dependencies. This simplifies management and orchestration, helping operators build large-scale multi-cloud infrastructures.
Terraform can help with multi-cloud environments by having a standard workflow, terraform init, terraform plan, terraform apply, and so on, for all clouds. The infrastructure that Terraform manages can be hosted on public clouds such as AWS, Microsoft Azure, and GCP, or on-premises in private clouds such as VMware vSphere, OpenStack and so on.
Terraform Key components
There are two key components on which Terraformโs working depend: Terraform Core and Terraform Plugins.
Terraform Core
Terraform Core is a compiled binary written in the Go programming language. Terraform Core uses Remote Procedure Calls (RPCs) to communicate with Terraform plugins and offers multiple ways to discover and load plugins for use. The compiled binary is the Terraform CLI.
The responsibilities of Terraform Core are:
- IaC: Reading and interpolating configuration files and modules
- Resource state management
- Resource graph construction
- Plan execution
- Communication with plugins via RPC
Terraform plugins
Terraform plugins are written in the Go programming language and are executable binaries that get invoked by Terraform Core via RPCs. Each plugin exposes an implementation for a specific service, such as AWS/Azure/GCP, or a Provisioner, such as Bash.
All Providers and Provisioners are plugins that are defined in the Terraform configuration file. Both are executed as separate processes and communicate with the main Terraform binary via an RPC interface.
Terraform plugins are responsible for the domain-specific implementation of their type.
The responsibilities of provider plugins:
- Initialization of any included libraries used to make API calls
- Authentication with the infrastructure provider
- The definition of resources that map to specific services
The responsibilities of Provisioner plugins:
- Executing commands or scripts on the designated resource following creation or destruction.
Providers define individual units of infrastructure, for example compute instances or private networks, as resources. We can compose resources from different providers into reusable Terraform configurations called modules, and manage them with a consistent language and workflow.
Terraform's configuration language is declarative, it describes the desired end-state for our infrastructure, in contrast to procedural programming languages that require step-by-step instructions to perform tasks. Terraform providers automatically calculate dependencies between resources to create or destroy them in the correct order.
To deploy infrastructure with Terraform:
- Scope - Identify the infrastructure for our project.
- Author - Write the configuration for your infrastructure.
- Initialize - Install the plugins Terraform needs to manage the infrastructure.
- Plan - Preview the changes Terraform will make to match your configuration.
- Apply - Make the planned changes.
Track our infrastructure
Terraform keeps track of our real infrastructure in a state file, which acts as a source for our environment. Terraform uses the state file to determine the changes to make to our infrastructure so that it will match our configuration.
Collaborate
Terraform allows us to collaborate on our infrastructure with its remote state backends. When we use Terraform Cloud (free for up to five users), we can securely share your state with your teammates, provide a stable environment for Terraform to run in, and prevent race conditions when multiple people make configuration changes at once.
We can also connect Terraform Cloud to version control systems (VCSs) like GitHub, GitLab, and others, allowing it to automatically propose infrastructure changes when we commit configuration changes to VCS. This lets us manage changes to our infrastructure through version control, as we would with application code.
Terraform with GCP
Activate Cloud Shell in Google Cloud
Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.
In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.
Click Continue.
It will take a few moments to provision and connect to the environment. When we are connected, we are already authenticated, and the project is set to our PROJECT_ID.
We can list the active account name with this command
gcloud auth list
We can list the project ID with this command:
gcloud config list project
Verifying Terraform installation
Terraform comes pre-installed in Cloud Shell.
- Open a new Cloud Shell tab, and verify that Terraform is available:
Terraform
The resulting help output should be similar to this:
Usage: terraform [--version] [--help] [args]
The available commands for execution are listed below.
The most common, useful commands are shown first, followed by less common or more advanced commands. If you're just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage.
Common commands:
apply Builds or changes infrastructure
console Interactive console for Terraform interpolations
destroy Destroy Terraform-managed infrastructure
env Workspace management
fmt Rewrites config files to canonical format
get Download and install modules for the configuration
graph Create a visual graph of Terraform resources
import Import existing infrastructure into Terraform
init Initialize a Terraform working directory
output Read an output from a state file
plan Generate and show an execution plan
providers Prints a tree of the providers used in the configuration
push Upload this Terraform module to Atlas to run
refresh Update local state file against real resources
show Inspect Terraform state or plan
taint Manually mark a resource for recreation
untaint Manually unmark a resource as tainted
validate Validates the Terraform files
version Prints the Terraform version
workspace Workspace management
All other commands:
debug Debug output management (experimental)
force-unlock Manually unlock the terraform state
state Advanced state management
Build infrastructure
With Terraform installed, we can start creating some infrastructure.
Configuration
The set of files used to describe infrastructure in Terraform is simply known as a Terraform configuration.
For example, write the first configuration to launch a single VM instance.
Step 1: In Cloud Shell, create an empty configuration file named instance.tf with the following command:
touch instance.tf
Step 2: Click Open Editor on the Cloud Shell toolbar. To switch between Cloud Shell and the code editor, click Open Editor or Open Terminal as required, or click Open in a new window to leave the Editor open in a separate tab.
Step 3: Click the instance.tf file and add the following content in it, replacing with your Google Cloud project ID:
resource "google_compute_instance" "terraform" {
project = "" name = "terraform" machine_type = "n1-standard-1" zone = "us-central1-a" boot_disk { initialize_params { image = "debian-cloud/debian-9" } } network_interface { network = "default" access_config { } } }
This is a complete configuration that Terraform is ready to apply. The general structure should be intuitive and straightforward.
The "resource" block in the instance.tf file defines a resource that exists within the infrastructure. A resource might be a physical component such as an VM instance. The resource block has two strings before opening the block: the resource type and the resource name. For this lab, the resource type is google_compute_instance and the name is terraform. The prefix of the type maps to the provider: google_compute_instance automatically tells Terraform that it is managed by the Google provider.
Within the resource block itself is the configuration needed for the resource.
Step 4: In Cloud Shell, verify that your new file has been added and that there are no other *.tf files in your directory, because Terraform loads all of them:
ls
Initialization
The first command to run for a new configuration or after checking out an existing configuration from version control is terraform init. This will initialize various local settings and data that will be used by subsequent commands.
Terraform uses a plugin-based architecture to support the numerous infrastructure and service providers available. Each "provider" is its own encapsulated binary that is distributed separately from Terraform itself. The terraform init command will automatically download and install any provider binary for the providers to use within the configuration, which in this case is just the Google provider.
Step 1: Download and install the provider binary:
terraform init
The Google provider plugin will be downloaded and installed in a subdirectory of the current working directory, along with various other book keeping files. We can see an "Initializing provider plugins" message. Terraform knows that you're running from a Google project, and it is getting Google resources.
our version number may be higher.
Step 2: Create an execution plan:
terraform plan
Terraform performs a refresh, unless explicitly disabled, and then determines what actions are necessary to achieve the desired state specified in the configuration files. This command is a convenient way to check whether the execution plan for a set of changes matches our expectations without making any changes to real resources or to the state.
Note The optional -out argument can be used to save the generated plan to a file for later execution with terraform apply.
Apply changes
Step 1: In the same directory as the instance.tf file we created, we have to run the command:
terraform apply
This output shows the Execution Plan, which describes the actions Terraform will take in order to change real infrastructure to match the configuration. The output format is similar to the diff format generated by tools like Git.
There is a + next to google_compute_instance.terraform, which means that Terraform will create this resource. Following that are the attributes that will be set. When the value displayed is , it means that the value won't be known until the resource is created.
For example, output:
If the plan was created successfully, terraform will now pause and wait for approval before proceeding. In a production environment, if anything in the Execution Plan seems incorrect or dangerous, it's safe to cancel here. No changes have been made to your infrastructure.
Step 2: For this case the plan looks acceptable, so we have type yes at the confirmation prompt to proceed. Executing the plan will take a few minutes because Terraform waits for the VM instance to become available.
Test the VM Instance
Step 3: In the Google Cloud Console, on the Navigation menu, click Compute Engine > VM instances to see the created VM instance.
Terraform will be written some data into the terraform.tfstate file. This state file is extremely important: it keeps track of the IDs of created resources so that Terraform knows what it is managing.
Step 4: In Cloud Shell, inspect the current state: terraform show
For Example, output:
These values can be referenced to configure additional resources or outputs.
Community and Social Footprints :
Happy Learning ๐
Thank you!