Best Practices To Promote From DEV To PROD Environments With…

When using HashiCorp Terraform to manage infrastructure as code (IaC), promoting changes from the development (DEV) environment to the production (PROD) environment is critical to the process of managing the development lifecycle of the infrastructure for a system. Most often, organizations will also have test (TEST) or staging (STAGE) environments for additional testing to move the configuration through before deployment to production. This is done to ensure that the infrastructure changes are fully tested and reliable before any changes are made to any production systems.

In this article, we’ll look at the two most common methods for managing the process of promoting deployments from DEV into the PROD environment. This is a tale of two method. First we’ll look at using Terraform Workspaces, then well look at using “Environment Folders” within the Terraform project. While Terraform Workspaces is the preferred method for managing multiple environments with the same Terraform project, Environment Folders is still a valid option that may make sense based on the unique requirements of the team’s workflow and the complexity of the project.

Using Terraform Workspaces

Terraform workspaces are a feature within Terraform that allows you to manage multiple distinct states within the same Terraform configuration. This is particularly useful when you want to apply the same infrastructure setup across different environments, such as development (DEV), testing (TEST), staging (STAGE), and production (PROD) environments, without having to duplicate the code.

Each workspace has its own state file, so the resources managed by each workspace are completely separate from each other, even though they are defined by the same configuration files. This separation is crucial for preventing conflicts and ensuring that actions taken in one workspace do not affect resources in another.

Workspaces are often used to manage environment-specific configurations with minimal changes to the actual Terraform code. For example, you might have different variables for resource sizing, names, or service configurations that are specific to the environment you are targeting. By switching between workspaces, you can apply these different configurations without altering the underlying codebase.

To summarize, Terraform workspaces allow you to:

• Maintain separate state files for different environments within the same codebase.
• Promote code between environments by switching workspaces rather than copying files.
• Customize variables and resource configurations per environment without changing the core Terraform configuration.

It’s important to note that while workspaces can be very useful, they are not always the best solution for every scenario. In some cases, especially when environments have vastly different configurations, it might be better to use completely separate Terraform configurations or modules.

How to Use Terraform Workspaces

Terraform workspaces allow you to manage multiple distinct sets of Terraform state within the same configuration. This method is useful for deploying the same infrastructure with different parameters.

To use workspaces, you would typically follow these steps:

1. Initialize your Terraform configuration if you haven’t already:

terraform init

2. Create a new workspace for each environment:

terraform workspace new dev
terraform workspace new prod

3. Switch between workspaces when you need to apply changes to a different environment:

terraform workspace select dev

4. Apply your changes within the selected workspace:

terraform apply

Managing Variables with Terraform Workspaces

Environment configurations will vary per environment deployment. When using workspaces, it’s convenient that each workspace gets it’s own state file for the environment that’s managed with each workspace. However, just running terraform apply by itself, will use the same parameters and configuration for each environment that’s managed. To vary the configuration per environment, you can create a separate variables file (.tfvars) for each environment, then use the appropriate one for the environment when running terraform apply for that environments Terraform workspace.

For example, when using Terraform workspaces to manage both Dev and Prod environments, environment specific variables files should be created:

dev.tfvars
prod.tfvars

Then when selecting the workspace for the specific environment and running terraform apply, the environment specific variables file will be pass to the command, like the following:

terraform workspace select dev
terraform apply -var-file=dev.tfvars

Resource Conditions Based on Terraform Workspaces

When using Terraform workspaces, a feature that’s supported is referencing the workspace name of the Terraform workspace being managed within the configuration code itself. This allows for any environment conditions to be implemented in the configuration directly. This may help implement a “convention over configuration” approach to your IaC deployments using Terraform, rather than having every environment specific change be input parameter driven.

The following is an example of referencing the Terraform workspace name in the configuration to setup conditional deployments, or just to help configure environment specific attributes:

locals {
  instance_size = terraform.workspace == "prod" ? "large" : "small"
}

resource azurerm_resource_group "b59_rg" {
  name = "b59-eus-${terraform.workspace}-rg"
  // additional resource configuration here
}

Best Practices using Terraform Workspaces

1. Workspace-Specific Variables: Use workspace-specific variables within your configuration to differentiate resource properties per environment.
2. Avoid Hardcoding: Do not hardcode workspace names in your configuration. Instead, use the terraform.workspace variable to make decisions based on the current workspace.
3. State Management: Be cautious with state files. Ensure that each workspace’s state is securely stored and backed up.
4. Automation: Integrate workspaces with your Terraform CI/CD pipelines to automate the promotion process.
5. Access Control: Implement access control to restrict who can apply changes to the PROD workspace.
6. Monitoring and Logging: Set up monitoring and logging to track the changes and state of your infrastructure.

Using Environment Folders within the Terraform Project

The environment folders method involves creating a separate folder within your Terraform project for each environment. The folder for each environment contains the configuration files specific to an environment. This approach is straightforward and allows for clear separation of environments.

With each environment in a separate folder, then each environment will essentially have it’s own separate Terraform project. When code and configurations need to be shared between environments, for code reuse and clean coding practices, then Terraform modules can be used. These modules will generally be put into a folder at the root of the folder structure, so they can be shared across each of the environment projects.

Specific configurations for the environments are kept within the environments folder. Then, any shared code, configurations, or common infrastructure that will be managed across multiple environments will usually be configured using a module. Then this module can be shared between the environments, and configured using input parameters on the module to vary the configurations needed per environment.

Structure of Environment Folders

When using environment folders, you would typically have a folder structure that resembles the following:

.
├── dev
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   └── terraform.tfvars
├── prod
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   └── terraform.tfvars
└── modules
    ├── module 1
    │   └──main.tf
    ├── module 2
        └──main.tf

Each environment folder contains its own main.tf, variables.tf, and outputs.tf files. These files define the resources, variables, and outputs for that specific environment. There is also a modules folder that contains subfolders for each of the Terraform modules to share between the environments.

Best Practices using Environment Folders

• Keep a Consistent Structure: Ensure that the folder structure and file names are consistent across environments to avoid confusion and errors.
• Use Variable Files: Define environment-specific variables in terraform.tfvars or equivalent within each folder to customize the configurations for each environment.
• Centralize Modules: Use shared modules for common infrastructure components to maintain consistency and reduce duplication.
• Version Control: Use a version control system like Git to track changes and manage collaboration between team members.
• Review and Test: Conduct code reviews and test infrastructure changes in the DEV environment before promoting to PROD.
• Documentation: Document the folder structure and the purpose of each environment to help new team members understand the setup.

Conclusion

Both HashiCorp Terraform workspaces and environment folders are valid strategies for managing Terraform deployments across multiple environments. The choice between them depends on your team’s workflow, the complexity of your infrastructure, and your preference for state management. By following the best practices outlined above, you can ensure a smooth and reliable promotion process from development (DEV) to production (PROD) environments with HashiCorp Terraform.

Original Article Source: Best Practices to Promote from DEV to PROD Environments with HashiCorp Terraform using Workspaces and Folders by Chris Pietschmann (If you’re reading this somewhere other than Build5Nines.com, it was republished without permission.)