Comparing different methods of testing your Infrastructure-as-Code

Comparing different methods of testing your Infrastructure-as-Code

Comparing different methods of testing your Infrastructure-as-Code

App complexity is growing, and the required cloud provisioning is becoming more complex as a result. Chances are that the infrastructure you’re building utilizes numerous frameworks, automation tools, open-source tech, and many different components.

Additionally, the infrastructure created by your IaC is vital. Your testing teams rely on it to build and manage multiple environments before moving to production, so if your IaC has any flaws, that will skew the results for the testing teams using your environments, resulting in wasted time and money.

Like all code, your Infrastructure-as-Code (IaC) should be tested before you run it, to check for bugs or mistakes.

Software engineers and DevOps teams working with IaC can use several different tests. There are different approaches to which tests to run, in what order, and whether to run them separately or all at once.

Main models:

  • Test for basic validity
  • Unit testing the individual components of the code
  • Module test and component tests
  • Integration testing to check that all the modules work together
  • Systems testing to verify that that code works when you deploy it
  • Template testing using linting and validation tools to check for errors

Some of these tests can be run manually, like validity testing or some unit testing, especially when the infrastructure is still small. But you’ll rapidly need to move on to automated tests, particularly for end-to-end (E2E) testing, which includes components testing, integration testing, and systems testing.

In theory, you can use the same automated tests that you use on application code, but that’s not always effective or efficient. It takes longer to set up and run tests that way, and some types of IaC code are less suitable for app code tests. It’s best to use a different approach and different tools to test your IaC code.

The best testing tool will vary depending on which IaC you use. For the purposes of this article, we’re assuming that you’re using Terraform and want to test your Terraform IaC before deployment. Here are some of the tools you can use.

1. Terratest

Terratest is a test framework developed by Gruntworks that uses the GO language to write automated tests for Terraform, Packer, and Docker infrastructure. Its main focus is integration testing, but it’s also effective for unit tests, module testing, and Kubernetes environments. It allows you to execute real infrastructure code and validate the results, which is the only way to be confident that your code really works.

But because it builds up and tears down your code, it can be slow to write tests and expensive to run them. For example, Terratest might not be the best choice if you’re testing configurations.

Some compelling new Terratest alternatives have been released that address these issues. OpenTofu solves the same challenges as Terratest through a native command in the Terraform/OpenTofu binary. It uses tests that are written in HCL, so you can use declarative language instead of GO.

2. Checkov

Checkov is a static analysis tool for Terraform, CloudFormation, and Kubernetes infrastructure that scans for misconfigurations and compliance violations. It’s based on Python and has a nicely intuitive UI, making it easy for most developers to use.

Checkov connects to several benchmarks and standards, such as HIPAA, CIS, and NIST, making it an excellent choice for validating adherence to compliance frameworks.

However, it’s not so great for E2E testing. The default tests are effective for static code analysis and best practice configuration tests, but you’ll need to write your own tests for any deeper analysis, which can be time-consuming.

3. InSpec

InSpec is the test framework for Chef. It allows you to write tests for Terraform-managed infrastructure to ensure that it complies with your security and compliance policies. InSpec is based on Ruby, a popular coding language, and it’s compatible with Chef and Ansible, too.

InSpec is a great choice if you’re working with several IaC tools and want a cross-platform test framework. It’s often faster and easier to use than Terratest.

However, you do need to write huge shell scripts to describe your tests, which isn’t the most efficient use of your time. The solution is to combine InSpec with Test Kitchen so that you can write a simple configuration file that wires up Test Kitchen to create, test, and tear down your Terraform infrastructure.

4. Kitchen-Terraform

Kitchen-Terraform is an extension of the Test Kitchen framework, offering a set of open-source plugins that use InSpec to allow you to verify the state of your infrastructure. Like InSpec, it’s based on Ruby, but it offers a much faster alternative to using InSpec alone to write your tests.

The main focus and best use for Kitchen-Terraform is to test infrastructure configurations. It’s excellent for module testing throughout the CI/CD pipeline. This way, you can run a quick test to validate a quick change and then deploy without worrying that you’ve messed up the entire code.

Kitchen-Terraform requires you to bundle multiple testing frameworks, and there’s less community support than for Terratest. However, its multi-test nature also means you can use it to test modules that were built in AWS.

5. Conftest

Conftest is built on top of Open Policy Agent (OPA), which is a general-purpose policy engine for writing tests. It’s effective in various structured data formats and Terraform, including Kubernetes configuration files and Dockerfiles.

Conftest’s main focus is on compliance and conformance testing, so it’s a good choice for those tests, but it’s less effective for integration or E2E testing.

Conftest uses OPA’s own policy language, Rego, which means that users need to learn a new language to use it to the fullest extent. However, Pytest uses Conftest to support tests written in Python, such as conftest.py, which makes it more accessible to the average developer.

The best IaC test framework is up to you

At the moment, there’s no single stand-out ideal Terraform testing tool. Most developers must use several tools to test different aspects of their IaC. The best test framework depends on which type of test you want to run, which coding languages you’re comfortable in, your need for speed vs depth, and many other factors. The options will likely keep growing for a while before some integration and a standard set of test tools becomes apparent, so this is a space worth keeping an eye on.

devxblackblue

About Our Editorial Process

At DevX, we’re dedicated to tech entrepreneurship. Our team closely follows industry shifts, new products, AI breakthroughs, technology trends, and funding announcements. Articles undergo thorough editing to ensure accuracy and clarity, reflecting DevX’s style and supporting entrepreneurs in the tech sphere.

See our full editorial policy.

About Our Journalist