Why You Should Be Using CloudFormation

Like many of you reading this, I think keeping up with AWS can feel like a full-time job in and of itself. When I started at AWS in 2010, there were exactly 11 AWS services. Today? There are over 125!

One announcement that caught my eye recently was the news that AWS had added CloudFormation StackSet support for GuardDuty. Oddly enough, I use neither StackSets nor GuardDuty, but this announcement got me thinking about CloudFormation, how much I use it, and how important it is to me and the AWS customers I’ve work with.

What is CloudFormation?

For readers new to the AWS scene, CloudFormation is an AWS service that allows you to describe and provision your AWS infrastructure using a simple text file. These text files, or templates, act as the single source of truth for your infrastructure and are important “infrastructure as code” artifacts. 

It’s common to author a CloudFormation template, place it under revision control, and then use it to provision and update AWS infrastructure. I’ve included a short snippet of CloudFormation template below.

For more info on CloudFormation templates, including how to build and use them, see the CloudFormation product page.

I use CloudFormation on an almost daily basis. In addition to acting as my “single source of truth,” I also use CloudFormation templates as placeholders or bookmarks for ongoing research and personal projects. Each time I make a change to a project, whether production or not, that change is added to a CloudFormation template. If I get distracted or redirected to another project, I know that I can always come back to the CloudFormation template I was working on, whether a day or a month later, and easily restart where I left off.

CloudFormation also helps me maintain control over my monthly AWS bill. If I have AWS resources that I’m not actively using or experimenting with, I can use CloudFormation to temporarily tear down a stack when I don’t need it. When I need it again, CloudFormation can easily rebuild it.

CloudFormation is at the top of my short list of essential AWS tools and one that I frequently recommend to others. Whenever I start a new project or new experiment, CloudFormation is the first tool I grab. It’s always at the top of my AWS infrastructure toolbox. Whenever a new AWS service or feature is announced, the first question I ask is “Does it have CloudFormation support?”

Why use CloudFormation?

If you asked me to name my most frequently used AWS service, I’d probably answer “CloudFormation” for a couple of reasons:

First, Amazon does a great job of keeping CloudFormation current and updating it as new services and features become available. This hasn’t always been the case—there used to be a significant lag between the announcement of a new service and CloudFormation support for the service. When a new AWS service is announced, I can be sure that CloudFormation support already exists or will follow very shortly.

Second, Amazon continues to evolve the CloudFormation service by enhancing existing features and adding new functionality, including the addition of supporting tools to help author and manage templates. Like other AWS teams, the CloudFormation team works closely with their customers to provide them with the features most important to them and to help them use CloudFormation more efficiently.

All of this innovation keeps them busy. A quick scan of the AWS news feed shows that there have been over 100 announcements since the launch of CloudFormation that involve CloudFormation in one way or another.

My favorite CloudFormation features

Here are a few of my favorite CloudFormation features so far:

Nested stacks

Nested stacks are awesome! They allowed me to encapsulate logical areas of functionality or usage within my templates and make large or complex stacks more manageable. As a developer, nested stacks seem natural to me. They are an intuitive option that allowed me to extend best practices I adopted when writing code: abstract, decouple, simplify.

With nested stacks I can group related resources into a single template, like databases or business logic resources. Or I can separate templates into those that change frequently, like the application layer, and those that remain fairly constant over time. For example, I find that when building a stack using CloudFormation, the VPC, subnets, and route tables remain fairly constant over time. More often that not, I'm making changes by adding new applications or components or changing the associated security groups.

I also create nested CloudFormation templates whenever I find duplication within a template. For example, in multi-AZ VPCs, the configuration of resources in a single AZ can be extracted and moved out on its own.

Lambda-backed custom resources

Lambda-backed custom resources allow me to implement custom actions during template instantiation outside of those AWS resources that are created for the stack itself. These resources make it very easy to implement utility functions using Lambda and reference them during stack creation.

The Lambda function associate with a customer resource is invoked when that resource it is created, updated, or deleted. When invoked, CloudFormation passes request data to the function, which can be used to perform customer actions.

As a simple example, you can use a Lambda-backed custom resource to perform dynamic AMI lookups. Instead of maintaining a static lookup in CloudFormation (which most of us do or have done), I can use Lambda to query a DynamoDB table that I use to maintain my AMI list. The use of DynamoDB to persiste my AMI list also makes it much easier to the custom resource.

The list of possibilities using Lambda-backed custom resources is endless. You’re limited only by what you can do with Lambda, which isn’t much of a limitation at all.

More on Lambda-backed custom resources

Change sets

Change sets allow you to see or inspect changes that CloudFormation will make to your stack before actually applying them. This makes it much easier to make changes to CloudFormation templates and validate that what you intend to change as a result of the update is the only thing that is going to change. Before the introduction of change sets, aside from visually inspecting the updated template or performing a diff of old and new templates, you might not ever be sure of what was going to change.

Knowing what is going to change before the fact gives me confidence that the changes that will occur as a result of changing the template are intentional and correct. If I’ve accidentally made a change that would update or delete a business critical resource, I want to know about it before any damage has been done. And even if the change is intentional, this feature gives me one last opportunity to review and verify the change.

There are many times before the introduction of change sets where I updated an AWS resource by changing an property of the resource, unaware that changes to that particular property necessitated the creation of a new resource. SQS queue names, Lambda function names, and EC2 private IP addresses are good examples. Change sets help mitigate that issue.

More on change sets

YAML

Finally, YAML and the introduction of CloudFormation template support for YAML. In my opinion (but it’s true), this is the single greatest improvement CloudFormation has made since launching in 2011. Functionally, it provides no additional benefit over the original JSON template format, but YAML is so much better than JSON ;)

The downside, of course, is that I spend my time hunting down template errors caused by indenting and spaces instead of curly bracket placement (or angle brackets for you XML nerds).

More on YAML

A gateway to infrastructure as code

I use SQS and S3, EC2, RDS, Lambda, and SageMaker like everyone else. These are the services we all typically use to build our software and services on AWS. But when I want to “build” infrastructure on AWS, I always turn to CloudFormation.

CloudFormation is the glue that holds my AWS infrastructure all together. The “one ring” if you’re a LOTR fan. Enjoy!