How To Import Existing AWS Resources Into CloudFormation

Looking to import manually created (existing) AWS resources into your SAM or CDK project (with CloudFormation) ? This is the video for you.

Often many developers start learning AWS by manually creating resources in the console. One thing leads to another and we end up launching full blown services by manually creating resources.

Fast forward 6 months and we learn about CloudFormation, a powerful Infrastructure as code toolkit that allows you to define your AWS using template files (or code if using CDK).

It turns out, using tools like SAM or CDK are the preferred ways of defining resources in AWS for any serious developer. It isn’t long when we try to switch over to using the ‘right’ approach.

Now we run into a problem – we now need to integrate those existing AWS resources into our project so we can start managing them using code. It turns out there’s no ‘suggested’ way to do this via AWS documentation, but there are some tricks you can follow to integrate these resources into your CloudFormation stack, and subsequently your SAM or CDK project.

In this article, I’ll show you an example of how to migrate a DynamoDB table into a SAM based project.

Now lets figure out how its done.

Starting State

Our starting state is a CloudFormation stack called cf-import-demo. Currently, the only AWS resource that it manages is an SNS topic.

This stack was setup using the SAM toolkit and running the sam init command and selecting the Hello World option. All boilerplate has been stripped out leaving us with a bare bones template.yaml file.

AWSTemplateFormatVersion: '2010-09-09' 
Description: >
  cf-import-demo
  Sample SAM Template for cf-import-demo

Resources:
  DemoSNSTopic: 
    Type: AWS::SNS::Topic
    Properties:
      TopicName: DemoSNSTopic

Now here’s our DynamoDB table called TableToImport that was created manually in the console. This is our target for importing into CloudFormation.

Note that I’ve made almost zero modifications to this table – the only configuration to note is the ProvisionedReadCapacity and ProvisionedWriteCapacity which are currently set to 1. Remember this for later on.

Our end state will be our CF stack having control over our DynamoDB Table.

So lets get into the how.

Step 1 – Modify Our tempate.yaml File

Our first step is to make some modifications to our template.yaml file in our SAM project file.

What we need to do is effectively create a CloudFormation resource that mimics the configuration we have specific on our manually created DynamoDB table.

We need to define an exact match (or as close as possible match) to this entry. This means manually looking through the configuration on your table, and looking up CloudFormation documentation to add the corresponding settings.

For example, in my manually created TableToImport table, the options that I’ve configured are a) Partition Key on the table, and b) read and write capacity. This means in my corresponding entry into my template.yaml file, I’ll have to specify my Partition key and capacity values.

Here’s what the corresponding DynamoDB resource looks like:

  TableToImport:
    Type: AWS::DynamoDB::Table
    DeletionPolicy: Retain
    Properties:
      TableName: TableToImport
      AttributeDefinitions:
        - AttributeName: "demo"
          AttributeType: "S"
      KeySchema:
        - AttributeName: "demo"
          KeyType: "HASH"
      ProvisionedThroughput:
        ReadCapacityUnits: "1"
        WriteCapacityUnits: "1" 

We’ll append the resource to our template.yaml file alongside our SNS topic.

Now we have a modified CloudFormation file that contains our soon to be imported resource. We now need to link the resource with our CloudFormaiton stack. To do this, we go to AWS console.

Our final template.yaml file looks like this:

AWSTemplateFormatVersion: '2010-09-09' 
Description: >
  cf-import-demo
  Sample SAM Template for cf-import-demo

Resources:
  DemoSNSTopic: 
    Type: AWS::SNS::Topic
    Properties:
      TopicName: DemoSNSTopic
  
  TableToImport:
    Type: AWS::DynamoDB::Table
    DeletionPolicy: Retain
    Properties:
      TableName: TableToImport
      AttributeDefinitions:
        - AttributeName: "demo"
          AttributeType: "S"
      KeySchema:
        - AttributeName: "demo"
          KeyType: "HASH"
      ProvisionedThroughput:
        ReadCapacityUnits: "1"
        WriteCapacityUnits: "1" 

Note that if using SAM + Serverless, you’ll need to remove the Serverless::Transform resource from the header of your generated template.yaml file. Not doing so will cause the next step to fail at the upload step.

Another important note is the DeletionPolicy: Retain component of our DynamoDB Table resource. THIS IS A REQUIRED ATTRIBUTE THAT YOU MUST SET. Again, failure to set this option on all resources that you are trying to port over will result in an error at the next step.

Step 2 – Link Our Resource to CloudFormation

We need to head over into the CloudFormation section of the AWS console and click on our Stack. From here, we select the Stack Actions menu and click Import resources into stack.

Clicking it will launch a Wizard that guides you through the import process. What we need to do in this step is upload our newly created template.yaml file into the wizard. This will give CloudFormation the ability perform a diff between what it thinks your stack is composed of, and what resources you’ve attempted to add.

It will automatically recognize the diff and import the existing resource into your current stack.

To keep moving forward, go the second step of the wizard and upload the updated file and click next.

Next, give your imported resource a logical ID to refer to late. I just name this the same name as my resource, in this case TableToImport.

You can effectively skip to the overview part of the wizard (unless you want to specify any additional parameters). CloudFormation will perform a diff of your template file and what it has on record and let you preview the resources it is about to import.

Before you proceed with the next step, PLEASE make sure to test out this process on your beta/test/development environment. Although this process is straightforward you want to be sure it is working correctly in safe environment before performing these steps in production.

If you’re satisfied, you can go ahead and click on Import Resources to perform the link.

If all went well, you should be forwarded to the Events section of the stack, and should see an update stream and hopefully IMPORT COMPLETE as the final update.

We can go to the Resources tab and should see our newly linked DynamoDB table!

Before we claim success, lets go back to our editor and make some updates to our imported resource. If all worked correctly, we should see corresponding updates to our cloudformation stack and DynamoDB table.

Step 3 – Test It Out

To see if the our setup is working, I’m going to modify a trivial value as part of the ProvisionedThroughput option. Originally, we set this value to 1. I’m going to change it to 2 and see if the update occurs.

      ProvisionedThroughput:
        ReadCapacityUnits: "2"
        WriteCapacityUnits: "2" 

After making the change and saving our work, we run sam build and sam deploy to deploy our changes. We’re greeted with a confirmation step asking us to make sure we want to deploy the change.

Notice that the prompt is showing that we are modifying our DynamoDB table.

Confirming the prompt causes SAM to perform the change. After its done, we go to the DynamoDB section to see if our Table has been changed correctly.

Bam! Our new change is now showing up on our resource.

I hope you found this article useful on migrating your resources into CloudFormation/SAM. If you have any questions or comments feel free to write me below.