Looking to download all the files in your S3 bucket? This is the article for you.
You have have noticed there’s no button in the AWS S3 console that allows you to download the entire bucket’s contents. Its a frustrating lack of such a basic feature. But don’t fret – in this article I’ll show you how to download your bucket in a few simple commands.
If you already have the aws cli installed and configured, feel free to skip down to Step 2.
Step 1 – Install and Configure AWS CLI
To get started, ensure you have the aws cli installed. If you don’t have it on your machine, use the below command to install it using pip.
sudo pip install awscliNext, ensure you have your CLI configured by running aws configure. You’ll be prompted with a series of questions like access key and secret access key. Make sure you have an IAM user created with AdministrativeAccess (or something close to it) and provide the relevant values.
If you don’t know how to create a user or gain access to your keys, watch my video tutorial on youtube.
Step 2 – Use the S3 Sync Command
To copy our data, we’re going to use the s3 sync command. First, you’ll need the name of your bucket so make sure to grab it from the AWS console.
To copy, run the following command:
aws s3 sync s3://<YOUR_BUCKET> <STORAGE_LOCATION>For example, my bucket is called beabetterdev-demo-bucket and I want to copy its contents to directory called tmp in my current folder. I would run:
aws s3 sync s3://beabetterdev-demo-bucket ./tmp
After running the command, AWS will print out the file progress as it downloads all the files. Since I only had one file in the bucket, here’s what mine looked like:
