S3 orphan buckets finder

The S3 clear orphan buckets is a simple script that allows you to find and delete S3 buckets that remain after deleting the Cloud Formation stacks with which they were created. This situation occurs when DeletionPolicy on S3 bucket object in the Cloud Formation template is set to Retain.

Features

• The script only performs actions on S3 buckets with a specific tag (key & value) and after deleting the Cloud Formation stack they were part of.
• S3 buckets can be listed or deleted.
• The script must be executed with the following arguments:
  • tag key (-k or --tag-key);
  • tag value (-v or --tag-value).
• As a result of invoking the script you will get the S3 bucket names, against which the action was taken (if any).

Requirements

• Python third party packages: Boto3
• Before using the script, you need to set up default AWS region value and valid authentication credentials for your AWS account (programmatic access) using either the IAM Management Console or the AWS CLI tool.
• The entity running the script should have the appropriate permissions to:
  • create, update & delete Cloud Formation stacks;
  • create, delete & list S3 buckets;
  • put, list & delete objects in S3 buckets.

Installation with venv

The script can be run locally with virtualenv tool. Run following commands in order to create virtual environment and install the required packages.

$ virtualenv venv
# or
$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install -r requirements.txt

Running the script

Script usage (detailed help):

(venv) $ python s3_orphan.py --help
usage: s3_orphan.py [-h] {list,delete} -k TAG_KEY -v TAG_VALUE

The orphan S3 bucket finder

positional arguments:
  {list,delete}         action performed on a found S3 bucket

optional arguments:
  -h, --help            show this help message and exit
  -k TAG_KEY, --tag-key TAG_KEY
                        perform action on S3 buckets with specified tag key
  -v TAG_VALUE, --tag-value TAG_VALUE
                        perform action on S3 buckets with specified tag value

You can start the script using one of the following examples:

# List S3 buckets with tag Key: 'Project' and Value: 'find-orphan' assigned.
python s3_orphan.py list -k Project -v find-orphan
# You should get the similar output:
S3 bucket "find-orphan-dummy-bucket-1-dev-s3bucket-abc12a34fb5c" is orphaned.
S3 bucket "find-orphan-dummy-bucket-2-dev-s3bucket-1ab2cde3456fg" is orphaned.
# or if no action has been taken
Nothing to do...

# Delete S3 buckets with tag Key: 'Project' and Value: 'find-orphan' assigned.
python s3_orphan.py delete -k Project -v find-orphan
# You should get the similar output:
S3 bucket "find-orphan-dummy-bucket-1-dev-s3bucket-abc12a34fb5c" deleted.
S3 bucket "find-orphan-dummy-bucket-2-dev-s3bucket-1ab2cde3456fg" deleted.
# or if no action has been taken
Nothing to do...