Amazon S3 Recover Deleted File

Question

Is it possible to recover a file I deleted from my S3 bucket?

Answer 1

If you have versioning enable, yes you can! On a versioned bucket, a delete action on a file does not really delete it but it adds a version with a "Delete Marker". You can delete the delete marker with the AWS CLI:

aws s3api delete-object --bucket yourbucket-name --key "yourfile" --version-id id_of_the_delete_marker

You can get all the files in the bucket with

aws --output text s3api list-object-versions --bucket yourbucket-name > files.txt

If you want to undelete all the files in the bucket you can try:

echo '#!/bin/bash' > undeleteScript.sh && aws --output text s3api list-object-versions --bucket yourbucket-name | grep -E "^DELETEMARKERS" | awk '{FS = "[\t]+"; print "aws s3api delete-object --bucket yourbucket-name --key \42"$3"\42 --version-id "$5";"}' >> undeleteScript.sh && . undeleteScript.sh; rm -f undeleteScript.sh;

Answer 2

According to the Amazon S3 Documentation:

Once deleted, there is no method to restore or undelete an object.

Answer 3

Just updating this question as I was looking for the answer:

You can add VERSIONING to S3 buckets now. This will cause S3 to keep versions of an object even after deletion. Full documentation: http://docs.amazonwebservices.com/AmazonS3/latest/dev/Versioning.html

Answer 4

You can now list object versions using the AWS console: http://docs.aws.amazon.com/AmazonS3/latest/UG/managing-objects-versioned-bucket.html

If you have the right permissions, then you could delete the delete marker (the latest version of a deleted object) and the result would be that the object is available again.

Answer 5

Just in case you would like to restore huge chunks of s3 objects in a more sustainable way, here is a python version and bash versions of the same solution that @chris Cinelli shared

Python Version, sourced from here

#!/usr/bin/env python
from datetime import datetime, timezone
import boto3


Empty Bucket of all delete markers from all objects.


-----------------------------------
Enter these values here:
thebucket = "<bucket_name>"
access_key = "<access_key>"
secret_key = "<secret_key>"
region_name = "<region_name>" # us-east-1
------------------------------------
s3 = boto3.resource("s3", region_name=region_name, aws_access_key_id=access_key, aws_secret_access_key=secret_key)
s3client = boto3.client("s3", aws_access_key_id=access_key, aws_secret_access_key=secret_key)
paginate 100000 at a time
page_size = 100000
folder_in_thebucket = "files/invoices"
paginator = s3client.get_paginator("list_object_versions")
pageresponse = paginator.paginate(
    Bucket=thebucket, Prefix=folder_in_thebucket, PaginationConfig={"PageSize": page_size}
)
deleted_at = datetime(2019, 10, 22, 20, 0, 0, tzinfo=timezone.utc)
def restore_all(pages):
    # iter over the pages from the paginator
    for page in pages:
        # Find if there are any delmarkers
        if "DeleteMarkers" in page.keys():
            for each_delmarker in page["DeleteMarkers"]:
                if each_delmarker["IsLatest"] is True and each_delmarker["LastModified"] > deleted_at:
                    restore(each_delmarker)
def restore(delete_marker):
    # Create a resource for the version-object
    # and use .delete() to remove it.
    file_object_version = s3.ObjectVersion(thebucket, delete_marker["Key"], delete_marker["VersionId"])
    # I added this output just so I could watch the script run.
    print(f"Restoring {delete_marker}")
    # Lastly, lets remove the del marker and recover one of many files.
    file_object_version.delete()
if name == "main":
    print(f"Restoring files deleted after {deleted_at} in {thebucket}/{folder_in_thebucket}.")
    restore_all(pageresponse)

Bash Version, sourced from here

#!/bin/bash
bucket=$1
prefix=$2
set -e
echo "Removing all latest delete markers in $bucket with prefix $prefix"
#versions=aws s3api list-object-versions --bucket $bucket | jq '.Versions | .[] | select(.IsLatest | not)'
latest_delete_markers=aws s3api list-object-versions --bucket $bucket --prefix "$prefix" | jq '.DeleteMarkers | .[] | select(.IsLatest)'
echo "latest delete markers:"
echo "$latest_delete_markers"
echo "removing delete markers"
for marker in $(echo "${latest_delete_markers}" | jq -r '@base64'); do
    marker=$(echo ${marker} | base64 --decode)
# echo "marker:"
# echo "$marker"

key=`echo $marker | jq -r .Key`
versionId=`echo $marker | jq -r .VersionId `

printf "Removing delete marker $key $versionId ... "

aws s3api delete-object --bucket $bucket --key "$key" --version-id $versionId > /dev/null
printf "✓\n"


echo $cmd
$cmd
done