Linux bash backup script - 12, 9, 6, 3, 1 months

Question

I am writing a bash backup script, and it's working excellent so far. The problem is that it clutters up my harddrive in no time.

The backup runs weekly on sundays.

I would like to:

Save the most recent 4 backups
Save the backup which is 3 months old
Save the backup which is 6 months old
Save the backup which is 12 months old

Now how do i achieve this? I think i can work out how to "check if file exists", but i'm having trouble getting my head around how to delete the correct backups.

The backup 3 months old, will be 3 months and 1 week old by next week - and thus be deleted.. Is there any geniously simple way to work around this that i may have overlooked..?

Thanks in advance,

Kip K · Accepted Answer · 2015-10-29T18:16:29.620

If you give your Backup files a nice naming scheme like: 10.29.15-BACKUP.zip you could always do it real easily. Easiest where you can just have 2 separate folders one for Daily Backups and one for Archives.

So in your bash script:

#BACKUP PROCESS HAPPENS HERE, PLACES BACKUP NAMED 10.29.15-BACKUP.zip in /home/User/DailyBackups FOLDER, WHICH WE WILL CALL $CurrentBackup

#Get Date from 3 months ago
ChkDate=`date --date="-3 months" +%m.%d.%y`

#See if this file exists
ls $ChkDate-BACKUP.zip /home/User/BackupArchive/

#If it does exist then copy current backup to BackupArchive Folder and Remove any backups older than 367 days from the BackupArchive Folder
if [[ $? == 0 ]]; then
    cp /home/User/DailyBackups/$CurrentBackup /home/User/BackupArchive/$CurrentBackup
    find /home/User/BackupArchive/*-BACKUP.zip -mtime +367 -exec rm {} \
fi

#Remove all but the most recent 4 Backups
for i in `ls -t /home/User/DailyBackups/*-BACKUP.zip | tail -n +5`; do
    rm "$i"
done

I used 367 to account for a 366 day leap year and just in case your one year backup was a bit off, like 366 days and 1 minute.

Oops, forgot about the 4 in Daily Backups, I'll add another line. — Kip K, Oct 29 '15 at 17:53
I think you may have the easiest solution here - with a 2 folder system... For now, i think i might go with simply doing CRON job every 3rd month, and saving 4 backups (deleting all older backups).. - It seems like the most sensible and manageable thing to do. :) - Thanks to you all for the input! #ThumbsUp — skywalkerdk, Oct 29 '15 at 21:43

score 0 · Answer 2 · answered Oct 29 '15 at 15:16

I had a similar task to delete files up to n date what i had to do was:

1. generate an interval date from todays date (like 3 months ago)
   [this post has a good writeup about getting specific dates
   http://stackoverflow.com/questions/11144408/convert-string-to-date-in-bash]

2. loop over all the files in the location and get their time\date stamp with
   date -r <filename> +%Y
   date -r <filename> +%m
   date -r <filename> +%d

3. Compare file date to interval date from todays date and keep if it matches or delete if not.

Hope this helps you to get the concept going

score 0 · Answer 3 · edited May 23 '17 at 11:58

Suppose you named the backup according to the date:

% date +%Y-%m-%d
2015-10-29

Then you can compute the date one year ago like this:

% date +%Y-%m-%d -d "-1 year"
2014-10-29

and the date 5 weeks ago like this:

% date +%Y-%m-%d -d "-5 weeks"
2015-09-24

So you can setup cronjobs with run every 3 months and every Sunday, and delete backups that occurred one year ago and 5 weeks ago like this:

# Every 3 months, run the backup script
1  2    *    */3   *    /path/to/backup/script.sh > /path/to/backupdir/$(date +%Y-%m-%d-Q)

# Every 3 months, delete the backup made on that date the previous year
1  2    *    */3   *    /bin/rm /path/to/backupdir/$(date +%Y-%m-%d-Q -d "-1 year")

# Every Sunday, if backup does not exist, run backup script
1  3    *    *   7    if [ ! -f /path/to/backupdir/$(date +%Y-%m-%d-Q) ]; then /path/to/backup/script.sh > /path/to/backupdir/$(date +%Y-%m-%d) fi

# Every Sunday, delete backup 5 weeks old
1  3    *    *     7   /bin/rm /path/to/backupdir/$(date +%Y-%m-%d -d "-5 weeks")

Note that

We want to be careful not to run the backup script twice on the same day, for example, when a quarterly backup happens on a Sunday. If the quarterly backup cronjob is set to run (at 2:00AM), before the weekly backups (at 3:00AM), then we can prevent the weekly backup from running by testing if the backup filename already exists. This is the purpose of using
```
[ ! -f /path/to/backupdir/$(date +%Y-%m-%d-Q) ]
```
When we delete backups which are 5 weeks old, we do not want to delete quarterly backups. We can prevent that from happening by naming the quarterly backups a little differently than the weekly backups, such as with a Q:
```
% date +%Y-%m-%d-Q
2015-10-29-Q
```
so that the command to remove weekly backups,
```
/bin/rm /path/to/backupdir/$(date +%Y-%m-%d -d "-5 weeks")
```
will not remove quarterly backups.

Thanks for the cron examples! They are very helpful! :o) – skywalkerdk Oct 29 '15 at 22:01 — skywalkerdk, Oct 29 '15 at 22:01

Linux bash backup script - 12, 9, 6, 3, 1 months

3 Answers3