What are the differences between the rsync delete options?

Question

I see on the rsync man page that there are a number of delete options, but don't really understand the differences between them. What are the differences between these options?

 --del                   an alias for --delete-during
 --delete                delete extraneous files from dest dirs
 --delete-before         receiver deletes before transfer (default)
 --delete-during         receiver deletes during xfer, not before
 --delete-delay          find deletions during, delete after
 --delete-after          receiver deletes after transfer, not before
 --delete-excluded       also delete excluded files from dest dirs

Answer 1

• --del/--delete_during: Deletes files from the destination dir as they are copied (saves memory compared to --delete-before: --delete-before makes a separate scan to look for deleteables)

• --delete: Deletes files in the destination directory if they don't exist in the source directory.

• --delete-before: Delete files in the destination directory before copying file-with-same-name from source directory

• --delete-during: Delete files in the destination directory WHILE copying file-with-same-name from source directory

• --delete-delay: Mark deletes during transfer, but wait until transfer is complete

• --delete-after: Receiver deletes after transfer, not before...If some other part of the rsync moved extra files elsewhere, you'd want this instead of --delete-delay, because --delete-delay decides what it's going to delete in the middle of transfer, whereas --delete-after checks the directory for files that should be deleted AFTER everything is finished.

• --delete-excluded: Deletes files from the destination directory that are explicitly excluded from transferring from the source directory.

The point of rsync is not copying, it is archiving. This is an important distinction. Processing deleted/changed files is critical, and in many cases nuanced.

The --delete flag in particular is one I've seen screwed up many times. A lot of people use rsync to move files to low priority storage, and in this case you want the files you're moving to still EXIST in the destination directory. That's not what delete does: --delete makes sure that, when you delete a file from the source directory, it is ALSO deleted from your destination directory, so your destination doesn't get full of junk... Once saw a guy wipe out his backup by putting in a new drive, and not turning off his nightly rsync script. Script saw that the source dir was now empty, and it deleted every file in the destination dir, so they'd match.

Most of the other options are space or performance related. When you delete the files is important if you want to make sure the transfer is successful before you do anything, but if your device is too small to handle 2 copies of all the information, you need to delete as you go, etc. It's a little wacky because of its long history across multiple platforms: some options have been added so that people who were used to certain behaviour wouldn't be confused.

Answer 2

By default rsync doesn't delete any files at the destination side. To make rsync delete files at all, you need to use at least one of the delete options.

If you don't care when files are being deleted, just use --delete and leave the choice to rsync. You can combine --delete with other delete options (this doesn't conflict) but you don't have to as all other delete options already imply --delete.

--delete-before works as follows: rsync looks which files are present at the source and which files are present at the destination, deletes all files found at the destination but not at the source and then starts the actual synchronization. This order is useful if the destination has little storage space as it will first free up more disk space at the destination before starting transferring any new files. The downside is that rsync will require more memory to perform the operation and the whole operation is a two step process and thus slower.

--delete-during works as follows: rsync immediately starts to synchronize files and when it comes across a file that exists only at the destination, it is deleted. That way there is no speed penalty and also no additional memory is required. The downside is that it may happen that first a lot of new files are copied to the destinations before removed files are being deleted, so the destination may require much more disk storage space during the operation than it requires in the end once the entire operation is done.

--delete-after works as follows: First synchronize all files, then perform the same operation that --delete-before performs before the synchronization phase. This is the worst choice in most common cases as it requires most memory, most disk space at the destination, and it is slower as it is a two step process; basically it combines all disadvantages of the other two methods. This option mainly exists for the case that you are using "merge files" (what merge files are and how they work is beyond the scope of this answer). As these files may contain rules for files to be excluded during delete, new merge files must be copied before the deletion phase if their content shall be considered during the deletion phase. Unless that is a requirement, --delete-after has no advantage.

--delete-delay is a rather new option (it's not available in rsync 2.6.9, which is still the default in macOS 10.15 for example). It works like --delete-during, except that it won't delete files immediately but after the synchronization is done, so it is a hybrid of --delete-during and --delete-after. The advantages are that it is faster than --delete-after and still supports merge files correctly, the disadvantage is that it requires more memory and disk space during the synchronization just like --delete-after.

--delete-excluded tells rsync to not just delete files that are missing at the source but to also delete files at the destination that were excluded from synchronization (--exclude or --exclude-from), regardless if these files would actually exist at the source or not.

Which one shall I use?

If you use merge files, you probably want to use --delete-delay, if available, otherwise --delete-after.

If you don't use merge files, you usually want to use --delete-during which will be fastest.

Only if the storage space at the destination is very tight and the data to be synchronized just barely fits there, you may want to use --delete-before instead as in that case --delete-during could fail as the destination could run out of disk space yet --delete-before would then probably still succeed.

As for memory consumption, that's usually not an issue at all with modern systems which have plenty of spare memory available. It may be an issue with tiny embedded devices but those typically don't use rsync in the first place for file synchronization and also have very little spare disk space, too.

Answer 3

One other point worth mentioning is that if your source directory ends with /*, then rsync will only consider those files and not the directory itself (and thus the absence of files that you want deleted on the destination).

If you're specifying a delete option above, but rsync looks like it isn't deleting, then check to make sure you're not accidentally globing and providing a list of files when you mean the directory itself.

Answer 4

There are two things going on:

1. Who does the deleting
2. When it happens

Either the sender or the receiver can be instructed to do the deleting (I'm not sure why this matters). So when rsync from one computer connects to the rsync server on the other side, this determines who is effectively issuing the delete command.

When it happens is pretty easy... before means all the files are deleted, and THEN rsync copies over the files. during means as it goes through the list of files, it deletes them when it comes to them, and after means it waits until all the files are transferred over and then deletes the remote side. This matters only when the transfer gets interrupted.