What HDD copy strategy is best between manually syncing and rewriting from scratch?

Question

To backup my personal data, I have two WD Red 4 Tb HDDs, labeled "disk 1" and "disk 2".

Regularly, I put all the files that I want to save on disk 1. Then, using this specific docking station, I proceed to clone disk 1 on disk 2.

The docking station does not allow me to connect the two drives at the same time on my computer. However, it has a built-in cloning function. Each time I use it, disk 2 is formatted and wiped, and disk 1 is cloned on it from scratch.

I have done a number of backups this way. However, the cloning takes more and more time as disk 1 gets filled. Also, I am not certain it is "healthy" for the drive to wipe it and rewrite it regurlarly.

I read that there are tools like Total Commander that allow you to only copy the differences between two drives. To use it however, I would need to format disk 2 again since it is currently a clone of disk 1 (my computer will only detect one if the two are connencted simultaneously, one via SATA on the mobo and the other on the docking station).

Should I keep cloning my drives this way, or is it better to wipe disk 2 and only copy the differences from now on?

Answer 1

The docking station does not allow me to connect the two drives at the same time on my computer

Everything seems to indicate that it should – the ASM1156 chipset that it uses, and the design and features it has in common with many other ASM1xxx-based two-port stations, and even the "Please do not remove your hard drive or insert a new one while the other hard drive is being accessed" note in the manual (typical of devices using this chipset, all of which support both disks simultaneously).

That said it might not perform as well with two disks compared to one – e.g. you can't swap disks in one bay while the other disk is in use (as that causes a disconnect), and at least with similar stations that used earlier ASM chipsets, only one disk could be given commands at a time so diskA-to-diskB copies would be quite slow in general (in contrast to computer-to-disk copies which would be fast). I'm not sure if they solved that limitation in the ASM1156.

I have done a number of backups this way. However, the cloning takes more and more time as disk 1 gets filled.

This is also slightly surprising as the cloning station should be doing just a 1:1 copy – it is unaware of which areas are in-use – but I suspect it's the result of WD Red being a SMR ("shingled" magnetic recording) disk instead of the regular CMR, so it's the writing to your 'target' disk that becomes slower over time.

Also, I am not certain it is "healthy" for the drive to wipe it and rewrite it regurlarly.

HDDs in general don't care; their physical storage doesn't wear out nearly as quick as flash storage of SSDs. For example, traditional CMR disks do not have any kind of wear-leveling and are fine with overwriting data in-place.

Although your WD Red is a SMR disk, which is somewhere in between HDDs and SSDs – the physical storage is still like that of a HDD (i.e. there is no "wear out" like flash storage has), however, due to the way data is physically arranged on the platters in SMR mode, such a disk has internal state like SSDs do.

Specifically, an SMR HDD keeps track of which areas are 'written', it has to rearrange them in background, and if a lot of data is written it might gradually lose performance unless you use TRIM (sort of like with SSDs). Again, it won't really wear out the surface, but the performance is not predictable.

Should I keep cloning my drives this way, or is it better to wipe disk 2 and only copy the differences from now on?

In your current method, you're one distraction away from accidentally swapping the disks and cloning your "last week" backup right over your "today's files" disk. Most file-level backup solutions would be more failproof as well as being much faster.

For example, Total Commander's "Synchronize Directories" feature is not very automatic, but at least it would make it a little more obvious in the initial 'Compare' window that you might be about to delete files rather than copy new ones.

(There are ways to automate it – e.g. a script can be written to check whether the target disk contains a "This is the backup disk" marker file, and to run robocopy.exe if everything matches.)

Your current method also doesn't make it easy to keep multiple versions (not without having a box of weekly HDDs or something like that). Many file-level backup systems do more than just synchronization – they support keeping a history of backups, so that e.g. in case of certain types of data corruption, you could still restore last month's files after you've realized you just did a backup of the corrupted version.

Answer 2

However, the cloning takes more and more time as disk 1 gets filled.

This suggests that the cloning is not designed to handle SMR drives sanely.

Properly cloning to an SMR drive requires issuing a DISCARD command (or the equivalent for whatever interconnect is being used for the drive) that covers the whole device, and then writing out the new data in a single pass.

Not doing that will result in an excessive amount of rewrites because of how SMR works, which in turn means that successive rewrites will get slower each time the total amount of data being written gets bigger.

Also, I am not certain it is "healthy" for the drive to wipe it and rewrite it regurlarly.

For an traditional hard drive this is functionally fine. Magnetic storage media does wear out over time, but for devices like hard drives that use a coated rigid surface where the media does not come into physical contact with anything, the impact is minuscule, and MTBF (the average time expected to pass between failures) for a media error caused by simple degradation is typically measured in centuries these days.

However, for a SMR drive being cloned in a non-SMR-aware manner as seems to be the case here, this is likely to put excessive wear on the head armature bearings due to the amount of seeking needed for rewrites. That on it’s own isn’t dangerous per-se, but it will wear the drive out faster than it otherwise would be.

Should I keep cloning my drives this way, or is it better to wipe disk 2 and only copy the differences from now on?

At minimum, you should not be directly cloning the drive unless you only care about protecting against media damage. In particular, if something goes wrong with the filesystem or the backup itself on disk 1, any error there is likely to get faithfully replicated to disk 2 with your current approach, and you then have no good backup.

I would also generally advise against using disk cloning in any kind of backup scenario because:

1. It makes it difficult at best to have both backup devices connected at the same time. Being able to do this is a useful property for both data recovery and integrity checking.
2. It makes it much easier to accidentally wipe the current backup with a previous backup.

Instead, you should at least be copying the backups at the file level, which eliminates the first issue I mentioned with cloning, slightly reduces the risk of the second, and mostly eliminates the possibility of replicating filesystem errors from the main backup to the secondary backup.

Ideally though, I would instead argue for backing up twice, once to each device. If done right with proper backup software (such as restic or borg) this will make it almost impossible to clobber a current backup by mistake, and will also mostly eliminate the possibility of replicating errors in the backups themselves from the main backup to the secondary backup. It also simplifies usage of proper incremental backups, which will both speed up your backups, and likely cut down significantly on the space requirements for them.

All of that advice about not cloning and instead backing up twice has absolutely nothing to do though with the fact that you’re using SMR drives, or that you’re using an external dock, or that you’re using a hardware disk cloning system.

Answer 3

Suppose the following happens:

• You clone disk 1 to disk 2
• Disk 1 dies during the operation
• You now have a dead disk 1 and a partially written, unusable disk 2

So not only is this method slow, it is also unsafe. In fact any kind of read error or loss of power will result in an unusable clone. If the HDD interface/cloner dies or misbehaves, it can also wreck both drives. Worst case scenario, if lightning strikes your PC with both drives attached, there's a chance of zapping both the main drive in the PC and both backups at the same time.

In addition it encourages you to keep both backups in the same place and near the PC, which is always a bad idea: a thief can steal both the PC and the backups.

It would be preferable to use a tool that copies changed/new files. I use Robocopy on windows, and rsync on linux.

You don't need to do the backup twice every time. It's fine if one drive has this week's backup and the other has last week's backup. It can be an advantage, if you realize you deleted an important file, and it was also deleted on the most recent backup, but the old one still has it.

You can keep one backup drive near the PC, and the other somewhere else, preferably in a place that will still stand if the house burns down. Even better if a thief can't steal all the hardware at the same time.

Then every time you do a backup, swap the drives.

Previously I put an encrypted SSD in the garden shed, in a ziploc bag. HDDs have lower tolerance for temperature and humidity, but SSDs don't seem to care.

I think the reason your PC only sees one drive is they're identical clones, with the same partition and volume identifiers. If you format the second drive it will have a different identifier and the PC should be able to see both.

However if you use the method described above, you don't need to connect both drives at the same time, so it doesn't matter whether you wipe the second one or not. What's important is to stop using clone and backup to one drive at a time while the other is offline in a safe place.