6

I have two macs running OS X 10.10.3. One has Server.app installed, is connected to the internet via ethernet, and also has a 2 TB disk attached via USB, with two partitions - "Backup" and "Storage". My other mac is a laptop used for schoolwork.

Using the Time Machine feature of Server.app, I was able to use the standard Time Machine interface to begin backing up my local mac onto the "Backup" volume on the server mac over the network.

However, I discovered that it takes 7 minutes to copy a 1 GB file over AFP, so backing up 300 GB on my local mac would take at least 35 hours.

So, I'm trying to figure out a way in which I could do the initial 300GB backup onto "Backup" over USB, and then connect that external hard drive to the server mac and continue doing backups over the network. That way, the incremental network backups would be much smaller than 300GB and could be done in a reasonable time period, e.g. overnight.

Any suggestions? I read the man page for tmutil but I'm not very comfortable with it just yet.

4 Answers4

6

Better later than never, here is a hack to achieve this :

sudo ifconfig lo0 alias 127.0.0.2/32

Plug your usb drive, then via System Preferences / Sharing add a smb share to a folder time-machine-macbook in the drive time-machine-usb Then, add a destination backup (Time Machine will see it as a network share)

sudo tmutil setdestination -a "smb://user:password@127.0.0.2/time-machine-macbook"

You can see if it's ok with:

tmutil  destinationinfo
> ==================================================
Name          : time-machine4
Kind          : Network
URL           : smb://kenji@realserver._smb._tcp.local./time-machine
ID            : D820D053-C74A-4A06-A7E1-E60C8EA7934F
====================================================
Name          : time-machine-macbook
Kind          : Network
URL           : smb://user@127.0.0.2/time-machine-macbook
Mount Point   : /Volumes/time-machine-macbook
ID            : F707BD0B-64DF-4DB6-A3B7-824470FB5EB2

Then start a backup with tmutil startbackup and mount will show:

/dev/disk3s1 on /Volumes/time-machine-usb (apfs, local, nodev, nosuid, journaled, noowners)
//user@127.0.0.2/time-machine-macbook on /Volumes/time-machine-macbook (smbfs, nobrowse)

You can see detailed progress with tmutil status:

Backup session status:
{
    BackupPhase = Copying;
    ClientID = "com.apple.backupd";
    DateOfStateChange = "2018-06-28 17:38:21 +0000";
    DestinationID = "F707BD0B-57DC-4DB6-A3B7-824470FB5EB2";
    DestinationMountPoint = "/Volumes/Time Machine Backups";
    FirstBackup = 1;
    Percent = "0.08711567546702646";
    Progress =     {
        TimeRemaining = 32679;
        "_raw_totalBytes" = 355694600192;
        bytes = 34429528173;
        files = 887351;
        totalBytes = 391264060211;
        totalFiles = 2922384;
    };
    Running = 1;
    Stopping = 0;
    "_raw_Percent" = "0.09679519496336274";
}

In my case, near 1000000 files were done in about two hours (USB2 disk penalty, versus more than 8 hours via Wi-Fi), I calculate percentages with a small script:

tm-progress.sh
Files : 918702 / 2922384 (31.43%) - Bytes : 32.21 GiB / 364.39 GiB (8.84%)

We can here that many files can make Time Machine to progress "slowly" in Bytes, but nearly 1/3 of files have been copied.

Next step, when backup is finished, plug the usb drive to your realserver and copy the sparsebundle over the "uncompleted one" (or in the shared folder). Of course, Time Machine should be disabled during this step to prevent mounting a partially copied sparsebundle (and maybe corrupting it)

I have used an apfs drive to benefit of snapshots (to be able to revert to a previous "backupdb" in case of sparsebundle corruption, not tested yet)

Edit: even faster mode :

Once the backup has reached BackupPhase = Copying to make it faster you can make the backupbundle mount directly (without sending every operation through smb) by canceling the backup, renaming My-Computer.backupbundle to something like 1My-Computer.backupbundle, and creating a symbolic link :

ln -s /Volumes/time-machine-usb/1My-Computer.backupbundle /Volumes/time-machine-macbook/My-Computer.backupbundle

This tricks timemachine into following the link to the backupbundle localy, and not mount it through smb, which speeds up the backup. Make sure you move the backupbundle back when you have finished your local backup.

ıɾuǝʞ
  • 241
2

See my other answer first which is a lot simpler!!

This is Kenji's answer ("ıɾuǝʞ" above), reworked painfully and explained by a complete noob (myself) for other complete noobs. I also added an important workaround in the last step:

Better later than never, here is a hack to achieve this:

  1. Open Terminal.app and enter the following command:

    sudo ifconfig lo0 alias 127.0.0.2/32

  2. Plug in your usb drive

  3. Go to System Preferences / Sharing

  4. Open the lock in the bottom-left corner

    (Whenever asked for your password, go ahead and enter it)

  5. Tick "File Sharing" on the left pane.

  6. Tick Options button and activate both SMB and AFP
  7. Click the + sign in the middle pane (titled "Shared Folders") and locate your USB drive. Create a folder (e.g. "time-machine-macbook") and select it to create the shared folder.

  8. Then, add a destination backup (Time Machine will see it as a network share), using the following command in Terminal.app:

    sudo tmutil setdestination -a "smb://user:password@127.0.0.2/time-machine-macbook"

    (If you get an error that Terminal needs more access rights to your disk, then go ahead and follow the instructions of the error message to give such rights and repeat this step.)

    You can check if it's ok with the following Terminal command:

    tmutil  destinationinfo

    And you will get the following reply on your Terminal window:

    ==================================================
Name          : time-machine4
Kind          : Network
URL           : smb://kenji@realserver._smb._tcp.local./time-machine
ID            : D820D053-C74A-4A06-A7E1-E60C8EA7934F
====================================================
Name          : time-machine-macbook
Kind          : Network
URL           : smb://user@127.0.0.2/time-machine-macbook
Mount Point   : /Volumes/time-machine-macbook
ID            : F707BD0B-64DF-4DB6-A3B7-824470FB5EB2

  9. Then start a backup with Terminal command:

    tmutil startbackup

  10. Terminal command mount will show (among others) something like:

    /dev/disk3s1 on /Volumes/time-machine-usb (apfs, local, nodev, nosuid, journaled, noowners)
//user@127.0.0.2/time-machine-macbook on /Volumes/time-machine-macbook (smbfs, nobrowse)`

  11. You can see detailed progress with Terminal command:

    tmutil status

    Where you'll get something like:

    Backup session status:
{
    BackupPhase = Copying;
    ClientID = "com.apple.backupd";
    DateOfStateChange = "2018-06-28 17:38:21 +0000";
    DestinationID = "F707BD0B-57DC-4DB6-A3B7-824470FB5EB2";
    DestinationMountPoint = "/Volumes/Time Machine Backups";
    FirstBackup = 1;
    Percent = "0.08711567546702646";
    Progress =     {
        TimeRemaining = 32679;
        "_raw_totalBytes" = 355694600192;
        bytes = 34429528173;
        files = 887351;
        totalBytes = 391264060211;
        totalFiles = 2922384;
    };
    Running = 1;
    Stopping = 0;
    "_raw_Percent" = "0.09679519496336274";
}

    Of course you can also have a rough idea of the progress by opening Time Machine where you will get the total MB copied as well as a calculation of the time remaining.

    In Kenji's case, near 1000000 files were done in about two hours (USB2 disk), versus more than 8 hours necessary via Wi-Fi. My results were quite faster with a USB3 disk. In 2h I had backed-up more than 500MB.

  12. Next step, when backup is finished, plug the usb drive to your realserver and copy the sparsebundle over the "uncompleted one" (or in the shared folder). Of course, Time Machine should be disabled during this step to prevent mounting a partially copied sparsebundle (and maybe corrupting it). I have used an apfs drive to benefit of snapshots (to be able to revert to a previous "backupdb" in case of sparsebundle corruption, not tested yet)

  13. If #12 above doesn't work and Time Machine cannot backup TO the really-networked drive (but may possibly be able to read it), then the workaround is to copy the backup bundle inside a new sparsebundle! Awkwardly, Time Machine can use networked sparsebundles as backup destinations (but not it's own backupbundle!).

  14. Have your drive connected via USB (for speed) and use Disk Utility to create inside it a sparsebundle with the necessary size to fit all your backup. This might take a few hours depending on the size and speed, so choose to do it overnight. If you don't have enough drive space, then unfortunately you should remove your previously created backup and re-create it later on from the beginning inside the new sparsebundle.

  15. To create the new spasebundle, open Disk Utility, click File/New Image/Blank Image and in the pop-up window add a filename (e.g. BackupBundle.dmg), no tags, choose to be inside your USB drive root, choose a volume name (e.g. BackupBundle), size larger than your HD size, format definitely Mac OS Extended (Journaled), no encryption, Single partition - Apple Partition Map, and finally sparse bundle disk image. Check again your selected size (sometimes it gets back to default) and hit save.

  16. Hours later, when it's over, you may test your new sparsebundle over the network. Connect your drive to your router, open Finder, and double click your BackupBundle.sparsebundle. You should see a new disk on the sidebar and on your desktop called BackupBundle. Then open Terminal and see if the sparsebundle can be added as a backup destination. If not, then you can set it manually via Terminal.app:

    sudo tmutil setdestination -a  "/Volumes/BackupBundle"

  17. After all this, you can copy Time Machine's backupbundle inside your BackupBundle.sparsebundle. This will also take hours, so you may want to do it the next night. Alternatively, if you didn't have room in steps 14-15 and you had to delete it, you can set Time Machine to use it as backup destination when connected via USB and then again after the first full backup is completed to select the same destination via network.

Good luck!

NikoSilver
  • 21
1

The tricky thing about moving a Time Machine backup from a local hard drive to a remote server (or vice-versa) is that Time Machine uses a native HFS+ partition directly for local backups, but when backing up to a server over AFP, it creates a "sparsebundle" disk image.

You can use Disk Utility to make a sparsebundle image of an HFS+ partition, and then put that image in one of the server's shared volumes, but I'm not sure if there's anything else you'd need to do to the sparsebundle to make Time Machine recognize it and use it automatically. For example, you might have to name it exactly right, and there might be some kind of hidden files it looks for inside the sparsebundle or something.

Spiff
  • 110,156
0

Before you do anything, make sure your usb disk volume can be used as a backup destination by Time Machine AND at the same time the disk is readable and shareable by your router! Most routers only support FAT32 (which makes too small volumes) and NTFS format for large volumes, which is not supported by Time Machine!

Your disk may be perfectly readable and writeable via network, but Time Machine will NOT backup to it unless you create a sparsebundle (= something like a virtual disk) on your USB disk and use THAT as the backup destination!! Awkwardly, Time Machine can use networked sparsebundles as backup destinations but not it's own backupbundle which actually IS a sparsebundle too!!

But then the problem is, that if you connect the NTFS disk via USB to your Mac it will not be writeable (Mac does not fully support NTFS), and if you connect it via network it will be too slow for the first backup! The solution I resorted to was to use NTFS for Mac by Paragon. This made the NTFS disk writeable even via USB. It costs zero for the first trial week, and if you need to keep it for more then it costs €19.95. Not too much but we only need it for a day or so with this procedure here, so opt to buy it later if you want.

  1. Download NTFS for Mac from the following link and install it:

https://www.paragon-software.com/home/ntfs-mac/

If your router supports Apple formats (which is rare), you won’t need to do step 2 below, so jump directly to step 3. To check your router’s supported disk formats refer to its documentation (or google it).

  1. If your USB drive is not already NTFS formatted, then go ahead and create an NTFS partition big enough to fit your backup. The disk will lose all its data, so take care to back it up elsewhere if you need it! I chose to format the whole disk as NTFS. To do so, open Disk Utility.app, select the whole USB disk from the left and click the Erase button on top. Then select:

    • a name for the disk
    • Microsoft NTFS format (which appears only if you have correctly installed NTFS for Mac) and
    • GUID Partition Map so that it’s readable by your router

Hit the Erase button and it only takes a few seconds to finish.

Close Disk Utility and reopen it for the next steps.

  1. While your drive is still connected via USB to your Mac (for speed), open Disk Utility and use it to create a sparsebundle (again with the necessary size to fit your whole backup). This might take a few hours depending on the size and speed, so choose to do it overnight. To do so, in Disk Utility, click File/New Image/Blank Image and in the pop-up window fill in the following:

    • a filename (e.g. BackupBundle.dmg),
    • no tags necessary
    • choose in “where” to be inside your USB drive root
    • choose a volume name (e.g. same as the dmg if you want, so “BackupBundle”)
    • size: larger than your HD size - or if your USB disk is dedicated only for backups, then use the full disk capacity (You can write the size like a human being, e.g. “2TB”)
    • format: Mac OS Extended (Journaled)
    • no encryption (unless you know how to do it)
    • Single partition - Apple Partition Map
    • and finally “sparse bundle disk image”.

Check again your selected size (sometimes it goes back to default) and then hit save.

  1. Hours later, when it’s over, and while still the drive is connected via USB, open Finder, and double click your newly created “BackupBundle.sparsebundle”. A new disk called “BackupBundle” should appear on the sidebar and on your desktop!

  2. Now open Terminal.app and type the following command:

    sudo tmutil setdestination -a  "/Volumes/BackupBundle"

This adds (“-a”) our new sparsebundle as a backup destination to Time Machine. So open Time Machine Preferences to check that it’s there!

  1. Now, while the disk is still connected via USB (for speed), you can choose to “Backup now” by clicking the Time Machine little icon on the top of your screen. Alternatively you can tick Back Up Automatically from the left in Time Machine preferences and your backup will start in two minutes. Normally it will take several hours to finish (but not several days that it would take through the network!), so you may choose to do that overnight too.

  2. Now Time Machine's first backupbundle is nested inside your BackupBundle.sparsebundle on the USB disk! Make sure you normally “eject” both these disks by right clicking on them and selecting “Eject” and then unplug them from your Mac. Then connect the disk to your router, open it via Finder, and double click the “BackupBundle.sparsebundle”. Again, a new disk called “BackupBundle” should appear on the sidebar and on your desktop, and it is perfectly usable by Time Machine as a backup destination! Choose Back Up Now to make sure it is! If not, run the Terminal command of step 5 again.

Now your incremental backups which are a lot smaller than the first backup can be done automatically in minutes over the network!

NikoSilver
  • 21