How can I delete Time Machine files using the commandline

Question

I want to delete some files/directories from my Time Machine Partition using rm, but am unable to do so. I'm pretty sure the problem is related to some sort of access control extended attributes on files in the backup, but do not know how to override/disable them in order to get rm to work. An example of the error I'm getting is:

% sudo rm -rf Backups.backupdb/MacBook/Latest/MacBook/somedir
rm: Backups.backupdb/MacBook/Latest/MacBook/somedir: Directory not empty
rm: Backups.backupdb/MacBook/Latest/MacBook/somedir/somefile: Operation not permitted

There are a number of reasons I do not want to use either the Time Machine GUI or Finder for this. If possible, I'd like to be able to maintain the extended protection for all other files (I'd like not to disable them globally, unless I can re-enable once I've done my work).

Answer 1

To work around "operation not permitted" errors, use the Time Machine Safety Net "bypass" program:

sudo /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass rm -rfv /Volumes/[disk]/Backups.backupdb/[path]

In 10.8 Mountain Lion, bypass moved into 'Helpers':

/System/Library/Extensions/TMSafetyNet.kext/Helpers/bypass

In 10.10 Yosemite, bypass moved here:

/System/Library/Extensions/TMSafetyNet.kext/Contents/Helpers/bypass

Beware when using this to delete specific snapshots: as Time Machine uses hard links, using rm -r on folders might also affect older and newer snapshots of the same machine. (See other answers referring to tmutil delete to safely delete a specific snapshot.) Using rm to delete all snapshots for a single machine is okay though. And so is using rm to delete a specific file, which will only remove that hard-linked file from the snapshot(s) you specify, assuming the file is not in a hard-linked directory as then you'd actually remove the file from all those hard-linked directories.

Answer 2

BLUF (bottom line up-front):

sudo tmutil delete _snapshot-dir_

---

Using sudo [chmod][chmod(1)] -R -N _folder_ to remove all ACLs from a folder hierarchy doesn't work on the files and folders in Time Machine's Backups.backupdb, because of the TM Safety Net mechanism and the criteria described in this 318 Tech Journal post (but possibly not exactly as described).

(Before learning this from looking up the Safety Net mentioned in Eric W's answer (which works), I had only tested on a folder cloned from a subfolder of a TM backup, and there chmod worked. But trying chmod on a folder in an actual TM backup gives the "Operation not permitted" error.)

Of possible use:

In Mac OS 10.7+, there's a tmutil command (which I haven't tried, since I'm still on Snow Leopard). It has a delete verb, which according to the description "can delete snapshots from backups that were not made by, or are not claimed by, the current machine" (where a "snapshot" is a dated folder representing a single incremental backup). It isn't clear to me if this means that it can't delete snapshots which are made by or claimed by the current machine.(?)

Answer 3

A warning about using the bypass command to remove an old backup: if the deleted backup has folders that are exactly the same in earlier or later backups, then files might be deleted from earlier or later backups as well!

Time Machine not only uses hard links for unchanged files, but also uses hard links for folders in which no files were added, changed or deleted at all. This results in something like:

/2014-11-06/folder/file1
                  /file2
                  /file3
/2014-11-13/folder/file1 = hard link to file /2014-11-06/folder/file1
                  /file2 (changed; new inode)
                  /file3 = hard link to file /2014-11-06/folder/file3
/2014-11-20/folder/ = hard link to folder /2014-11-13/folder/
/2014-11-27/folder/ = hard link to folder /2014-11-20/folder/

With the above, deleting any file from /2014-11-06/folder/ is fine, and only affects the backup for that date. The hard link reference counts are decreased, so the "inode" for file2 will be removed, but inodes for file1 and file3 will still have a reference count of 1 due to the later backups. Hence, rm -R /2014-11-06 is fine too.

However, removing any file from either /2014-11-13/folder/, /2014-11-20/folder/ or /2014-11-27/folder/ will effectively remove it from all those 3 folders.

The problem is that rm -R does not care about hard-linked folders. It just recurses into any hard-linked folder it finds, boldly deletes all its files, and then removes the empty folder.

So: when removing an old backup one should not recurse into a hard-linked folder and delete its contents. Instead, one should only remove the hard link for the folder itself. So, rather than rm -R use tmutil delete as explained in Arne's answer.

As an aside, it seems that the OS X unlink command cannot be used on folders: "only one argument, which must not be a directory, may be supplied". The OS X API can remove hard-linked folders, and so can GNU Coreutils, like installed using Homebrew.

Finally, to prove all of the above, a test-case (OSX 10.6.8):

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-06-012454/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-20-014044/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-27-025033/Users/USERNAME/Library/Safari/TopSites.plist

Note that the number of links for each occurrence is 2 (second column). Let's remove the first occurrence:

sh-3.2# /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass unlink 2014-11-06-012454/Users/USERNAME/Library/Safari/TopSites.plist 

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-20-014044/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-27-025033/Users/USERNAME/Library/Safari/TopSites.plist

So, after unlinking one of the files, the number of links dropped to 1 for each occurrence, though the file is still shown 3 times. No problems yet. Remove the first occurrence again:

sh-3.2# /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass unlink 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist 

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
ls: 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist: No such file or directory

Now all are gone. Apparently the file TopSites.plist was last changed 2014-11-06 and hard-linked on 2014-11-13 as then some other files were added, changed or removed in the Safari folder. Next, the contents of the Safari folder did not change in the subsequent two backups, so on 2014-11-20 and 2014-11-27 the Safari folder was hard-linked to the previous backup.

Indeed, the 4 folders only use 2 inodes (first column):

sh-3.2# ls -lFaid 2014-11*/Users/USERNAME/Library/Safari/
648651968 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:06 2014-11-06-012454/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-13-024438/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-20-014044/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-27-025033/Users/USERNAME/Library/Safari//

Answer 4

Time machine works like rsnapshot. It creates a tree of hard links for each new backup. Hard links to files already existing in a previous backup use very little additional space. Only when the last hard link to a file is removed is the file actually deleted from the filesystem.

Removing an entire individual backup won't hurt. You're just removing hard links. No other backup will be affected. But that can be accomplished via tmutil.

One scenario in which it may be necessary to bypass protection is to remove a specific file from all backups (and the reason why I ended up on this post).

My backup disk is full. I have a very large file (many gigabytes) that has been backed up for months. There is one physical copy of it, but many snapshots with hard links to that copy. To actually get rid of that file, I need to remove the hard link from every backup.

Note that the inode number is the same for all hard links to the same file.

% cd /Volumes/WD\ 500G\ USB/Backups.backupdb/csm-laptop
% ls -li */Macintosh\ HD/Users/csm/vm.img
...
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-005636/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-015812/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-030036/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-041307/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 Latest/Macintosh HD/Users/csm/vm.img

(Latest is just a symlink to the last dated directory)

% sudo bypass rm -f */Macintosh\ HD\Users\csm\vm.img

The file is removed from all backups, and space is returned. If the file has been changing over time, each backup will have a full copy and the space returned will be enormous.

Answer 5

Note: Due to the "TM Safety Net" mentioned by Eric W, this answer doesn't work for the case of a Time Machine backup, which the question specifically pertains to. But for most every other case, the information about how to get rid of ACLs is relevant.

---

There's no need to use ACL tools copied from an older OS.

Use ls -le to view ACLs and chmod to alter them.

For more info, type man chmod and look under ”ACL Manipulation Options” .

The command to remove all ACLs from a folder hierarchy is:

chmod -R -N foldername

Answer 6

If you're not executing the command as the user that "owns" the backup, you're going to have a hard time deleting from the command line. I just had this issue with a migration, and we had to ditto the entire Time Machine backup (1tb+) and format the drive before we could get any sort of access to it - and trust me, I tried everything to override the permissions.

Answer 7

I needed this combination and order of commands for removing leftover files while removing a time machine directory on an external drive. Only use these if no other option worked for you.

1. Start in Single User Mode (hold CMD+S on boot)
2. sudo chflags -R nouchg <path to some.backupdb>
3. sudo chflags -R noschg <path to some.backupdb>
4. sudo chmod -R ugo+rw <path to some.backupdb>
5. sudo chmod -R -N <path to some.backupdb>
6. sudo rm -rf <path to some.backupdb>

Explanations: Why does chown report "Operation not permitted" on OS X?

Answer 8

If you want to delete all files in a folder and not just specific files, you can accomplish this by adding the folder to Time Machine's exclusion list. (System preferences -> Time Machine -> Options. Drag the folder here.)

Next time you perform a backup, copies of that folder will be removed from previous backups.

Now, if you really want to do this from a CLI, there's a way, albeit a little cumbersome.

1. Make a backup of /Library/Preferences/com.apple.TimeMachine.plist
2. Copy /Library/Preferences/com.apple.TimeMachine.plist somewhere where you can toy with it.
3. Cd into whereever you put it.
4. Execute
   plutil -convert xml1 com.apple.TimeMachine.plist
   to convert it from binary form.
5. Open the converted plist in preferable text editor, search for "skippaths"
6. Insert a new line in that section, formatted as <string>/Path/To/Exclude</string>
7. Save and exit, convert back by executing
   plutil -convert binary1 com.apple.TimeMachine.plist
8. Copy your edited plist back into /Library/Preferences/
9. Start a backup by executing
   /System/Library/CoreServices/backupd.bundle/Contents/Resources/backupd-helper -auto

Edit: When you perform step 9, all copies of the newly excluded folder will be erased from previous backups.

To remove the exception, copy your backup back into /Library/Preferences.

Answer 9

You can make ls list extended attributes in a long view using the -@ flag. It will list ACLs when you supply the -e flag. So, you can find out what you're dealing with by using ls -lea@ DIR.

Judging by my local Time Machine backups, it looks like Time Machine applies extended attributes with metadata about the newest and oldest snapshots. The data stored by the xattrs looks to be a binary plist. These seem innocuous.

Time Machine also looks to apply ACLs to certain directories it knows about, such as the ones placed in a standard user directory. There are two kinds of ACLs potentially standing in your way: those applied directly to the file or directory that deny delete, and those applied to a parent of the file that deny delete_child.

Unfortunately, Mac OS X does not supply the user utilities getfacl and setfacl specified by POSIX.2c to view and manipulate ACLs. To mess with ACLs, you'll have to do some programming; see the acl(3) manpage.