Mike Bland

My MacBook’s hard drive decided to go insane this past weekend. The signs have been there for weeks, but I didn’t know how to recognize them. While trying to create a new Time Machine backup on my new network drive, Time Machine would suffer consistent and severe hang-ups in progress. While attempting to eliminate all other possible culprits, I realized that the hard drive was on the verge of complete catastropic failure. Disk Utility reported errors in the drive’s partition map, as well as the drive itself, that it couldn’t fix, and it was impossible to boot from the Lion recovery partition. Bad, bad news.

I’d started keeping both Time Machine and bootable backups about a year or so ago, and now, I’m sure glad I did. I didn’t lose anything. But, I didn’t do everything exactly right such that I could just plug in my new bootable backup hard drive and just go. So, for the sake of not having to scour through all my old notes the next time (!) this happens, and potentially for the benefit of others, here’s what I’ve learned about what to do, and what not to do when implementing a backup strategy for the Mac.

These steps are what I use for my Lion-based Mac.¹ They should largely work for many other Macs, with the exception of encryption for a USB drive/partition for versions before Lion (OS X < 10.7). For a extensive guide to Time Machine across all versions of Mac OS X, see Pondini’s website, though that site doesn’t include tips for setting up a custom sparsebundle for network-based backups.

• The Short Version
• How many backups?
• Creating a Time Machine backup on a USB drive
• Creating an encrypted sparsebundle for Time Machine backup over a local network
• Monitoring Time Machine backups
• Problems with Time Machine backups
• Setting up a bootable backup
• Fun with rsync on OS X
• Bootable backup script
• Booting from the backup
• Backing up before system updates
• Footnotes

The Short Version

If you just want the minimum backup protection—which will go a very long way, indeed—just plug a sufficiently-large USB hard drive and point Time Machine at it.

For minimum instructions on creating a bootable backup using command-line tools already available on the Mac, go straight to Jamie Zawinski’s Backups post. I embellish on it below, in Setting up a bootable backup, but his version is a lot shorter and gets to the point. Just three caveats:

• Make sure to format the replacement drive as HFS+J, possibly encrypted, with a GUID partition map, but do not make it case-sensitive, since there may still be critical programs you depend on which somehow rely on case-insensitive behavior. Consequently bad things may happen.² I realized this only as I was restoring my system, and instead of using my bootable backup to plug in and go, I had to do a fresh Lion install on top of it and then restore from my last Time Machine backup.
• If you want to mimic the way the original hard drive looks in Finder, name your backup Macintosh HD when you format the disk.
• I didn’t bother with cron, since I also use Time Machine.

There are other, graphical programs for creating straightforward bootable backups on the Mac, such as Carbon Copy Cloner (which uses rsync under the hood) and Super Duper. But, of course, I had to make things more complicated…

How many backups?

That depends. If you don’t mind the downtime from a hard drive failure and the time required to restore everything from a Time Machine backup, just use Time Machine. If you can’t afford to be down very long at all, go with a bootable backup. Either strategy is infinitely better than having no backups at all.

But if you want extra redundancy, or just want a reason to spend money and geek out, go with both, which is arguably the best of both worlds: In the case of a drive failure, you can install the new hard drive immediately (given the right physical tools, as described in the bootable backup section below) and then restore what you need from the Time Machine backup.

Actually, there is a case to be made for both in that Time Machine has more moving parts, and can produce obscure failures if you’re unlucky. Straight block-copy-based bootable backups, such as those created by using rsync, are far less error-prone. Time Machine gives you greater flexibility, but rsync gives you greater stability. Plus, for files you might want to access from other computers or systems besides your Mac, such as music and video files, rsync backups of just those files are the one best option.

Creating a Time Machine backup on a USB drive

The easiest and fastest way to set up a backup on the Mac is to plug in a USB drive and point Time Machine at it. Get a large-enough portable USB drive, one that’s at least twice the size of your hard drive. If the drive isn’t already formatted as HFS+ with journaling (HFS+J) and a GUID partition map, use Disk Utility to reformat the drive. If you don’t want to use the entire drive for Time Machine, use Disk Utility to create multiple partitions. You could use the hdiutil command-line tool instead for these tasks, but I haven’t used it for this particular aspect of the process, as Disk Utility is straightforward enough for this purpose.³

Plug it in, open the Time Machine pane in System Preferences, select the USB drive/partition, check the box for Encrypt backup disk if you like, and go do something else for a few hours. After that, subsequent backups will take only a few minutes, usually. If you’re paranoid (like me), get a second drive and rotate backups between the two—ideally rotating them between home and an offsite location, such as your office at work—but don’t worry about keeping the two drives in sync with one another. You could keep them synchronized using a block-level copy via Disk Utility or dd or Carbon Copy Cloner, but it’s of dubious added value.

Creating an encrypted sparsebundle for Time Machine backup over a local network

While the USB solution isn’t too onerous, it’s still nice to be able to back up using Time Machine over a local, WiFi network at home. It provides an added layer of security that backups are happening in the background on a regular basis without having to futz with physical drives and wires here and there. It serves as a redundant backup in addition to a USB drive. It’s easier for non-techincal folks in the house to deal with, since you can easily set them up with automatic backups without asking them to remember to do anything explicit or mess with a USB drive. Or, if you’re like me, it gives you an excuse to obsess over something else technical.

The host for my Time Machine network share is a Network Attached Storage (NAS) device which supports AppleTalk (not a Time Capsule). This share is mounted under the /Volumes/ directory, represented by $MY_NETWORK_SHARE in the command below. Time Machine can automatically detect and use such a compatible network share.

If you don’t care about encryption or size constraints, just select the share and skip the rest of this section. If you do care, you’ll need to create a sparsebundle by hand as explained below. Time Machine will not use encryption for network shares by default (it will for USB drives), and it will attempt to consume the entire size of the share. Creating a sparsebundle in advance will both enable encryption and set a hard limit on the space Time Machine can consume on the network share.⁴

To create such a sparsebundle, execute the commands below (assuming bash as the shell), substituting the following values as necessary:

• $MY_HOSTNAME: Open System Preferences > Sharing and observe/update the Computer Name: field.
• $MY_UUID: Open Apple Menu > About This Mac > System Report and look for the value of the Hardware UUID: field.
• $MY_TM: Set this to the name you would like for the mounted sparsebundle image, as it will appear under /Volumes/.
• $SIZE_IN_GIGS: Time Machine backups will be constrained by the size of the sparsebundle as specified by this value, even though Time Machine will report the entire size of the available free space on the NAS in the disk-selection screen.

$ cd /Volumes/${MY_NETWORK_SHARE}

$ hdiutil create -size ${SIZE_IN_GIGS}g -fs HFS+J -volname "${MY_TM}" \
  -type SPARSEBUNDLE -encryption AES-256 ${MY_HOSTNAME}

$ cat >${MY_HOSTNAME}.sparsebundle/com.apple.TimeMachine.MachineID.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
 "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.backupd.HostUUID</key>
	<string>${MY_UUID}</string>
</dict>
</plist>
^D

For versions of OS X before Snow Leopard, I believe, the process is slightly different; you don’t bother with the cat command above to create the .plist file, but just append the machine’s hardware address to the name of the sparsebundle thus:

$ MY_MAC_ADDR=$(ifconfig en0 | grep ether | awk '{print $2}' |
  tr -d ':')

$ hdiutil create -size ${SIZE_IN_GIGS}g -fs HFS+J -volname "${MY_TM}" \
  -type SPARSEBUNDLE -encryption AES-256 ${MY_HOSTNAME}_${MY_MAC_ADDR}

You can use Disk Utility in place of the above hdiutil commands, but it requires stepping through a few menus. hdiutil wins the simplicity battle in this case.

Use Finder to mount the new sparsebundle. Check the box for “Remember password in my keychain”. Once that’s done, eject the sparsebundle, and open the Keychain Access utility. Select the login keychain in the upper left, right-click the key that matches ${MY_HOSTNAME}.sparsebundle, and select Copy. Then select the System keychain, right-click in an empty slot in the keys pane, and select Paste. Now Time Machine will be able to unlock the encrypted sparsebundle.

A Google search for time machine sparsebundle turned up Time Machine on a network drive, which explains a lot of the above in greater detail, with lots of nice screenshots, but does not cover the Lion-style sparsebundle + plist file. This Mac OS X Hints article might’ve been where I got the tip, but I couldn’t figure out what web search I originally used to find it.⁵

Monitoring Time Machine backups

Before launching a new Time Machine backup, I like to open the Activity Monitor utility to monitor disk and network activity, and start log tails so I can see the log messages as they stream in, grabbing a little back history to try to get a little context on the previous run (if it’s there):

$ tail -f -n 1000 /var/log/system.log | grep backupd

You can also do the above via the Console app, but I prefer to keep all the log monitoring command-line based, since that’s what I’m comfortable with. I use tmux to split the terminal window into panes, so I can easily see several different log tails at once.

Now launch the Time Machine backup. The log tail should show output like:

Sep 23 12:13:45 MY_HOST com.apple.backupd[1162]: Starting standard backup  
Sep 23 12:13:46 MY_HOST com.apple.backupd[1162]: Attempting to mount network  destination URL: afp://…`...._afpovertcp._tcp.local/MY_NETWORK_SHARE
Sep 23 12:13:47 MY_HOST com.apple.backupd[1162]: Mounted network destination at mountpoint: /Volumes/MY_NETWORK_SHARE using URL: afp://...@...._afpovertcp._tcp.local/MY_NETWORK_SHARE  
Sep 23 12:14:11 MY_HOST com.apple.backupd[1162]: QUICKCHECK ONLY; FILESYSTEM CLEAN  
Sep 23 12:14:13 MY_HOST com.apple.backupd[1162]: Disk image /Volumes/MY_NETWORK_SHARE/MY_HOST.sparsebundle mounted at: /Volumes/MY_TM  
Sep 23 12:14:13 MY_HOST com.apple.backupd[1162]: Backing up to: /Volumes/MY_TM/Backups.backupdb  
Sep 23 12:14:14 MY_HOST com.apple.backupd[1162]: NN.NN GB required (including  padding), NNN.NN GB available

You can also view the Time Machine log file directly:

# This should be all on one line with no break in the path.
$ sudo cat /Volumes/${MY_TM}/Backups.backupdb/${MY_HOSTNAME}/
  *.inProgress/.Backup.*.log

You’ll want to use cat or just tail as opposed to tail -f since you don’t want to keep any files open in the backup directory, lest Time Machine fail to unmount the network volume when it’s finished with the backup. You may want to substitute the * glob characters with the specific names matching the lastest backup and file, but it’s not strictly necessary, and more typing. Computer time is cheap relative to human time nowadays—at least in this case.

Problems with Time Machine backups

If things seem stuck for a disproportionate amount of time, you can use lsof to see what file the backup daemon is currently trying to read:

$ sudo lsof | grep ^backupd | grep -v /dev | grep ' [0-9][0-9]*r '

On my machine this past weekend, even after installing a new hard drive, Time Machine continued to get stuck on backing up files from Xcode, in the /Developer/ directory. Not completely stuck, but it was moving very slowly. I deleted the Xcode entry from my Purchased apps in the Mac App Store, then deleted Xcode from Launchpad, but that wasn’t enough; I also moved the entire /Developer/ directory to the Trash and emptied it.

With Xcode gone, I restarted the backup. It still got hung; after a few web searches, I found out that disabling Spotlight was the solution, at least until the problems with OS X 10.7.5 are resolved:

# To disable Spotlight:
$ sudo launchctl unload -w \
  /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

# To reenable Spotlight:
$ sudo launchctl load -w \
  /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

After this, things moved along swimmingly. Then I re-installed Xcode and backed up again.

Pondini’s Time Machine troubleshooting page has extensive information on various Time Machine-related issues across all versions of OS X.

Setting up a bootable backup

First, get at least one decent replacement drive at least as big as your current one, two if you can afford it, including enclosures. For my MacBook, I sprung for WD 320 GB Black SATA II drives and Vantec NexStar TX 2.5-Inch SATA to USB 2.0 enclosures. I also got a #00 Phillips screwdriver for my MacBook screws, but the Vantec enclosures actually come with one. You’ll also need a T6 screwdriver for the screws anchoring the hard drive inside the chassis, for when the time comes to actually swap out the drive in the MacBook.

Once you’ve installed the drive in its enclosure and plugged it into your machine via USB, use Disk Utility or hdiutil to format the drive using the HFS+J file system—not the case-sensitive version, as I mentioned above. Use the entire drive or partition it however you like, just make sure you’ve enough room for all the data and programs you plan to keep. Once that’s done, mount the new drive if it isn’t mounted already (unplug it and plug it back in). Right-click/control-click the drive’s icon on the desktop and select Get Info (or just click the disk and press Command-I). At the very bottom of the Info window, click the little lock, enter your password when asked, and deselect “Ignore ownership on this drive”, also near the bottom of the window.⁶

Now from here, you can do one of three things:

• Use jwz’s rsync command as published, as mentioned above.
• Use Carbon Copy Cloner or Super Duper, as mentioned above.
• Apply OS X-compatibility patches to the rsync source code to build a new rsync and use that.

Fun with rsync on OS X

Guess which one I picked. When I was first messing around with backups, I was trying to avoid Time Machine and implement my own incremental archive system using rsync,⁷ and hit some limitations of the somewhat dated version that ships with OS X. I discovered that the author of Carbon Copy Cloner had actually helped to create these patches to overcome these limitations. As per Rsync 3.0.9 for Lion (which didn’t exist when I figured this out):

$ curl -O http://rsync.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz
$ gzip -dc rsync-3.0.9.tar.gz | tar xf -
$ curl -O \
  http://rsync.samba.org/ftp/rsync/src/rsync-patches-3.0.9.tar.gz
$ gzip -dc rsync-patches-3.0.9.tar.gz | tar xf -
$ cd rsync-3.0.9
$ patch -p1 <patches/fileflags.diff
$ patch -p1 <patches/crtimes.diff
$ patch -p1 <patches/hfs-compression.diff
$ ./prepare-source
$ ./configure
$ make
$ sudo make install

This should install /usr/local/bin/rsync, and you’ll have to make sure to specify the full path when executing it, lest you get the older /usr/bin/rsync instead. Or, you could hack your $PATH to put /usr/local/bin before /usr/bin if it isn’t set up already; execute which rsync to see which version you get.

Bootable backup script

I like to write tiny shell scripts for relatively long or complex commands I’ll execute with some frequency. Here’s my script for performing the rsync to update a bootable backup (assuming I named the bootable partition Macintosh HD, and it’s mounted as /Volumes/Macintosh HD 1):

#! /bin/bash
#
# Adapted from http://www.jwz.org/blog/2007/09/psa-backups/
#
# Make sure to Get Info on the drive and un-check "Ignore ownership on
# this drive" under "Ownership and permissions" before running.

sudo tmutil disable
sudo /usr/bin/time /usr/local/bin/rsync -vaxAX --delete \
  --ignore-errors --exclude=.MobileBackups/ --exclude=/private/var/vm/ \
  / /Volumes/Macintosh\ HD\ 1/
sudo tmutil enable

Since I’m also using Time Machine, the script uses tmutil to turn Time Machine backups off before running rsync, and turns them back on afterwards; it’ll prompt for your password each time if you haven’t been running sudo in another terminal around the time you’re running the backup script. I exclude a couple of directories that aren’t very useful for a bootable backup. Like with Time Machine, the first backup will take several hours to run; subsequent backups take on the order of 15-20 minutes. You could probably set up/modify this script so you can execute it automatically via cron—you’d have to run it as an administrator to execute the tmutil commands—but I just run it by hand every couple of days or weeks, since I can use my most recent Time Machine backup to restore more recent versions of programs and files that have changed since the last run of the script.

Booting from the backup

Once your bootable backup is finished, leave it plugged in and restart your Mac. After the screen blanks out, just after it comes back on again, hold down the Option key. You should see a graphical selection menu showing both your Mac’s hard drive and the bootable USB disk. Use the arrow keys to select the USB disk, and make sure the system boots from it successfully. If it does, congratulations! As long as you update the bootable backup somewhat regularly, if your Mac’s drive ever does fail, you should be able to remove the backup from its enclosure and replace the old hard drive with it and hit the ground running from there.

If not… Reboot from your Mac’s hard disk, and use Disk Utility to try to verify and repair the bootable backup drive, via the First Aid tab. If no errors are found, or if all errors are fixed, repeat the steps to create a bootable backup. If Disk Utility encounters errors it can’t fix, perhaps you got a dud hard drive, and should return it for a new one.

Backing up before system updates

If there’s one other thing I wish I’d thought to do before this weekend, it’s to have made both Time Machine and bootable backups before installing the latest OS X system update. Applying that principle in general means that if any issues with the update do arise within a few days or a couple of weeks, it’s relatively easy to either revert to the previous system version, or to plug in the bootable backup and go from there. Sometimes Apple does get it wrong, as evidenced by the 10.7.5 Spotlight/Time Machine performance issues mentioned above.

Footnotes