Sunday, January 04, 2004

Panther's Best: Sparse Disk Images on an SMB share

Apple - Discussions - Disk Images on an SMB share
Other than a more robust file system, the single best feature of Panther for my purposes may be the ability to host a sparseimage (sparse disk image) on an SMB share. Jaguar did not allow sparseimage nor regular HFS+ disk images to be mounted when hosted on an SMB share unless the images were quite small.

[Addendum 1/9/04: From a MacDev article - FileVault uses sparse image -- "FileVault uses a special disk image format: USDP or SPARSE. These files have a .sparceimage extension. Their specificity is that the resulting volume expands as needed to accommodate more data without requiring you to manually create a new image and copy data back and forth -- or requiring the system to do so."]

Using Disk Utility one can create a sparse image (not a read/write image) on a local drive or a share, including an SMB share. The file created has the extension "sparseimage". The size one specifies in the create image dialog is the MAXIMUM size, the actual file created is quite small -- about 4MB where the maximum size is DVD size (4.7 GB). Even over an 802.11b WLAN connection it only takes a few minutes to create and format the image.

Once the image is created and mounted one can copy files to it. I experimented with copying a 500MB iPhoto Library to the disk image. The sparseimage file grows only as needed to incorporate the added files, so the physical file grew 500MB after copying. (The Finder reports 4.2GB free, as far as the Finder knows this is a 4.7GB disk.)

The image can be quickly mounted and dismounted. Using a G3 iBook and a slow 802.11b connection the 500MB sparseimage (4.7GB capacity) mounts in seconds.

Once mounted it is accessed like an HFS+ share. Using iPhoto Library over the effective 2mbps (802.11b) connection I loaded the 500MB library into iPhoto 2.0. Access was quite fast with a delay of a few seconds when displaying a full screen image for the first time. I expect there'd be no perceptible delay on a 100mbps wired LAN.

Why is this so valuable? SMB storage is very cheap. Any ancient box running Win2K and some dirt cheap IDE controllers easily host 5 or so inexpensive 120GB drives. Unfortunately the OS X SMB client cannot take full advantage of this. Problems with character encoding and perhaps permissions mean some HFS+ data cannot be placed on an SMB share. In particular iPhoto 2.x libraries, which can require a large amount of storage, cannot be moved to an SMB share. (If you try you will get messages that some files cannot be copied.) They can, however, be quite safely copied into a disk image on an SMB share. One might consider the same approach for iTunes, though I've been hosting my iTunes files on an SMB share using the standard approach without known problems.

The main drawback is backup. If one is doing backup via the SMB host then changing a single bit in a file on the disk image will alter the disk image. Even the kind of disk based backup I use [1] will run out of space. I will probably exempt the disk image from my standard backup and just copy it weekly to my backup store, while doing monthly DVD burns of the sparseimage to provide serial backup. I will experiment with Retrospect server to see if Retrospect will do a file level backup if I simply leave the image mounted on my laptop overnight.

I await reports from other experimenters!



meta: jfaughnan, jgfaughnan, OS X, diskimage, disk image, sparse image, sparse disk image, SMB, windows, share, network, iPhoto, backup, server, storage management, Panther, 10.3, 10.3.2

No comments: