Macintouch reports: slide and tape digitization

These MacInTouchreports are always excellent. Great community:

We have a great report today about Digitizing Slides collected as part of a family project and another report with lots of advice about Digitizing VHS Tapes and converting them to DVD.

I scanned the slide digitizing report. Walmart was cheapest for digitizing slides and one guy found the most efficient approach was to photograph the slide projection (I'd never have thought of that one). This is a market problem that will eventually have a better solution. I'm amazed we still don't have bulk solutions for print imaging -- a much easier problem. Maybe later this year or next year ...

Ars Technica reviews: iPod speaker systems

High-end iPod speaker systems

The Apple iPod Hi-Fi did surprisingly well. If compactness is important I'd say it's the winner, but the Klipsch outpointed it. Nice review. I'm tempted ...

GMail's been eating comment notifications!

Gordon's Notes: Missed comments: Gmail eating them!

Very annoying. Sorry!

I hate Palm. Outlook too. Oh yeah, also Microsoft.

Another fight lost in the HotSync wars.

Today Outlook/Palm/Exchange/BeyondContacts (OPEB) duplicated all of my tasks. I'm so used to cleaning up this nonsense I have special Outlook views defined to help the cleanup.

Why did it do this? Beats me. Every so often OPEB decides to punish me.

I hate sync. I hate Palm. I hate Outlook. Exchange too for that matter. Oh, yeah, Microsoft too.

Sync is hard enough when a single vendor has complete control of both transaction partners. When sync crosses vendors (bilateral messaging, reconciling disparate data models, lack of well managed identifiers), and when sync is three-way (PDA to Outlook to Exchange) it's like juggling antimatter. Every so often, it explodes.

BTW, most healthcare environments do the equivalent of "sync" across vendor systems every day. It's ugly there too.

PS. When I first tried to post this I got a "blogger has gone splat" message (ok, they said "down for planned maintenance -- at 6pm CT). I hate Blogger too. Also Google.

iBook: enhanced batteries - TruePower

Macintouch had this one:

TruePower Apple PowerBook & iBook Batteries

FastMac introduced a new line of TruePower high capacity batteries for the PowerBook G4 (Titanium and Aluminum), iBook (all models), and PowerBook G3 (Wallstreet, Lombard and Pismo). The TruePower batteries claim capacity up to 56% more than the Apple originals (depending on model) and include safety features to protect against overcharge, short circuit, and power surges, along with a one year warranty. Introductory prices range from $99.95 for most models to $119.95 for the PowerBook G4 Aluminum 17" and PowerBook G3 (2000) Pismo models.

Aperture woes - and iPhoto 6 image export

[PS. If you have trouble reading this, note that Blogger's formatting and editing tools are astonishingly broken. Don't use bullets!]

Apple - Support - Discussions - Aperture performance vs. iPhoto ... sums it up. Aperture 1.1 is just too darned slow save on the very highest performance machines and the very highest end GPUs. It's an application for the next generation of Macs. The only caveat is that there may be a way to make it workable by constraining views to smaller thumbnail subsets.

So now I'm on Plan B. This is using Adobe Elements for initial RAW and JPG image review and edits, then export as JPG 99% and store in iPhoto 6. Alas PNG won't work since it strips metadata and JPEG2 won't work because of unclear metadata standards and an uncertain future. TIFF 6/EXIF with compression or DNG (basically TIFF 6 with better metadata) is the other option, but when iPhoto edits TIFF I think it stores the edited file as uncompressed TIFF -- huge.

In the meantime I did some experiments with iPhoto image formats and export/import. I used GraphicConverter 'save as' to make some of these inferences. I'll update what's below with more testing:

• If I import RAW from my Digital Rebel XT into iPhoto 6.02/10.4.6 the resulting image uses the Adobe RGB color space.
• iPhoto image export does not include an image preview (good!)
• If I edit a RAW file in iPhoto the edited file format is JPG. Oddly enough iPhoto has a "reprocess RAW" option, which can only transform the original, not the JPG. I assume that's related to how RAW processing occurs in Aperture. [Note: There's an advanced iPhoto option to save edited RAW as 16 bit TIFF. This would not only change this behavior, I suspect export original might export the edited TIFF rather than the CR2 file. Since 16bit TIFF files are enormous I didn't do this.]
  
• When iPhoto exports as PNG, JPG or TIFF it exports the edited image. When it exports as "Original" it exports the edited image for PNG or JPG, but the original image for RAW/CR2. The latter makes sense when one realizes that iPhoto always converts CR2 to JPG for editing, there's no "edited CR2" to export. So the app has to choose between exporting a JPG (clearly wrong) and exporting the original (not so clearly wrong, but not right either).
  
• When iPhoto exports TIFF the metadata is well preserved on round-trip but there's no compression at all. A 5-6 MB PNG vs a 22MB TIFF!
• When GraphicConverter converts TIFF to JPG (resource fork turned off, the fork adds about 100K, use QuickTime for compression) there's very little growth in file size from a 90% to 100% image quality setting. When using GC to produce images that will be archived, there's no reason to drop below 100% quality.

Some interesting comparisons on an exported image from iPhoto (image sizes in millions of bytes, not in MBs). Note that Preview is an amazing little application, but it doesn't let one save CR2 or 16bit TIFF as anything but lossless.

TEST ONE

Original CR2: 13.27 converted to:

JPG w/ iPhoto: 4.1
PNG w/ iPhoto: 13.1
TIFF 8 bit w/ iPhoto: 23.9
TIFF 16 bit LZW w/ Preview: 62.23 (Note that CR2 RAW images have 12 bits of color. JPG only has 8. TIFF has either 8 or 16; one sacrifices date, the other wastes disk space.)
PSD w/ Preview(!): 47.7

The 8bit TIFF converted to:

JPG BEST w/ Preview: 8.56
JPG 100 w/ GC: 4.11
JPG 99 w/ GC: 4.03
JPG 95 w/ GC: 4.01
JP2 Lossless: 8.63

TEST TWO

CR2: 11.1
JPG w/ iPhoto: 2.9 (23.83 as TIFF)
JPG in camera: 3.54 (23.89 as TIFF)

Wow. This surprised the heck out of me. A few comments as I figure this out:

Preview has a lot of unexpected capabilities. Most apps, including GraphicConverter, can't fully handle 16 bit TIFF, but it has no problem.

I'd figured from my first test that iPhoto's JPG export was minimally lossy, the only less lossy export coming with Preview -- it seems to save as lossless JPEG (rare format). In my second test the in camera JPEG is significantly larger than the out-of-camera iPhoto CR2 to JPEG result. I wondered then if the camera was losing less data than the iPhoto CR2 to JPG conversion. However, when I normalized both by using GC to save as TIFF, the sizes were 23.89 and 23.83 respectively. In other words it in-camera JPEG is simply a less efficient compression, it holds no more real data than the JPEG iPhoto produces from the CR2 file.

More conclusions later, but for now:

CR2 is a very compact format. In terms of capturing all image data, the next jump is to PSD. Almost 400% growth required.

iPhoto's JPG output is better than I'd expected. It's the equivalent of GraphicConverter's 100% option. Also to my surprise, there's no reason to use anything less than 100% when saving images in GC -- file size savings from greater compression are minimal.

Preview may support lossless JPG, which is almost unheard of. JPEG2000 lossless gives similar results.

iPhoto is a very sophisticated product.

My new workflow is as follows:

Use RAW in my camera.

Import images from camera into an 'iPhoto Intake Library' (I use IPLM to switch libraries).

Edit in the Intake Library. When done export as JPG. Note, however, images will have Adobe RGB color space rather than my preferred sRGB.

Rename the exported images using my personal convention (date/roll info is part of file name, no possible future name collisions).

Import into iPhoto for storage.

My primary concern with this is that I'm working in Adobe RGB rather than, as I prefer, sRGB.

Why PNG sucks: It's the metadata, stupid

I've been experimenting with various file formats to use with iPhoto and RAW images. iPhoto generated TIFF files completely uncompressed and thus quite large, iPhoto doesn't provide JPG fine tuning (lossy compression control), JPEG2000 is still not widely used, etc. PNG seemed a good option -- reasonable sizes, well specified, decent color tables, etc.

But PNG has a lethal flaw, as documented by the Library of Congress digital preservation project:

PNG

The PNG specification allows labeled text (ASCII or UTF-8) elements to be embedded in text chunks and predefines a few standard keywords(element labels): Title, Author, Description, Copyright, Creation Time, Software, Disclaimer, Warning, Source, Comment. The compilers of this resource are not able to assess the degree to which such metadata is found in practice or whether other keywords are in common use. An attempt in 2000 to develop open source tools to convert EXIF images (including EXIF metadata) to PNG seems to have been abandoned. See
http://pmt.sourceforge.net/exif/drafts/d020.html. Without such tools and agreed practices, PNG can not rank highly for self-documentation.

In other words, there are no standards, or even exif-like pseudostandards, for embedding metadata (time image acquired, etc) in PNG images. Obviously there are no standards in common use for associating a portable metadata XML file (for example) in a bundle with a PNG image.

So 'save as PNG' means toss out the metadata. PNG is worthless as an image archival format. Shame.

BTW, this LOC site is an exception reference. In contrast to the PNG tragicomedy, note the discussions of JPF (still in the twilight zone) metadata ("self-documentation"). There's some very sophisticated thinking there, unfortunately no-one supports this yet ...

All JPEG 2000 files are made up of "boxes," as described in the Notes below, including an XML box typically used for metadata. Regarding JPX_FF, Annex N of Part 2 of the specification provides detail about metadata and offers but does not require a specification based on DIG35 elements. This metadata specification includes four broad metadata categories: (1) image creation ("how," e.g., about the camera), (2) content description ("who," "what," "when," and "where"), (3) history ("how the image got to its present state," i.e., provenance metadata in the digital library lexicon), and (4) intellectual property rights (IPR) metadata (which may be used in conjunction with technological protection systems). Additional boxes inherited from JP2_FF include one for a unique identifier for the image or identifier-references to other digital objects, e.g., a UUID, and another for IPR metadata, possibly redundant with that included in the XML box.

JPF wraps JPEG2000 compression formats, which are proprietary and have lots of IP issues. Contrast now to Adobe's open DNG (described as a "subtype of TIFF 6"):

See TIFF_6. Additional metadata may be embedded in a file using tags from (1) TIFF/EP or EXIF_2_2 (see also TIFF_UNC_EXIF), (2) IPTC (TIFF tag 33723), and (3) XMP (TIFF tag 700). Regarding TIFF/EP and EXIF, the specification states that TIFF/EP stores the tags in IFD 0, while TIFF_UNC_EXIF stores them in a separate IFD. Either location is allowed but the EXIF location is preferred. Proprietary metadata that may be used by camera manufacturer's raw convertors is to placed under private tags, in private IFDs (Image File Directories), and/or a private MakerNote. (pp. 12-13)

TIFF 6 appears to be the preferred format for image archiving by the Library of Congress. I can't tell what version of TIFF iPhoto produces. It's not compressed, but that doesn't say much. It does contain EXIF metadata, so maybe it's TIFF_UNC_EXIF.

On review I'd say we need iPhoto to export to DNG with compression and use DNG as a native lossless format.