Managing video files with the Finder and Spotlight

Imagine, for a moment, you have been engaged in months of ugly struggle with PureDigital's Flip Camcorder, 3ivx's codecs, QuickTime, AppleScript, Automator, OS X, iMovie HD, iMovie '08 and too many other bits of software to name.

Imagine then that QuickTime 7.45, OS X 10.4.11 and the 3ivx codecs PureDigital's Flip camera uses stop working together. (thank you Perian)

It's not just me. Apple's been making lots of customers miserable. I think we're all entitled to free versions of OS X, by hook or crook, for all eternity.

So I was ready to try something very simple for managing our video fragments (there's no equivalent of EXIF for video btw):

Here's how it works with the Flip camcorder:

1. Drag them off the camcorder into my custom video folder (see custom Finder column settings in image below, the built-in software no longer works since I had to remove the official non-functioning Flip codec).
   
   
2. Using 'A Better Finder Rename' I rename the videos to the form: YYMMDD HH MM_##.AVI where the YYMMDD is their last modified date (basically the time Flip stopped writing to the AVI file).
   
   
3. As per Image 1 (below) I click on file and then do cmd-opt-I to open the "Inspector". Now when I go from image to image I can see a thumbnail of the video, click to play the video in the Inspector, and type descriptive comments that Spotlight will search (Spotlight Comments in Inspector). These are part of the standard OS X file metadata. I can also rename the image from the Inspector. No extra clicks or delays. Videos I don't want get a "d" in comments, I delete them later. (It's not shown here, but I've since customized my labels to correspond to the usual 1-5 star rating, renamed labels with the ratings in Finder options, and display the label name in the custom folder view.)
   
   
4. My spotlight searches (Image 2) use the comments. Spotlight (sigh) doesn't show comments in search results (argh), but the Inspector works on Spotlight results. I just open it and view details as I go from result to result.

It's a pretty darned simple workflow, and it's not dependent on any special software functionality. Just QuickTime, Perian, and OS X. I don't file the videos, I toss them into the folder and rely on search. If I end up storing them in some future OS X app the metadata will still be around, and potentially scriptable into some future repository.

Image 1: View of the video folder with inspector open. Files where comment = "d" will be deleted.


Image 2: Results of a Spotlight search, with Inspector open so I can click on a search result and see image and comments.



Update 6/6/08: When you drop an AVI file to iPhoto, iPhoto does not copy the extended attributes. In particular, it doesn't copy the Spotlight comments or the labels. I'm going to go and drink now.

Update 6/6/08b: The loss of extended attributes was a 10.4 defect. It's fixed in 10.5.3.

Nisus Writer Professional - the manual is a work of love

Nisus Writer Professional is not cheap for families. The license covers a single user at a time, so a family like ours with 5 users would need to spend $200 for use on a single machine. Other software has similar policies, but Nisus is very explicit about this during installation. Word and Pages leave such details to the imagination.

On the other hand, some testing shows NWP doesn't enforce the license (though for all I know NWP reports violations). They also have a generous upgrade policy for past licensees.

What else can I criticize about NWP/Nisus Writer Express?

• HTML export is pretty darned unimpressive
• I'm annoyed that NWE didn't survive the migration from 10.4 to 10.5.3. Many of my old apps run fine in 10.5.3, but NWE locked up when I tried to import a simple Word 97 document.
• NWP can't translate a Word Table of Contents into a NWP Table of Contents and vice-versa.
• NWP doesn't do image compression! Word has fabulous image compression, so a 2MB Word document can balloon into a 32MB NWP/NWE document.
  

I'll have more to say about the good things after I play with NWP a bit more. NWE has been my word processor of choice, and, even with the family licensing costs, we're staying with Nisus.

I'll point out one marvelous sign that most everyone else will miss. Nisus has a 400 page work-of-love PDF manual, which is sold as a bound book for $25.

These people love their product. That's a very encouraging sign.

There are some other things, from my time with Nisus Writer Express, that I expect

Corruption of Time Machine files can cause recurrent kernel panics

Modern computers have too many emergent properties.

Corruption of the the Time Capsule sparse image file Time Machine writes to can cause recurrent Kernel panics.

Gruber ran into the same problem.

The source of the bug has been located in Darwin code, the fix for now is to run Disk Utility to fix the Time Capsule store.

Of course not only should Time Machine not crash under these conditions, but it’s also really wrong for a TM crash to bring down the OS. Of course it may be that the flaw is in the OS and TM is only an innocent bystander.

Bad bug, no doubt.

The good news is that it’s well understood by Gruber-class geeks, so the fix might make it to Apple.

This is where the GBs go

The stinkin' printer driver is 1/4 GB?!

MacInTouch: timely news and tips about the Apple Macintosh:

... Apple's Canon Print Driver Update 1.1, a 267MB download for Mac OS X 10.5.3 or later, brings updated drivers for the Canon Pixma Pro9500 and Pro9000....

This is the digital equivalent of a single person commuting in a monster SUV. It's obscene.

How to use Google Calendar's Quick Add pseudo-natural language interface

Quick Add: the faster way to create events is a guide to entering strings so Google Calendar can parse them. It's a faster way to enter calendar items than using the GUI, and it can represent more recurrence patterns than the GUI (ex. Every 2nd and 4th Friday of each month, thanks DLW.)

Quick Add is a kind of mid-point between Natural Language Processing (NLP) and a programming language. Perhaps for performance reasons, the gCal text entry interface is not as flexible as a typical NLP interface. It's much less capable of parsing and "understanding" a phrase than a human reader.

On the other hand, it's more accepting than a compiler or interpreter. It will ignore things it doesn't "understand" and make some inferences. The inferencing is often incorrect, so I haven't used Quick Add very much.

If you know what Quick Add expects, however, you can dramatically improve its interpretations. The key pattern is (sorry, you have to memorize this pattern -- it's in english alpha sort order until "Where"):

What(title)-When-Who(invitee list)-Where

What and When are required ...

• What: This can be any text; the event title is created from this.
• When: This can be nearly any date and/or time expression. Using “at” and/or “on” can help the recognition.
• Who: This should begin with “with” followed by a list of email addresses; these are added to the guest list.
• Where: This can be any text following “at” or “in.”

In other words (the real strings don't have the [] brackets), something like (bold is required):

[Meet Emily] on [7/12/2008] with [emily@somewhere.111, fred@somewhere.111] at [Minneapolis metrodome].

The logic for recurring events extends the "when clause":

When creating recurring events, the time expression has three parts: the start, the repetition, and the end.

So:

[Meet Emily] on [7/12/2008 every month for five months] with [emily@somewhere.111, fred@somewhere.111] at [Minneapolis metrodome]

I think you can see why this interface is nowhere near "natural", but with practice it looks powerful and it's likely to improve.

Call it pseudo-natural.

QuickTime 7.4.5 for 10.4.11 disabled audio on FLIP Video (3ivx codec) - get Perian

It was broken in April 5th! How time flies. I only noticed today. It's a known problem with QT 7.4.5 and 10.4.11, but there hasn't been much noise. Neither Apple nor FLIP has been particularly helpful ...

Apple - Support - Discussions - When will Apple fix QT 7.4.5: no sound ...

...FLIP uses the 3ivx codec. I recall that the version they were distributing last I checked was described by 3ivx as having severe security issues. 3ivx distributed a newer version but FLIP doesn't distribute it.

It may be that Apple's 7.4.5 Quicktime update fix took care of the security problem -- by deliberately or unavoidably disabling the 3ivx codec. If so, that would be, shall we say, not very nice of them.

Under the circumstances it would have been nice of Apple to have also created a kb article about the problem.

Of course FLIP should also have created a kb article, and by now they should have had a fix.

It's noteworthy though, how few complaints there are in this forum. Evidently the group of people who are using FLIP video with 10.4.11 and applying security fixes and who know about discussion forums is really, really, tiny...

So if my FLIP camcorder stops working because the software is broken, can I make a warranty claim?

No, I didn't think so.

Update 6/5/08: If you try exporting from QT Pro, you'll see no options to export the audio stream. QT Pro is not "aware" of an audio component.

Update 6/5/08b: I installed the 3ivx.com 5.0.2 codec (trial version). It doesn't work either. Happily, we're seeing more complaints on the 3ivx forum. Frustrated users may wish to install the 3ivx s/w, then register for the forum and complain it doesn't work.

Update 6/5/08c: I uninstalled 3ivx.com 5.0.2 and installed the Perian preference pane codec collection. It works. I'm able to export to Apple Intermediate Codec. Uninstalling the 3ivx.com codec will disable the Flip import utility, you will need to drag files over by hand. The import utility reads Flip metadata and puts the image acquisition date in the file name, I'm sorry to lose that.

A pox on Apple, PureDigital and 3ivx.com alike. Blessings to the Perian LGPL development team. (Now if only they'd use something other than PayPal for donations ...)

Leopard (10.5) Sparse Bundle .IMG files are packages, not files

This is surprising. I will have to test with Retrospect and see how it treats these things. I assume Retrospect will see these IMG objects as Packages, and thus back up only the changed "band".

mac.column.ted: Leopard still holds some small surprises - MacFixIt

...So why was the bundle image format added in Leopard? Because there was a significant problem with plain sparse images. A sparse image is essentially a single file. When backing up your drive, a backup utility thus sees the image as a single file, regardless of how many files are stored within the image. Further, any addition or subtraction you make to the image (such as adding even a measly 5K text document) registers the image as a modified file. This means that, if an image file were 1GB in size, the entire 1GB would need to be recopied to a backup each time the image was modified, even if the only change to the image was a 5K file addition. Not very efficient. And unnecessarily time consuming.

The sparse bundle format avoids this dilemma. Essentially, the bundle format divides the content of the image file into smaller separable bands. The image still appears as a single file in the Finder. However, it is actually a package. If you select Show Package Contents from the image's contextual menu in the Finder, you will find a bands folder containing the individual band segments (as shown in the figure below). Each band, at least in my testing, was 8MB or less. Assuming your backup software recognizes and works correctly with the bundle format, only the modified bands are copied over when backing up the image. This means that backing up the aforementioned 1GB image, with a 5K file addition, would require copying only 8MB or less!...

...Apple, in Disk Utility's Help pages, recommends using the sparse bundle format whenever you want to create "a blank disk image for storage." Indeed, Apple takes its own advice and uses the new format for FileVault (rather than the sparse image format used by FileVault in Tiger)...

The division of a .IMG file into arbitrary packaged Bands is a clever mitigation of a problem that's had many variations over the years.

Update 3/9/09: I looked into these as a way to share an iPhoto Library between multiple users. It looks like Retrospect Pro does NOT backup .sparsebundle images correctly. Yech.

Update 5/6/09: Hoisted from comments (DocIceT):

Re your attempt to make sparsebundles work with Retrospect, I had some partial success.

Firstly the backup needs to include what Retrospect sees as top level directory of the bundle. Finder shows this as the name of the package file.

More interestingly, the restore works if it is done to the original drive. If the bundle gets restored to a different drive then the bundle is not seen as a mountable file system any more.

With that said, there is some kind of permissions change going on when restoring to a different drive and I had to tweak that manually. This could break some part of the OS X structure for making those bundles work.