AVI video files in 2014 on the Mac -- you should probably convert them while you can.

In the process of slogging through an unexpected iMovie migration, I came across some old AVI files that iMovie 2013 basically swallows and hides. 

I dug them out of the Package and started searching for AVI conversion tools (bear with me, I’m out of video practice). Google returned pages of spammy looking hits marketing suspiciously “free” third party tools. 

That’s a clue folks. It typically means a market segment has died because Apple has made it part of OS X. Sure enough, FaceTime in Mavericks and later now converts most AVI files (AVI is a container, the real problem is the funky video compression standards inside the container) to .mov.

I experimented with one old AVI file — the original (low res) 40MB camera clip becomes a 64MB quicktime .mov file with H.264 compression and Microsoft ADPCM audio (I wonder if that was unchanged from the AVI file). The 50% growth is typical of migrating from one lossy format to another lossy format — it would be quite bad news if .mov file were the same size as the AVI file. That would indicate too aggressive compression.

I played the converted file back in QuickTime (Mavericks) and the original in QuickTime 7 and they didn’t look too different. Not bad. (As noted below I actually have Quicktime Pro 7. I’m not sure AVI files play in QT7 without the now defunct Perian plugin).

You can’t control the codec or parameters QuickTime Mavericks/Yosemite uses for conversion, for that you can try QuickTime 7 Pro ($30, I suspect part of that is for licensing codecs). Yes, it’s on the App store! First you download the Snow Leopard installer (works on Mavericks/Yosemite) then you pay. I was about to buy it, but then I thought I should check I didn’t already own it. I bought it in 2008 (!). I really should use it a bit more often.

You can also use Handbrake to convert AVI files, or VLC to view them.

I really need to convert those old files; conversion is only going to get harder. However, it’s weirdly good that QT Pro from Snow Leopard still works and is still sold on the Apple store. So we have a bit of time.

See also

General refs on AVI  

• OS X Mavericks: Some files may be converted when opened in QuickTime Player - Apple Support
• QuickTime will not play AVI files after updatin... | Apple Support Communities
• How to play .avi movies in Mavericks | Macworld - Play in VLC, QuickTime 7/Perian, or convert with Handbrake.
• Run QuickTime Player 7 in OS X Yosemite & Mavericks

Older posts of mine on the horrors of video codecs and compression — there’s been limited progress since 2008

• Why I hate video: Format, codecs, DRM and m4v vs mp4 7/2011
• How should I transcode my home analog video? 2/2012. H.264/MPEG-4 Part 10 (AVC) is as close as we have to a standard
• Video: The state of compression is not good. Blame it on Congress 2/2012
• FLIP Video Ultra camcorder: iMovie HD works, iMovie ’08 might 3/2008 (The short-lived pre-smartphone Flip Video camera taught me a lot about codecs. Specifically, a lot of bad things.

I migrate from iMovie whatever to iMovie 10/2013 whatever and of course it hurts

Dilbert, 4/14/94

In retrospect, things started to go wrong for Apple 9 years ago. Back then Apple’s iPhoto, iMovie and iTunes were all running pretty well on Leopard. Then Apple jumped from iMovie HD 6 to “iMovie ’08”, a complete reboot and major regression. Since then we’ve lived through a series of half-baked regression-heavy reboots all too reminiscent of SONY’s Spiderman. There’ve been reboots of FinalCut, iWork/Pages, iPhoto and now Aperture/Photo.app…

Oops. I forgot iTunes. It gets randomly rebooted yearly.

Three years after iMovie ’08 Apple recovered most of the lost ground with iMovie ’11. Which was then followed by iMovie 10, another reboot with feature regressions.

Yeah, Apple went from “’08” to “’11” to “10”, sometimes called “2013”.

I picked up iMovie 10 as an automatic update — I don’t use iMovie all that much (it’s been a discouraging ride) and I didn’t realize the product had had yet another reboot. So when I launched it yesterday to work on my daughters dance performance it took me a while to realize I was in a world of hurt. iMovie 10 has a new file structure and Apple cut a few corners on the migration tools - particularly for Libraries located on external drives.

The conversion and recovery process was too painful to recount here, but here are a few of things that might help if you’re trying to figure this out:

• Read these 3 Apple tech notes: 
• iMovie (2013): Updating projects and events from previous versions of iMovie. Note the classy way iMovie 2013 prevents old data being imported more than once — it writes a zero byte file “UpdatedtoiMovie10”. Big catch — the iMovie 2013 update process works only on iMovie ’11 files stored in the Movies folder or at the top-level of an external drive.
• iMovie (2013): Removing old libraries after updating. See my note below before doing this. Note there’s nothing in the update process to tell you that you have now doubled your movie storage budget.
• iMovie (2013): Work with multiple libraries
• The new file structure resembles Aperture and iPhoto; iMovie now uses Packages. You can right click on your Movie Library to see the internal folders; they resemble the old file structure.

For my conversion I did the “Update Projects and Events” twice. I first did it for the Projects and Events in my iMovie folder. Then I moved my external (large capacity) files to the root of an external drive and repeated it. Once the new Library Package is created you can move it wherever you want.

I then compared what I had in the old Projects and Events folders to what I saw inside the new Packages. When I was convinced those lined up I tested viewing each movie/video. I found a number of problems that I was able to fix with iMovie by changing metadata and creating events.

I also ran into a curious problem where one Project seemed to have pieces in two Libraries. The fix for that included creating a new empty Library and moving events one at a time into it from a damaged Library. There doesn’t seem to be any Library rebuild/repair function.

There’s a native menu option for moving Events between Libraries, so one can theoretically move them from an SSD to an external drive and vice-versa.

Good luck!

Wanted - A tool to chop Pinboard HTML export into Spotlight searchable HTML files

Pinboard provides a nice range of export options including XML, JSON and HTML.

The HTML export uses the ancient Netscape bookmark file standard (I love simple, ancient standards). My 10MB export file starts like this:

There are also XML and JSON versions:

I’d like to follow the example of my nvAlt/Simplenote archive [1] and have every bookmark as a Spotlight searchable file. The HTML file seems the easiest to process — wrap each DT/DD pair with HTML tags and save to file system with the file name taken from the Initial “Description” text field (truncate at nn characters, append ADD_DATE to make unique).

Seems like something somebody could whip out in AppleScript or Python and then sell on the Mac App store for a few dollars. Wouldn’t make a fortune, but I think Maciej Ceglowski would add this to the Pinboard Resources page. So maybe good for a few hundred bucks?

- fn -

[1] Simplenote is all-but-dead today. The beauty of the nvAlt (or Notational Velocity) and Simplenote system is that one can easily transition to Dropbox and something like Notesy, and all notes are available on my Mac and Spotlight indexed with all metadata all the time. That means I can hang with dead Simplenote, just like I hang with dead Aperture. It helps that Brett Terpstra’s (not free) nvAlt replacement is almost here…

 

dd, the Mac OS X (Unix) native solution to hard drive testing, cloning, drive recovery ...

App.net’s gaelicwizard posted this approach to hard drive testing back in Oct 2013. I’d lost it for a while but just found it again today:

$ dd if=/dev/disk1s2 of=/dev/null If it hangs indefinitely, replace the hard drive. If it I/O Error’s, replace the the hard drive. If it completes without problem, the hard drive has no bad blocks.

I’ve not seen this mentioned anywhere else recently, but I found a related MacOS X Hint from 2005 that explains what dd is, and how it can be used for drive recovery and as an alternative to “Norton Ghost”:

The Unix program dd is a disk copying util that you can use at the command line in order to make a disk image. It makes a bit-by-bit copy of the drive it's copying, caring nothing about filesystem type, files, or anything else. It's a great way to workaround the need for Norton Ghost.

Normally, in order to make a disk image, the disk you're copying from has to be able to spin up and talk -- in other words, it's OK to make a copy if the disk is healthy. But what happens when your disk is becoming a doorstop? As long as it continues to spin, even with physical damage on the drive, dd and Mac OS X will get you out of the fire.

We had a situation recently where a friend sent a disk to us that had hard physical errors on it. It would boot in Windows, but then it would hit one of these scratch marks and just die. We fired up dd, and it started OK, but stopped at the same physical error location -- complaining about a Hard Error.

So the workaround was to designate the dd mode as noerror -- which just slides over the hard stops, and to add the mode sync, which fills the image with nulls at that point. We did it on BSD Unix, but as long as you can get the hard drive attached to your Mac, the command is the same:

dd bs=512 if=/dev/rXX# of=/some_dir/foo.dmg conv=noerror,sync

… Once you’ve established the disk image (in this example, foo.dmg), you're almost home. Here's where your Mac OS X box is far and away the best thing to have. In this example, the dd output file is foo.dmg. You have to realize that this is an exact copy of a busted drive, but the "holes" are filled with nulls. As long as the damage isn’t to the boot sector, though, when you double-click on it, Mac OS X mounts it without breathing hard … who cares if it's FAT32, NTFS, whatever...

On a Mac or Unix machine “man dd” will provide the explanations of the parameters in @gaelicwizard’s example:

• if=file Read input from file instead of the standard input.
• of=file  Write output to file instead of the standard output. 

So the command reads every byte of disk1s2 and writes it to “null” (Nowhere). Why disk1s2? It’s a common boot drive. You can use “diskutil list” to find out the name of attached drives. On my machine the result looks like:

/dev/disk0
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *1.0 TB disk0
 1: EFI EFI 209.7 MB disk0s1
 2: Apple_HFS Stanford 999.3 GB disk0s2
 3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *2.0 TB disk1
 1: EFI EFI 209.7 MB disk1s1
 2: Apple_HFS Media 2.0 TB disk1s2
/dev/disk2
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *3.0 TB disk2
 1: EFI EFI 209.7 MB disk2s1
 2: Apple_CoreStorage 3.0 TB disk2s2
 3: Apple_Boot Boot OS X 134.2 MB disk2s3
/dev/disk3
 #: TYPE NAME SIZE IDENTIFIER
 0: confidential *102.4 MB disk3
/dev/disk4
 #: TYPE NAME SIZE IDENTIFIER
 0: Apple_HFS BACKUP_B *3.0 TB disk4

I tested dd on disk4 (my backup disk). I found I had to run from my admin account, I needed to use sudo, and I had to unmount the drive first (or I got “resource busy” error message):

Admin$ sudo dd if=/dev/disk4 of=/dev/null

Please be careful what you type — especially for the of parameter. One little typo and you could be writing data anywhere.

It runs silently, you can stop with a Ctrl-C. It then reported:

1451352+0 records in

1451352+0 records out

743092224 bytes transferred in 48.218709 secs (15410869 bytes/sec)

I’ll let this run overnight on one of my backup drives some time, but it looks like an inexpensive way to test a hard drive. I’m surprised it’s not described more frequently. gaelicwizard knows the dark arts.

Aperture Tip: ⌘↑ or ⌘↓ will change focus to selected image

You select an image in Aperture, one among thousands. Now scroll up. How do you get back to your selected image? ⌘↑ or ⌘↓ will change focus to selected image.

Google Voice for the home - Obihai SIP/VOIP devices and porting landline number to Obihai/Google Voice.

Obihai Technology is marketing Google Voice use with their Obi VOIP-landline bridge products (SIP & OBiTALK VoIP services, $40 on Amazon).

Unfortunately there’s no way to port a landline number directly to an Obi device. Instead you have to use the T-mobile hack - port to T-Mobile then port to Google Voice (best documentation of this hack I’ve seen by the way). Once you’ve made this irksome port however, you do get to use GV’s great features from a home landline and VOIP. (Compare, however, to mix of GV, Bluetooth, Mobile, landline.)

Alas, the future of Google Voice is problematic — we know it’s going to be replaced by Hangout, but we don’t know what Google Voice features will survive. 

Still, it’s amazing to watch the twisty-turny evolution of voice communication. This is NOT what we expected back in 1994; then we expected voice communication to become essentially “free” by 1998. Remember that the next time someone predicts the evolution of the marketplace based on technological innovation. The market is very good at fighting back - for example. (Use a fax machine lately?)

See also

• Making better predictions 3/2011
• Canopy Economics 5/2004

Wanted - a way to make an old style landline work over a cellular connection.

My 93 yo father is a resident of a Canadian Veterans long term care facility. He’s doing pretty well there, but it’s a bugger to reach him. Their landline costs are very high, and installation seems to take eons. Vets who can use a cell phone are fine, but that’s hard for him.

What I need is a cell phone that looks and acts like this phone:

I’m sure they make these for the China market, they’ve turned everything into a cell phone.

The closest I can find in the US market is the Panasonic Link2Cell Phone (note, however, complex compatibility grid.pdf - 3GS is borderline).

I could put an old iPhone in his room, leave it permanently plugged in and maybe this would work. It’s pretty complicated though he has used similar devices. (I think these are sold to people who  get rid of their landline and transfer the landline number to a cheap cell, but want to share a home phone.)

I fear what I really want is only sold in China  It’s either mobile that looks just like old red except that it has a plug rather than a phone cable, or it’s a bluetooth device that fools an old style phone into thinking it’s on a phone line.

Update: I found a few other options…

• Amazon.com: Xtreme Technologies Xlink BT Bluetooth Gateway: This should work with the red touchtone phone! So it seems ideal. More here
• eBay has some that look like they might be out of China
• The Clarity-Sempre is a very interesting device for extending a cell phone for an elderly person, but it’s $290 here — cheaper on Amazon. More discussion of Sempre’s alternatives here, the BT914 is much more senior friendly that the Panasonic.
• Cobra PhoneLynx is $20 - on Amazon.
• Obihai sells a solution made up of their Obi202 and a simple bluetooth adapter.
• In Canada, Fido, an MVNO sells this service for $30/month