How should I transcode my home analog video?

Twelve years ago I was keen to digitize my home video tapes - on a 400 MHz Celeron system.

Eight years ago I wrote "The thought of losing the kids' taped videos is not comforting. I need to do this sooner rather than later.". At that time I experiment with transcoding through a Canon digital camera using iMovie and a G3 iMac. I figured my G5 would do the job in 2005, maybe with its capacious 40GB drive and a stack of DVDs.

Now I have a slightly aged 2.7GHz i5, 8GB RAM, and several TB drives -- and my analog tapes are 8 years older. Maybe I'll do it this year, even though I hate digital video [1].

I'm trying to warm to the idea. The big difference over 8 years ago isn't processing power [2], it's cost/MB and the gradual emergence of H.264/MPEG-4 Part 10 (AVC) as a relatively standard format for video. The National Archives recognizes there's no real AV standard, but their list of formats they work with includes MPEG4.

If I do finally walk though this, I don't want to bother with DVDs. My current plan is:

• Purchase two 1TB drives. Digitize everything we have to one of the drives as MPEG4 video and MPEG4 audio.
• Clone that drive and keep one drive on site and one drive offsite, repeat every few years.[3]
• Use my old Canon digital camera with passthrough to digitize my analog tapes, then compress on my iMac and store with a file naming convention to reflect date and time information. (I'll consider the Canopus though.)
• Convert my collection of digital tapes (DV) to H.264 as well

I don't know what software I'll need. Can I do what I need to do with iMovie/QuickTime Pro, or should I buy Final Cut Pro X? I'm not planning to edit, but if I were I'm pretty sure the current fork of iMovie won't do the job.

Update: Apple offers a free download for FCP/X. So I will probably try it out.

See also:

• Good advice on digitizing video (Tidbits) 12/2004
• Home Video Editing started 6/2000! (Pass through with a Canon ZR 60 in 2003)
• Digitizing Video: Tapes - extensive Macintouch archives
• BYTE Media Lab 2004 Imaging Awards: Analog to Digital Converters 4/2004
• Why I hate video: Format, codecs, DRM and m4v vs mp4 7/2011
• Amazon.com: Canopus 77010150100 ADVC110 Converter: Electronics (old tech, good reviews, $218) 2/2012
• REC.VIDEO Google groups/usenet posting: Video degradation with S-Video analog transfer vs. A/V mini-jack connection 3/2005

[1] In Jan of 2010 I discovered a 12 yo WMV formatted video was unplayable; I was able to use Windows Movie Maker to convert the 23MB WMV file to a 311MB DV AVI file. Digital image formats are a friggin' mess, but video is at least ten times worse. No metadata standards, crazy patents, non-standard containers, audio codecs, video codecs - yech.

[2] Yeah, the machines are a hundred times faster. But compression can run overnight.

[3] By 2020 I assume we'll have 100TB main system and backup drives, so storing my old video with my routine data will be trivial. By then a single snapshot will be a GB.

Update 12/14/2012: Emily picked this project for her Valentine gift, so we're off. I found a 300GB drive and LaCie firewire enclosure I'd forgotten about. So far video capture seems to be proceeding well from a 12 yo Hi8 analog tape; there's a thin sliver of artifact at base; I think capture frame is bit taller than the image frame. Setup at the moment:

• SONY Hi-8 TRV65 with S-video and RCA audio out to ...
• Canon Optura 50 with firewire out via secondary port on LaCie to ...
• iMac i5 2.7GHz, 8GB RAM
• Final Cut Pro X (trial version. There's no more academic pricing for Apple products distributed through app store. I expect I'll buy full version. So far this is the big expense) saving project to ...
• External firewire LaCie drive (300GB)

Both cameras are on external power. I needed my old manuals, had the SONY and found the Optura online. Getting passthrough working was balky, FCPX only gave me about 10 seconds to start then it turned off import due to lack of a signal. It can't operate the Optura 50 in pass-through mode, I haven't tried yet with a tape installed. Once I hit the play button fast enough I was ok.

I'll have data on file size tomorrow. FCPX imports using its default Apple ProRes format.

Operating the old SONY feels primeval. It's 15 years old, feels 50. It is weird to see the kid video. Plan is to put a library of MP4 versions (no editing really) on iTunes on a an old G5 with a largely unused drive. We can browse, play from iTunes.

Update 2/15/2012: Not so good. Nothing captured! Looks like I need to read the manual, or perhaps FCPX won't work at all. It certainly showed the video, but it didn't save the event. It also didn't stop recording when the input ended.

Update 2/15/2012b: QuickTime Pro is not an option. I was able to create a @2 hour 25GB .dv file using iMovie 11, but then iMovie 11 got stuck 'generating thumbnails'. I think that bug, however, was triggered by connecting an iPhone to my iMovie machine. Disconnecting the iPhone did not clear iMovie, I had to kill it. On restart my video appears to have been captured and iMovie seems to have stopped input about 2 minutes after the end of tape. Thumbnails are fine - all 1.5GB of them. More on the results and process in a follow-up post. I'd like to take a look at Adobe Premiere Elements.

Update 2/16/2012: Gordon's Tech: iMovie 11 and analog to digital videotape conversion by passthrough.

Update 2/17/2012: I experimented with QT 10 movie capture. The trick with passthrough conversion is to first start the video stream, the click record. Otherwise QT exits. I captured 30 min of video at 9 GB; QT Inspector says the codec inside the saved .mov file is dv.

I've read that this will import into FCPX, so I may try that again. My next experiment will be to see if I can record over 10 GB (1 hour of video). There are many reports of QT recording stopping with a 'size limit' message at varying file sizes, in my case that's what displayed after I stopped the video stream.

Warning - if you delete your G+ Profile you may turn off sharing from your Picasa albums

A friend of mine complained he was unable to download some wedding pictures I'd taken. Last summer downloads worked, but as of a few months ago they stopped working.

Just around the time I deleted my Google G+ Profile.

My friend told me the original sharing link worked, but the download button was grayed out. Sure enough, when logged out of Google I couldn't download images. It was easy to see why, image downloading was turned off in my Picasa preferences.

I don't remember turning that off. Given the way I share images, that wouldn't make sense. My best guess is that when I deleted my G+ Profile Google reset my sharing preferences to the default settings. if I'd known I could have fixed it immediately. Instead I've been sending share links out for months that weren't working.

I assume most of the people I sent shares too simply gave up in frustration, or ended up downloading a low res thumbnail.

Google Fail.

Time Machine for Lion supports encrypted backups

In the midst of tweaking a Time Capsule backup, I noticed a grayed out option to encrypt my backup.

Turns out 10.7 enables Time Machine encryption on Lion - but only for directly connected drives.

Reading the MacOS X hints commentary I think they're missing the point. A good backup strategy requires offsite backup, and taking a non-encrypted backup offsite is somewhat risky. I don't take my Time Capsule backups offsite, but I do rotate my Carbon Copy Cloner images/backups -- and they are encrypted.

This is a good feature if used wisely.

Obscure bug: Reeder share to Facebook will post to last used Page Identity

This is an oddball bug. I suspect it's not fixable.

I used iOS Reeder to share a Facebook post.

Problem was, when I last used Facebook I used it as one of the Pages I maintain. That's how Facebook Pages work; you assume the Page identity while you post to it. Then you flip back to your own identity.

So Reeder shared my personal post on my public Page. It wasn't too embarrassing, but it was clearly wrong for the Page (org).

It's a weird bug; I suspect the flaw is in Facebook's API. It probably isn't designed for this use case. I also wonder if this is a side-effect of Facebook introducing the Timeline to my personal account.

Wiki impressions: XWiki

After a disastrous SharePoint 2007 to 2010 migration [1] it was clearly time to replace my team's use of SharePoint wiki. The mangled conversion was a red-flag-and-air-raid-siren declaration that SP was going to be a longtime world of hurt.

We looked around at the wiki options, and considered TWiki, XWiki and Confluence (WikiMatrix). Atlassican Confluence is the best of breed, but for corporate users like us it's expensive. Hardware we have, platform licenses we have, networks and backups we have, and expertise we have. What we don't have is a budget.

TWiki is the easiest of the three to configure and it has the fewest technical requirements. XWiki looked like more work, but we liked the rich text editor and (limited) table support. We're now starting up on XWiki enterprise. Here are a few impressions from the process that may be of interest to other wiki searchers ...

• We started out on a Linux server but couldn't get XWiki working. We're not Linux experts. We switched to Windows and our engineer resource had XWiki setup within a day. I wasn't impressed with the ease of installation.
• XWiki requires a Java Servlet Container. That rules out Dreamhost and many other hosting options. (TWiki can install with Dreamhost.)
• XWiki includes a blog service but there's no support for Blogging APIs. So you can't use MarsEdit or Windows LiveWriter to write.
• The documentation is weak. For example, how do you delete a Space? Turns out it's easy if you have privileges, but the documentation claimed we needed an extension.
• XWiki Enterprise is a geek platform, though much of the complexity can be hidden.
• I like the approach to URLs and page titles. You can change a page title, but the page URL is fixed. XWiki provides an effective UI for looking up local pages during link creation. (SocialText retitling creates a redirect page with the old title/url and a link to the new title/url. Sharepoint changes the URL and title, but updates intra-liki links. Foreign links break.)
• The rich text editor is simple but works fine for my purposes.
• IE 9 doesn't work with the controls on gadgets, but seems ok for non-admin users.
• XWiki has the universal wiki curse -- no ability to migrate posts between systems. I wonder if people who complain about being unable to move their medical records between providers see the similarity. OTOH, when our Linux install failed it was easy to move to Windows.

I'm reasonably optimistic that XWiki will work for us. I'm glad to be free of Sharepoint 2010.

See also:

• XWiki - Open source, LGPL, WYSIWYG
• Offline Browsers (web robot) – options to review
• How can I create a local copy of a Sharepoint wiki?
• A workaround for image uploading to Microsoft’s Sharepoint Wiki.

[1] With SharePoint it's not possible to separate software issues from implementation. Maybe Microsoft provides wiki conversion tools that our IT department didn't use. I have discovered that if I copy/paste of our 500+ SP 2007 wiki created pages into Word, then remove all styles, then past back into SP 2010, I can edit them again.

Time Machine backups of Aperture are not reliable?!

I've restored my Aperture Library from backup twice in the past few months. Two months ago Aperture crashed when a bad block corrupted a file. I restored from Carbon Copy Cloner. A month later I figured a dying iMac drive was the root cause, and my backups weren't as robust as I expected.

After a mildly painful Apple service call i had to restore all my data from backup. I backup the server nightly to Carbon Copy Cloner and hourly to Time Capsule, so I had to pick a backup source. I decided to restore from Carbon Copy Cloner, though I did consider Time Machine. It seems to have worked.

Today I learned that I might have dodged a bullet. I was wise to choose CCC over Time Machine ...

Macintouch Backup

Derek L

...Thanks to Antonio Tejada for his note about Time Machine and Aperture. The combination remains unreliable for me, as I'd described in my comment on this topic last June: not all items in my Aperture library get included in my Time Machine backup. Although I've been able to temporarily repair it via forcing a "deep traversal", the problem recurred in fresh TM backup sets, on multiple disks, and through several point releases of Snow Leopard and Aperture 3 (I've never used any other version, and I also haven't migrated to Lion). I gave up on it and instead depend on Aperture Vaults.

Skot Nelson

... However, after having Aperture randomly lose some of my masters -- old photos from a concert that I hadn't touched in month and just happened to click on as part -- I no longer trust my Aperture library's integrity...

Richard Tench

... Seeing the warnings here about Aperture and Time Machine, I decided to do a test recovery (to my desktop). It failed due to permissions on the Time Capsule. Though one would think that's an easy problem to fix, it isn't. At the end of 2.5 hours on an AppleCare call, they were unable to help me. I was told that Aperture doesn't work well with Time Machine...

Sigh. I miss Dantz Retrospect. I'm grateful for CCC, but old Retrospect combined the reliability of CCC with the features of a true backup.

It's been noted that in 2012 Apple OS X development is the equivalent of Siberian exile. Time Machine work must be reserved for unwanted interns.

See also:

• Backups - why you need two methods and abundant paranoia 12/2011
• Freeing up Time Capsule space – and documentation for Time Machine and Time Capsule 12/2009 (Time Machine is flaky)
• Lessons from Apple Store out-of-warranty repair of a Seagate drive 1/2012
• Lion continues to disappoint: the Duplicate/Save problem 1/2012
• 7 facets of a good Mac backup strategy « Haystack Software Blog 1/2012 - advice from the developer of Arq (backup software for S2) -- but absolutely correct. I do this and I'll evaluate Arq.

SharePoint 2010: migration of a SP 2007 wiki

My business group has maintained a large SharePoint 2007 wiki.

Recently we had to migrate to SP 2010. The wiki went along for the ride.

I've seen some bad outcomes in software over the past 20 years. I mean -- really bad products.

I've never seen anything like this.

Wiki pages that look like they were edited by intoxicated baboons. Pages that can't be edited in IE 9, but can be edited in Chrome. Pages that can't be edited in Chrome but can be edited in IE 9. Pages that can't be edited anywhere, save as HTML.

SharePoint Designer is an option. I tried it. I shed tears for the lost spirit of FrontPage and Vermeer.

I despair, sometimes, when I see what Apple's interns are doing to iPhoto or OS X. Even they, even they, have never done anything like this.

This is the worst.

Microsoft, astonishingly, is truly finished. We really are in the twilight of the personal computer.