Digitizing home video (once again)

I have made (2000) several (2004) tries (2012) at digitizing our family video but my lost best effort only got about 10% done. I figured I’d just hire a local service and then I let it slide.

Apple’s recent codec retirement announcements prompted me to check what was available locally. I found ancient web sites that were internally inconsistent, no noise reduction prior to compression, unclear codec choices … none of it gave me much confidence. (But see [1,2])

So I’m back at it again. This time I might have an accomplice — someone who needs money and would benefit from learning the tech. So maybe we’ll make a better go of it.

The delays may have let to some data loss, but on the other hand the tech is a bit better. My first attempt would have been with a 400MHz Celeron. Yes, that’s an ‘M’. In those days hard drives were measured in tens of GBs. Now the cheapest hard drive I can buy would hold all of our video.

The tech is a bit better, but choice of codec is still an issue. In 2004 I favored H.264/AAC. I ran into an amazing number of headaches with the Apple software I was using.

For the modern era I found three good references:

• DIY VHS Preservation: Planning for Video Digitization at the American University Library
• Audio Visual Digitization Hardware and Software - Marriott Library - The University of Utah
• Building a Video Preservation Rack for In-House Digitization AV CLUB | Issue 1 | Cardinal Tales

They give me a feeling of how tricky it is to do analog video capture well. Time Base Corrector?  BNC terminations? Waveform monitors? CRT monitors?! Yikes.

I did like the sounds of the BlackMagic Intensity digitizer ($240 for T3, $200 for USB 3) used at AUL (Amazon reviews are not great however). It can save output as a lossless file. I want to capture the video as “uncompressed 10-bit 4:2:2” then denoise it, then export as ProRes. Since my accomplice is a student I’ll probably buy the Pro Apps Bundle.

Ideally the process would be automated - capture uncompressed, denoise and incorporate metadata, save as ProRes.

What would I do with this material once it’s digitized? The tech isn’t here yet, but eventually I’d like to incorporate brief (silent) video fragments into my screensaver library. So between showing 10,000 images, show a 30sec of video from our family @ 1995. One day?

- fn- 

[1] From my 2000 page I see Walmart and Target were do video to DVD-R conversions for $35 a tape with YesVideo. They are still around! The price is now $26 for one 2hr tape.  

[2] Pogue years ago recommended Southtree (his screenshot of a VHS tape on a modern laptop screen is remarkable — 333x480 pixels). Their site is impressive; at the moment they’re advertising $57 for up to 3 tapes on 1 thumb drive. I contacted Southtree to ask about denoise/ProRes/etc but they kindly responded that they are consumer-only, so just mp4 compressed.

Converting my old video formats to something that might persist

Apple is killing QuickTime 7 (download 22MB Snow Leopard version here)[1]. The announcement has some useful references.

So I’m back to thinking about one of the worst topics in the geek world — video file formats and codecs. Almost all of which are encumbered by walls of vicious patents (AV1 is the great hope).

It’s been about 4 years since I last made stab at this topic. I reinstalled QT Pro then too!

Conversion from old formats is a PITA. In 2015 I converted an old WMV file (I think I used Flip4Mac [2]) to (I think) lossless uncompressed AVI. The original was 23MB, the AVI output was 311MB. Today I used QuickTime Player 10.5 to open the AVI and it created a 52MB (lossy) .MOV file (Info says it’s H.264 for video, AAC for audio). There’s no control on the conversion compression.

To get this done I’m going to have locate my video files (some in iMovie projects, some in the file system, some in Aperture) and apply some kind of batch conversion where needed. Spotlight’s ability to search for codecs might help, I suspect some useful utilities will show up now that this is getting some attention.

What should I convert to? Ideally I’d choose something designed for video editing. The Smithsonian recommends "Motion JPEG 2000, MOV, AVI”; I don’t understand how MOV gets in there, I thought it was just a container. Also MOTION JPEG 2000 is on Apple’s kill list.

Ok, so things are bad. But we knew that.

Apple’s ProRes is one example of what’s known as an “intermediate codec”. Apple seems to be dedicated to it and I gather it’s widely used in the video industry. So let’s see what other choices there are ...

Fifty. That link lists 50 intermediate codecs. Ugh. Maybe ProRes 442 HQ isn’t the worst option.

I’ll probably have to play around and study some more. Some things I’ll look at ...

1. Wondershare is $40 for 1 year. There’s a free demo that converts the first 1/3 of the video. It converted an old .wmv file to an HEVC (H.265, AAC) without blinking an eye. Judging by the 1/3 converted the 23MB WMV file would convert to an HEVC file of similar size.
2. Apple Compressor: $50, or have my college son buy the whole ProApps bundle (Compressor, FinalCut, etc) for eduction for $200.
3. QuickTime Player’s built-in conversion.

See also:

• “Swiss Army knife for QuickTime” Perian turning off the lights 2012
• Why I hate video: Format, codecs, DRM and m4v vs mp4 2011. Still hate ‘em.
• MPEG Streamclip video converter as Compressor alternative 2012. This was last update for Mountain Lion!
• AVI video files in 2014 on the Mac -- you should probably convert them while you can.
• FLIP Video Ultra camcorder: iMovie HD works, iMovie '08 might 2008. FLIP used 3ivx MPEG-4, iMovie ’08 couldn’t use it, but iMovie HD could if the 3ivx coded was installed.
• Video codecs: iPhoto, iMovie HD, iMovie '08 and FLIP Video (2008 again!)
•  . Fun stuff.

- fn -

[1] Yes, that’s MB. Wow. It does indeed run on Mojave! Not only that, but I dimly remembered that I had a registration code for QT 7 Pro. Being a total nerd I still have it on hand. I entered the code and clicked “register”, but the registration server is long gone. It still registered though, I quit and restarted and I have the Pro features back.

[2] Replaced by Switch. For $10 you can convert wmv to mp4, for $200 you can to more output formats. If you search wmv to mov conversion you get a lot of junk, so this is worth paying attention to. I might buy it.

Samsung sells a security cam DVR, but not a TV DVR.

I’m old enough to remember when it was inexpensive and convenient to time shift football games. In the US and Canada that died with the analog to digital conversion. In some other countries there’s push-button record to USB from every TV, but not in the US. (I blame a VW-diesel class conspiracy, we now know those can happen.)

Periodically I look to see what’s sold without a monthly fee. Today Amazon pointed me to a $140 Samsung security cam DVR. There’s no tuner of course, so it can’t be used to record OTA TV.

The time-shifting story seems a small thing, but it convinced me markets don’t work the way people imagine they work. And the world doesn’t work the way I once thought it worked.

Nursing home personal videoconferencing and iPad photo slideshow - a successful project

My 94yo father lives in a veterans long term care facility located in Ste Anne de Bellevue, a small community at the west end of the island of Montreal. It has been a good home for him despite some difficult organizational transitions. Canada’s last major war ended over 70 years ago, their veterans system is fading away.

It’s a costly 6 hour flight (fly+security+etc) to see him, so I only get out there every 3-4 months. I send a weekly email that staff print for him, and every other week I send him a PhotoCard featuring one of the kids or a family thing. Phone calls really don’t work though — he didn’t do well with them even when his cognition and hearing were better.

The facility was keen for me to try videoconferencing with him using a Skype workstation. I was a bit skeptical, but they were right. He does much better with videoconferencing than with a phone call. The audio quality is much better than a modern phone call, and it’s a lot easier for me to see how he’s doing with the conversation. I can tell, for example, that he’s enjoying just hearing me talk. He really doesn’t need, or want, to say much himself.

The Skype workstation had problems though. Most of the time scheduled calls failed. Technical and organizational issues made it too unreliable. 

I didn’t want to give up on the videoconferencing, so I researched LTE videoconferencing for a longterm care facility resident. I considered WiFi but the costs at his facility are higher than LTE and in my experience institutional WiFi is often unreliable. He was already using an LTE Rogers Wireless device to connect an old school landline phone to a cellular network [1] so I was reasonably sure the LTE solution would work from his room.

At the end of the day we deployed a new LTE iPad Air 2 in a minimally modified “CTA digital” anti-theft stand. Here are some images of the stand the Vets built for him; during this first conference he spoke with a younger brother he’d not seen in over 10 years:

 

The wall stand was build by “Jean-Paul”, a staff and facilities person at the Vets. It’s a work of art and an unexpected key to this successful deployment. He built it around the iPad locking stand and incorporated a simple turntable. My father can do the videoconferences from his wheelchair or he can view the 3,000 image family photo slideshow from his lift chair. 

Dad hasn’t tried to operate the iPad. I think he could learn some things if I were there to work with him, but he’s a passive user at this time. A private aide visits him weekly and I schedule the videoconferences with her. I initiate the call, she taps the green button to answer. We use FaceTime because it’s very reliable, has great sound and video quality, and very efficient compression. A typical 15 minute call uses about 25-40MB of data, he has no trouble staying within his monthly Rogers data cap.

I often do the calls from my iPhone and I usually incorporate some kind of walking video tour. The last tour was of a CrossFit gym I’d just finished working out at. The walking tours are very popular, he reports on them to friends and family.

When the iPad is not being used for videoconferencing it’s displaying images using Picmatic.app. I was irritated when Apple dropped its original iPad slideshow functionality, but I figured there would be many fine replacements. I was wrong. There is exactly one - Picmatic. Miraculously it’s well done. It’s also ridiculously cheap at $2. It’s configured to randomly display full screen images from an iCloud photostream; I put images on there from Aperture and my iPhone. Images display full screen with an integrated clock and cycle every 30 seconds or so. He, or an aide, taps the bottom right icon to start the show. It automatically turns off at night. I wish it were more automatic but Apple is not terribly helpful in this regard. There is only so much developers can do when using iOS.

 

I’d put some other apps up there I thought might be useful: Notes for memory aide, Mail to show old emails I’d sent him, Podcasts for entertainment, Great Courses.app to play his history audiobooks, Contacts as an address book, Facebook to see our family activity, Weather, FaceTime.app (of course), Messenger.app (for non-Apple videoconferencing) and Calendar.app. Only Picmatic.app and FaceTime are being used. As Dad’s moderate dementia progresses he is less able to follow things like an audiobook history talk. He might do better with a brief Ted Talk video.

I’ll conclude (out of time :-) with some quick notes for anyone considering a similar project:

• Theft is a problem in longterm care facilities. Lots of visitors and impossible to screen them all, not to mention residents with impaired judgment. When the staff heard he was getting an iPad they expected it to disappear. This device would not be terribly hard to steal — the cable is only attached to a wall screw and the stand could be unscrewed from the turntable. It’s been enough so far though — just awkward enough to take that it hasn’t been stolen so far. There are two keys for the cradle lock; one in a lockbox in his room, the other in the nurse manager’s desk. The iPad stays in the cradle.
• I like the cradle but with the cable lock installed it’s hard (almost impossible) to rotate orientation. It stays landscape and that works well.
• For security I set a passcode and assigned Dad and his aide’s prints to the device. If stolen it’s iPad locked so wouldn’t be useable anyway.
• My brother has power of attorney. He had to send a copy of that to Rogers so he could get added to the Rogers account. Then he could add me to the account. This was the hardest part of the project. When I arrived in Montreal I took the documents to the Rogers wireless office. They had a hard time setting things up because Roger’s standard software couldn’t handle my US address, they did it using old paper forms. Once that was done the SIM worked fine.
• I bought the device and did all the setup in the US, wasn’t time to do something like that in Canada. I tweaked setup for weeks. I should have put more things in the hidden folder. I wanted everything on one screen to minimize confusing.

See also: 

• Bandwidth use over 5 min video call: FaceTime << Facebook Messenger < Skype
• Father longterm care iPad videoconferencing project: Securing the iPad

- fn - 

[1] I wrote about that project in Wanted - a way to make an old style landline work over a cellular connection. Service was a bit flaky at first, but quality improved substantially and it’s fine now. It turned out to be quite economical to pay for a family member’s iPhone on Rogers then add the “wireless home phone” for $10/month and subsequently add the LTE iPad for $10/month, all sharing data. With this device he can change rooms without a service disruption, and his entire monthly service bill is less than the institutional landline charge.

Bandwidth use over 5 min video call: FaceTime << Facebook Messenger < Skype

As part of my Father longterm care iPad videoconferencing project I compared cellular data use during an approximately 5 minute videoconferencing call made from my iPhone (LTE) to an iPad Air 2 (WiFi). To measure data use I “reset statistics” for Cellular data before, then refreshed the view after concluding the call. I turned microphones off.

The results were:

FaceTime: 7.5MB (repeated, this is correct)s

Messenger: 32MB

Skype: 46MB

FaceTime gave the best image quality. The data use with FaceTime was so low I repeated the measurement with a similar results. Data use can vary with image activity by up to 25%.

I was very surprised by my results. FaceTime had excellent image quality despite exceptional compression. Skype is a real data hog.

The user interfaces were quite similar; names on the left, a details pane on the right. I liked Messengers easy messaging integration, but FaceTime was a 1 touch call from the left side.

My sister and I can do FaceTime, but my brother has an Android phone. I’ll suggest he try Facebook Messenger as he uses Facebook and the data usage was less than Skype.

Father longterm care iPad videoconferencing project: Securing the iPad

My father has been doing well in a Quebec long term care facility for veterans (in Canada that has historically meant WW II, he’s in his 90s). Things are getting tougher though — the facility is shifting from federal to provincial control. Great staff are leaving and programs will be stressed.

I see him every 3-4 months, but in between I was surprised how well Skype worked with him. He does much better speaking when he can see me than he does on the phone. It seems to be related to knowing when to try speaking and when to listen. He also seems to hear Skype sound better than mobile phone sound. (It’s likely much higher quality.)

Even with the old regime though the Skype conferences often failed. Tech complexity and organizational issues forced us to discontinue them.

So now I’m going to try bringing him an LTE iPad Air 2. I’ll get a Rogers SIM card when I visit in a few weeks and we’ll see if it works from his room. If all goes well it will cost him an extra $10-$15/month — and the iPad cost [1].

Dad’s lost a few wallets from his room. I think most longterm care facilities see this kind of problem. Visitors can have issues. So we need to secure his iPad. Other than photo display I think he’ll only be using it for conferencing. So it needs to be secure [2], continuously powered up, stored somewhere he can sit, and not take up much room. The secure device needs to leave speakers and camera clear.

After some thought I ordered the $33 CTA Digital Universal Anti-Theft Security Grip with POS Stand for Tablets - iPad Air 2, iPad mini 4, Galaxy Tab, Note 10.1, 7-10-inch Tablets (PAD-UATGS) (grip and stand). It seems solid enough, it will keep the iPad off his desk, and there are screw holes (but no screws included). It may screw into his (antique) desk, but, even though it’s not shown in the picture, the lock comes with a cable. So I might be able to secure it to his desk in a less damaging and harder to remove way.

Of course the iPad Air 2 is way too thin for this device. It flops around. There’s supposed to be an included adapter strip, but mine was missing. I don’t think it would have worked — this home made setup seems a lot better. I had some TrueValue gripping pads (549104, TV23148) lying around…

I put those inside the corner retainers:

and it works pretty well:

So the first step is complete. Next step will be to test some of the conferencing options for data use and usability with various iPad accessibility features enabled: Skype vs. FaceTime vs. Facebook Messenger (Hangout is not very useable.)

I don’t expect Dad will use it by himself, we’re hoping a friend who helps with him will get things set up. I want it to be useable for them though.

- fn -

[1] (Rant) Incidentally, the iPad Air reminds me what a mixed bag Apple is these days. Nice device in many ways, but when I brought my mother an iPad six years ago one of the features she loved most was it could be used as a high quality digital photo frame. It was easy to launch from the lock screen. She loved that.

So, of course, Apple pulled it from the lock screen around iOS 7 and then ditched the replacement with iOS 9. There’s exactly one half-decent alternative, an app called Picmatic. Not to be confused with spammy copycat apps of the same name in the kinda broken App Store.

I don’t know if Apple is merely senile, or if the app had to be reworked for iOS 9 and it got ditched in a last minute panic to get that half-baked release out the door. Either way, the good news is that now that Ive has retired there’s only Cook to launch.

[2] Would it have killed Apple to incorporate some sort of secure lock feature in the iPad? Ok, yes, it would have.

AVI video files in 2014 on the Mac -- you should probably convert them while you can.

In the process of slogging through an unexpected iMovie migration, I came across some old AVI files that iMovie 2013 basically swallows and hides. 

I dug them out of the Package and started searching for AVI conversion tools (bear with me, I’m out of video practice). Google returned pages of spammy looking hits marketing suspiciously “free” third party tools. 

That’s a clue folks. It typically means a market segment has died because Apple has made it part of OS X. Sure enough, FaceTime in Mavericks and later now converts most AVI files (AVI is a container, the real problem is the funky video compression standards inside the container) to .mov.

I experimented with one old AVI file — the original (low res) 40MB camera clip becomes a 64MB quicktime .mov file with H.264 compression and Microsoft ADPCM audio (I wonder if that was unchanged from the AVI file). The 50% growth is typical of migrating from one lossy format to another lossy format — it would be quite bad news if .mov file were the same size as the AVI file. That would indicate too aggressive compression.

I played the converted file back in QuickTime (Mavericks) and the original in QuickTime 7 and they didn’t look too different. Not bad. (As noted below I actually have Quicktime Pro 7. I’m not sure AVI files play in QT7 without the now defunct Perian plugin).

You can’t control the codec or parameters QuickTime Mavericks/Yosemite uses for conversion, for that you can try QuickTime 7 Pro ($30, I suspect part of that is for licensing codecs). Yes, it’s on the App store! First you download the Snow Leopard installer (works on Mavericks/Yosemite) then you pay. I was about to buy it, but then I thought I should check I didn’t already own it. I bought it in 2008 (!). I really should use it a bit more often.

You can also use Handbrake to convert AVI files, or VLC to view them.

I really need to convert those old files; conversion is only going to get harder. However, it’s weirdly good that QT Pro from Snow Leopard still works and is still sold on the Apple store. So we have a bit of time.

See also

General refs on AVI  

• OS X Mavericks: Some files may be converted when opened in QuickTime Player - Apple Support
• QuickTime will not play AVI files after updatin... | Apple Support Communities
• How to play .avi movies in Mavericks | Macworld - Play in VLC, QuickTime 7/Perian, or convert with Handbrake.
• Run QuickTime Player 7 in OS X Yosemite & Mavericks

Older posts of mine on the horrors of video codecs and compression — there’s been limited progress since 2008

• Why I hate video: Format, codecs, DRM and m4v vs mp4 7/2011
• How should I transcode my home analog video? 2/2012. H.264/MPEG-4 Part 10 (AVC) is as close as we have to a standard
• Video: The state of compression is not good. Blame it on Congress 2/2012
• FLIP Video Ultra camcorder: iMovie HD works, iMovie ’08 might 3/2008 (The short-lived pre-smartphone Flip Video camera taught me a lot about codecs. Specifically, a lot of bad things.

I migrate from iMovie whatever to iMovie 10/2013 whatever and of course it hurts

Dilbert, 4/14/94

In retrospect, things started to go wrong for Apple 9 years ago. Back then Apple’s iPhoto, iMovie and iTunes were all running pretty well on Leopard. Then Apple jumped from iMovie HD 6 to “iMovie ’08”, a complete reboot and major regression. Since then we’ve lived through a series of half-baked regression-heavy reboots all too reminiscent of SONY’s Spiderman. There’ve been reboots of FinalCut, iWork/Pages, iPhoto and now Aperture/Photo.app…

Oops. I forgot iTunes. It gets randomly rebooted yearly.

Three years after iMovie ’08 Apple recovered most of the lost ground with iMovie ’11. Which was then followed by iMovie 10, another reboot with feature regressions.

Yeah, Apple went from “’08” to “’11” to “10”, sometimes called “2013”.

I picked up iMovie 10 as an automatic update — I don’t use iMovie all that much (it’s been a discouraging ride) and I didn’t realize the product had had yet another reboot. So when I launched it yesterday to work on my daughters dance performance it took me a while to realize I was in a world of hurt. iMovie 10 has a new file structure and Apple cut a few corners on the migration tools - particularly for Libraries located on external drives.

The conversion and recovery process was too painful to recount here, but here are a few of things that might help if you’re trying to figure this out:

• Read these 3 Apple tech notes: 
• iMovie (2013): Updating projects and events from previous versions of iMovie. Note the classy way iMovie 2013 prevents old data being imported more than once — it writes a zero byte file “UpdatedtoiMovie10”. Big catch — the iMovie 2013 update process works only on iMovie ’11 files stored in the Movies folder or at the top-level of an external drive.
• iMovie (2013): Removing old libraries after updating. See my note below before doing this. Note there’s nothing in the update process to tell you that you have now doubled your movie storage budget.
• iMovie (2013): Work with multiple libraries
• The new file structure resembles Aperture and iPhoto; iMovie now uses Packages. You can right click on your Movie Library to see the internal folders; they resemble the old file structure.

For my conversion I did the “Update Projects and Events” twice. I first did it for the Projects and Events in my iMovie folder. Then I moved my external (large capacity) files to the root of an external drive and repeated it. Once the new Library Package is created you can move it wherever you want.

I then compared what I had in the old Projects and Events folders to what I saw inside the new Packages. When I was convinced those lined up I tested viewing each movie/video. I found a number of problems that I was able to fix with iMovie by changing metadata and creating events.

I also ran into a curious problem where one Project seemed to have pieces in two Libraries. The fix for that included creating a new empty Library and moving events one at a time into it from a damaged Library. There doesn’t seem to be any Library rebuild/repair function.

There’s a native menu option for moving Events between Libraries, so one can theoretically move them from an SSD to an external drive and vice-versa.

Good luck!

ViewTV At-163 OTA Digital TV recorder - my review

Remember the great American Digital TV Transition of 2008? Our elected representatives explained that while some would see more channels and far better image quality, those who did not have “a properly oriented, high-gain antenna mounted 30 feet in the air outside” would lose free access to broadcast television. They told us that, sadly, a combination of patents, the DMCA and the burgeoning power of cable TV would vastly increase the cost of time-shifting broadcast (OTA) television. They hung their heads and admitted they were completely corrupt …

Ok, so they didn’t say any of those things. We had to learn most of them the hard way, though we knew about the corruption bit. I’d mostly ignored television from 1997 to 2013, so my light bulb only lit up when I bought a Samsung Smart TV and learned that the USB digital output was disabled [1]. Suddenly, everything made sense.

Ever since then I’ve been looking for a sane way to record OTA TV without devoting my life to hacking MythTV. Sure, we could afford to pay the monthly Tivo tax (which presumably goes to the cartel patent holders), but this kind of thing bugs me.

Alas, this is harder than you’d think. A legitimate manufacturer is going to be hit with the patent/cartel tax — turning a $50 product into a $600 (over 5 years) monster. There’s no Chinese market to draw from, I assume they just stick a USB device in their TVs. There are several startups that do sort-of-interesting time shifting things with OTA DTV, but none of them connect easily to, you know, a television. Presumably to dodge the cartel they have to connect to everything BUT a television (so you use AirPlay to get signals from an iPad to a TV).

The only source is going to be shady, something from China sold under a range of disposable brand names for a few months at a time. That’s what I found on Amazon, the home of things that fell off the back of container ships. Alas, the quality was incredibly low — it’s easily the crummiest product I’ve bought in decades. Even so, I was tempted to keep it. It kind of worked…

Amazon.com: John G … s review of Viewtv At-163 ATSC Digital TV Converter…

This is a FASCINATING device in so many ways. It is pure Chinese manufacturing of absolutely the lowest possible quality (one of the four base pads was missing), but it’s only of interest to the US market. The rest of the world doesn't need this -- they simply stick USB devices into their TVs and record directly. It's only in the US that a combination of cartel patents and corrupt politicians makes recording of OTA TV extremely expensive and difficult. When we buy a Samsung TV, for example, the recording features are disabled.

In China this is probably used to convert digital signals for the few analog TVs left in China. It probably costs $10 to make, including what I assume is a very low quality (risky!) internal switched power supply (no power brick). In the US it's sold, somewhat covertly, as a low cost no-subscription simple digital TV recorder.

The good news is that if you ignore the manual, and work through the incredibly crude displays hacked to to show English labels, and convert from the strings on the weird remote to the screen, it's not hard to configure this. You can even set up the antenna as pass-through so the signal goes to the ViewTV and is also available for TV input. It does record (!) and you can play back the recording. It even shows a list of available TV shows.

The bad news is that the tuner in my device sucked. OTA Digital TV barely works in many US settings -- we need top notch tuners. The one in our Samsung Smart TV gets barely enough channels to meet my son's sporting thirst. The tuner in this device missed most of them.

I also worry about the internal switched power supply. This ships with all the usual inspection/certification labels, but I'd be amazed if any of them were legitimate.

If only they'd shaved their profit margin a bit. It costs nothing to build what's in this box. We actually own a decent (Chinese made of course) Digital TV converter box that cost $45 — it’s much higher quality than the “ViewTv" with a far better tuner.

I hope someone else will do that -- but we have to remember the reason these features are disabled from US TVs. Anyone doing this properly is going to be hit by the US OTA patent cartel -- and forced to charge high subscription fees.

There may not be anything better in the near future. So if you have a strong antenna and good signals, and you're feeling lucky, it's worth a shot. Just be ready to return, and if you're feeling nervous leave it unplugged when you're not using it.

I tried to make this thing work, but the tuner was just too crummy. I had to return it.

Sniff.

- fn -

[1] The only explanations I’ve read are something like "Disclaimer: Only European TVs have USB recording. US models have the feature disabled due to legal issues.” I presume the “legal issues” are DMCA and patents. Explaining the full story would take a real journalist — would be a fun article for, say, The Atlantic. Might run into some dangerous characters though…

See also:

• Gordon’s Notes: After the Smart TV debacle: what we really want: modular design
• Channel Master is developing a whole-home DVR solution for cord cutters — Tech News and Analysis: seems to be dying already. Very expensive for what it does - $300 to start, iffy tuner, no antenna pass through, mediocre Amazon reviews. Most of the cost must be paying patents.
• Gordon’s Tech: How to clean up the Samsung Smart TV you shouldn’t have bought 12/2013
• Gordon’s Notes: Four decades of television: last of the rabbit ears 2/2007. An A/D converter, a simple monitor, and an old VCR has advantages over our modern Samsung.

Update: I updated my 12/2013 Samsung Smart TV post with notes on hacking it to restore recording functionality.

iPhoto to Aperture migration: movie support is weak in Aperture, but missing thumbnails are no longer a problem

In the final stage of my iPhoto 8 to Aperture 3.23 migration I imported my a 62 GB iPhoto Library of about 18,629 "Photos", of which, by keyword, 259 were actually "Movie".

Yes - iPhotos support of Movies is half-hearted.

In iPhoto all but one of the 259 videos displayed with a preview image.

After the 8 hours of importing and processing were done I had a 66GB Aperture Library. The first thing I did was check the Movies.

This time there were 270; 259 with the keyword iPhoto Original and 11 with keyword iPhoto Edited. Aperture stacks them as it does photos (option-;).

Of the 270, 36 showed icons indicating they could not be played within Aperture -- though they could play in QuickTime. Another 43 could play in Aperture, but they had no thumbnail (black thumbnail). There didn't seem to be anything different about these movies. Another two had incomplete thumbnails.

There's nothing to be done about the unsupported video formats -- they area  good reminder that the 2010 video format situation is a bloodymess. I will put these on my list of videos to transcode to a modern format -- hopefully without too much quality loss. [1]

The missing thumbnails seemed likewise intractable. I couldn't find any web resources on fixing them. Aperture Help was more ... helpful. Turns out there's now a menu command to generate thumbnails (new in 3.x), that took care of the problem.

Alas, there's no fix for a much more severe problem. Aperture imports iPhoto image Titles and Descriptions, but not iPhoto Movie Titles and Descriptions. The Aperture "Captions" for these movies were empty.

[1] The more I learn about transcoding video, the more terrible it seems. To do it well seems to be dark magic involving several pre-compression processing steps. Maybe I'll transcode to Motion-JPEG and be done with it.

Update 4/2/2012: The Title and Captions for the movies appears to be tied to the "Version Name" bug that can affect importing of iPhoto Titles from images. When I set Version Master = Master Filename on iPhoto import I got Aperture.VersionName and Aperture.Caption values from iPhoto. I also noticed that the Built in "Videos" smart album doesn't show videos Aperture can't manage internally -- so it's misleading (another bug).

The unsupported video format may be from our Canon SD camera, and there are some odd bugs there too. I can open the unrecognized video in QuickTime, save it as .MOV (same data, only the metadata changes) and import it back in -- and now Aperture will recognize and play it. So the problem isn't a codec issues, it's packaging/metadata problem. During the export/import process The Version Name was lost (not to surprising), but in addition the prior version name was set to the Date Created (bizarre beyond words).

Update 4/3/2012: I'm still trying to understand what Aperture is doing with movies imported via browser vs via iPhoto Library import. I think both methods are unforgivably unreliable and buggy, but they have different bugs. I'm experimenting with combining both, then reviewing metadata in list view to decide which to keep.

I recommend using the list view and adjusting metadata to show file size, project name, file name, version name, caption, rating, date and so on. I discovered that several videos were tagged as "iPhoto Externally edited", but they were in reality JPEG thumbnails.

See also

• Gordon's Notes: Why doesn't the world's richest company make better software? 4/2012
• Gordon's Tech: iPhoto 8 to Aperture 3.23: Migration notes 3/2012

iMovie '11: deleting rejected clips reduces video quality

The great thing about my home video archive project [1] is that there are always more pitfalls. I suppose it's a bit like golf that way.

In the last few days, for example, I discovered that the 10 or so hours of analog video I'd painfully captured had only one audio channel [2]. Prior to that I figured out that the existence of an Event repository on an external firewire drive was preventing thumbnail generation with new event capture.

Yes, it reminds me of the joys DesqView/QEMM DOS memory management. Be glad you don't know what I mean.

Even better, last night I discovered that iMovie '11s "Move Rejected Clips to Trash" feature destructively degrades archival video quality. Without, I might add, any warning.

Like all video pitfalls, this surprised me. I remember QuickTime Pro was very good at excising bits of DV files [3]; select the region, delete, quickly get a smaller file. Alas, iMovie doesn't work this way.

Instead iMovie decompresses (decodes to lossless) the clip, which in the case of my Analog to Digital capture is the entire file. [4] Then it excises the unwanted segments and it re-compresses to back to DV-25, changing the packaging as well (from .DV to .MOV). Since DV-25 is a lossy form of compression [5] there's a significant quality degradation from removing unwanted clips, such as the trailing frames iMovie creates after my camera's DV stream ends.

Oh joy. Now I get to redo that tape [6].

So why does Apple do this [7]? Probably because nobody seems to have noticed. Even when I knew what to search on Google found only one reference from 2007 [8]...

harder & harder to remove unwanted...: Apple Support Communities

... I can still crop bits from my events and send those bits to the trash BUT doing so converts the underlying DV movies into .MOV files and DOES modify the quality of the video. I'm not sure if it's significantly worse quality, but I can do a side by side comparison of the original DV and the MOV file that iMovie 08 insists on making if I delete bits permanently from my events and there IS a difference...

... However, they get "squeezed" for some reason, as you pointed out, so that when played directly (i.e. from Finder in QuickTime) they get reduced from their real/original 853x480 to a smaller, squished 720x480...

Even in that thread the participants missed that DV (MPEG-2) is lossy compression; the reassurances were misplaced.

Someday I'll have to see what Final Cut Pro X does. iMovie's callous recompression doesn't give me warm and fuzzy feelings. (Admittedly, nothing about Apple gives me warm and fuzzy feelings any more.)

[1] For the rest of the current series over the past week or two, see:

• How should I transcode my home analog video? 2/13/2012
• iMovie 11 and analog to digital videotape conversion by passthrough
• MPEG Streamclip video converter as Compressor alternative
• Mac mass storage for video archiving - options
• Video: The state of compression is not good. Blame it on Congress. 2/18/2012 (and, perhaps, soon we can blame Google)
• iMovie 11 does not completely import iMovie HD projects

[2] I'd swear one of the capture UIs I used appeared to show stereo. I think the problem was that the Canon STV-250N video cable only looks standard; I think it flips an audio and video input. So one audio was going to the video channel, but since I was using an S-video cable I didn't see this. I'd long ago mixed that cable with others, and it took some detective work and web image study to identify it.
[3] DV-25 technically, though I believe that label conceals a wide variety of messiness including variations in codec version, variations in metadata, variations in sound encoding, differences in 'pixel aspects', interlacing variations and so on. 
[4] Maybe it always does all clips in an Event. I don't know. In this case the Event was a single 10GB clip and it was entirely re-encoded. I also don't know if this continues to happen after the initial DV/MOV reencoding, perhaps iMovie can work with some forms of DV more directly.
[5] Quite lossy by still image standards. It feels comparable to 60% compressed JPEG, so a single re-encoding cycle causes noticeable damage. I have a theory that low compression MPEG-4 (30 Mbps+ data rates) is less lossy than DV, but saying this tends to enrage video "experts" in Apple's Discussion groups.
[6] It looks like I'm going to be running through each A to D conversion an average of 2.5 times. If I could travel back in time two weeks I'd send me to a commercial A to D conversion service.
[7] Aperture does something similar, albeit less obnoxious. Even if you don't edit a JPEG image, if you export it as JPEG Aperture always decompresses and recompresses. There's no intelligent export of the original.
[8] Not the point of this post, but this bit of the thread helps explain why video work is so nasty (more on formats some other day) ...

... QuickTime always reports two dimensions, the native format size — 720x480 — and the playback display size, shown in parenthesis, e.g. "(853x480)". The native size is measured in non-square pixels and the display size in square pixels. (Or is it the other way around?)

.. Throughout the history of QuickTime the playback of DV video has evolved from always displaying it as 720x480 — which confused viewers because the pixel shape of DV caused the video to appeared "stretched" — to adjusting the playback display to 640x480 for 4:3 video and 853x480 for 16:9 video. Sometimes QT doesn't choose the display size we expect, but that shouldn't affect the size or quality of the underlying video. (To force the video to use the aspect ratio we want so it looks right, QuickTime Pro lets us adjust the display size, just like we used to adjust all 720x480 DV to play as 640x480.) ..

iMovie 11 does not completely import iMovie HD projects

I was infuriated by the data loss during iPhoto to Aperture migration, but that's nothing compared to iMovie HD to iMovie 11 migration [1]. All titles and effects are lost.

This happened years before the better known debacle; Final Cut Pro X does not import Final Cut Pro 7 projects.

How does Apple get away with this?

It's not just a rhetorical question.

[1] Prior versions don't import at all, though you can dig the .DV files out of the Packages and import them by hand. Or open prior projects in iMovie HD, then save them then import.

MPEG Streamclip video converter as Compressor alternative

I came across this reading a well written response on the FCP discussion group:

Squared 5 - MPEG Streamclip video converter for Mac and Windows

... MPEG Streamclip is a powerful free video converter, player, editor for Mac and Windows. It can play many movie files, not only MPEGs; it can convert MPEG files between muxed/demuxed formats for authoring; it can encode movies to many formats, including iPod; it can cut, trim and join movies. MPEG Streamclip can also download videos from YouTube and Google by entering the page URL....

Those are impressive features. The post claimed conversion quality equalled that of Apple's Compressor app.

Reading the release notes it's typical video software - somewhat flaky and tricky to use. Some functionality requires companion apps. It's not fully Lion compatible and OS updates may require app reinstallation. Some details on functionality (emphases mine)...

MPEG Streamclip lets you play and edit QuickTime, DV, AVI, MPEG-4, MPEG-1; MPEG-2 or VOB files or transport streams with MPEG, PCM, or AC3 audio (MPEG-2 playback component required); DivX (with DivX 6) and WMV (with Flip4Mac WMV Player). MPEG Streamclip can export all these formats to QuickTime, DV/DV50, AVI/DivX and MPEG-4 with high quality encoding and even uncompressed or HD video.

Video conversion is performed in the YUV color space; you can choose to scale video to any frame size using a professional 2D-FIR scaler (better than bicubic) but you can also leave it unscaled. Other optional video processing features include a powerful motion-adaptive deinterlacer, a field dominance converter, a chroma reinterlacer and an option to perform interlaced scaling instead of progressive scaling. Audio can be converted to uncompressed or to IMA, AAC, MP2 or AMR using the high-quality MP1/MP2/AC3/PCM built-in decoders of MPEG Streamclip; it is also kept in perfect sync with video using a timekeeping system.

MPEG Streamclip can save edited movies as MOV files, and (when possible) as AVI or MP4 files. Edited MPEG-1 or MPEG-2 files are saved as MPEG or TS files.

MPEG Streamclip (with or without the MPEG-2 Playback Component) can also convert MPEG-2 transport streams into muxed MPEG-2 files, for immediate burning at full quality with Toast 6 or 7 and Sizzle; it can also demux MPEG-1 and MPEG-2 files and transport streams with MPEG, AC3, PCM audio to M2V and AIFF (or M1A or AC3) files, for immediate burning at full quality with DVD Studio Pro or Toast 6 and 7. A special demuxing option is available for Final Cut Pro 4/5: this application does not work well with M2V files, but MPEG Streamclip can write a special "unscaled" M2V file that preserves full video quality when imported into Final Cut Pro.

MPEG Streamclip can handle files and streams larger than 4 GB, split in any number of segments, or with multiple audio tracks, and can also optionally handle timecode breaks. It is compatible with MPEG-1 and MPEG-2 video, MPEG layer 1/2 (MP1/MP2) audio, AC3/A52 audio, and PCM audio.

The player included in MPEG Streamclip lets you preview the files and transport streams before doing the conversion; it also lets you visually set the In and Out points for the conversion so you can convert just the part of the file you are interested in, and also cut commercials and other unwanted parts, or edit the stream and join two streams with Cut/Copy/Paste.

MPEG Streamclip supports batch processing: just drag some files in the batch list, choose a conversion and a folder, click the Go button and MPEG Streamclip will automatically convert all your files.

Reading between the lines I suspect it's sometimes used for non-approved editing of commercial products.

Mac mass storage for video archiving - options

I'm in the midst of a bit of project from heck - assembling and managing our home video recordings. This comes to about 40 hours of analog Hi8 and a comparable amount of 2007 DV technology plus an array of camera files from pretty crummy to modern dSLR HD. I estimate the total archival store will be about 1TB -- mostly using DV-25 files (MPEG-2 codec) and similar acquisition formats. (There are no archival video formats in 2012. More on that in a separate post.)

This is too much data to fit my current backup system - a redundant mixture of Time Capsule and Carbon Copy Cloner with offsite rotation. Once the archive is created I don't expect a rapid expansion rate, so the key backup feature is offsite coverage. I don't think this will be on the same rotation schedule as my current backups.

So I think I'm looking at a completely independent system - at least until we can buy $10TB of storage for under $200 (2016?).

I would like to spend less than $1200. I'd like a RAID-1 solution that would allow me to take a drive off-site, bring the off-site drive onsite, and rebuild the mirror. I'd prefer Thunderbolt though it sounds like Firewire 800 may be good enough for what I want to do. I do want it to be fairly quiet, though in fact it needn't run very often.

This is the list of options I'm coming up with:

• Prosumer video storage: sub $1000... (Apple discussion thread I initiated)
• LaCie - LaCie 2big Thunderbolt Series: not clear this is really available yet. Reputation for noisy fans.
• OWC Mercury Elite Pro mini Portable RAID: can be combined with Sonnet's eSata to Thunderbolt converters.
• NewerTech Guardian MAXimus
• Pegasus RAID Storage with Thunderbolt

I may also opt for basic Firewire 800 drives and do the cloning using CCC (will take a while!)

Any suggestions or comments are welcome ...

Video: The state of compression is not good. Blame it on Congress.

The more I study analog to digital video conversion and export compression the more I see I've left civilization.

This is a domain of arcane knowledge and of unknown unknowns -- "experts" who really don't know what they're talking about. I'm sure there are real experts, but they're bored silly by the level of net discussion -- including Apple's own technical notes. It reminds me of my 1981 medical school lectures in renal physiology [1], or my @2002 research into color profiles.

In the absence of publicly accessible expertise I hope I'll have time to do my own experiments, such as creating a range of QuickTime 10.x exports with a span of export parameters. I'm afraid, however, that I don't have that much time.

Which means I probably need to give up on my idea of using .mp4 files the way I use .JPEG -- as an archival storage format [2].

So that means I need to buy more storage -- 2TB should do it. From that point on, given the amount of video we capture, I will just need to upgrade every 2-3 years (4, 8, 16, 32....). And I need to invest in either firewire 800 or thunderbolt.

So now I'm looking at the 1995 era ".dv" format [3]. Should be straightforward. Right?

Yeah, that's what I thought until I read this wikipedia article, which, as is usual for wikipedia, reads like a mixture of a true expert and some confusing amateurs [5] (but is still better than the alternative...):

DV - Wikipedia, the free encyclopedia

... The DV codec was launched in 1995 with joint efforts of leading producers of video camcorders...

... DV video employs interlaced video scanning with the luminance sampling frequency of 13.5 MH...

... One video frame is formed from either 10 or 12 such sequences, depending on scanning rate, which results in a data rate of about 25 Mbit/s for video [4] and an additional 1.5 Mbit/s for audio. This results in a compression rate of 5 to 1...

... All DV variants except for DVCPRO Progressive are recorded to tape within interlaced video stream...

... When video is captured onto a computer it is stored in a container file, which can be either raw DV stream, AVI, WMV or Quicktime. Whichever container is used, the video itself is not re-encoded and represents a complete digital copy of what has been recorded onto tape. If needed, the video can be recorded back to tape to obtain full and lossless copy of the original footage....

So even .DV covers a multitude of sins -- and is a significant compression from the original stream.

Even so, a mixture of .DV and Apple's proprietary professional formats seems the only way to go for now. I hope within a decade we'll have the standards and software to do better.

Or perhaps not. It's worth asking why progress has been so slow over the past decade -- and why the "standards" for still and video images date from the early 1990s. The answer seems to be something Congress and the USPTO did in 1996 ...

USPTO issued Final Computer Related Examination Guidelines stating that "A practical application of a computer-related invention is statutory subject matter. This requirement can be discerned from the variously phrased prohibitions against the patenting of abstract ideas, laws of nature or natural phenomena".

We may need legal reform before we have an archival video format. This is not a technology problem.

[1] After weeks of confident and contradictory statements from researchers and textbooks, I wondered whether they knew they were faking it, or if they'd fooled themselves. I never did find out.
[2] JPEG itself is a non-specific marketing term that hides a very wide range of codec tweaks and metadata "standards". Still, it's a recognized library of congress standard (TIFF is even less standard.). There isn't an image format today that has a comparable archival lifespan, so I use 99% (minimally compressed) JPEG and delete the RAW images -- holding my nose the whole time. Yes, I wish JPEG-2000 had succeeded. Our IP (patent) framework is intensely dysfunctional. 
[4] Now I know what the data rate setting in iMovie 11's QuickTime export parameter means.
[5] In particular there's confusion about DV handling during capture vs. how it's represented during editing/playback and confusion about what containers are. Of course, as a non-expert, I corrected that section.

iMovie 11 and analog to digital videotape conversion by passthrough

it's been six years since I've had much good to say about Apple's OS X software (we miss you Avie). So I'm mildly surprised to say something good about iMovie '11.

I haven't paid much attention to iMovie since the iMovie HD debacle, but after Final Cut Pro X and QuickTime Pro failed me I turned to my free copy of iMovie [1]. Short of Adobe Premiere, it was the last tool to use for my family video consolidation project ...

Gordon's Tech: How should I transcode my home analog video?:

... Emily picked this project for her Valentine gift, so we're off. I found a 300GB drive and LaCie firewire enclosure I'd forgotten about. So far video capture seems to be proceeding well from a 12 yo Hi8 analog tape; there's a thin sliver of artifact at base; I think capture frame is bit taller than the image frame. Setup at the moment:

• SONY Hi-8 TRV65 (1997) with S-video and RCA audio out to ...
• Canon Optura 50 (2004) with firewire out via secondary port on LaCie to ...
• iMac i5 2.7GHz, 8GB RAM
• Final Cut Pro X (trial version. There's no more academic pricing for Apple products distributed through app store. I expect I'll buy full version. So far this is the big expense) saving project to ...
• External firewire LaCie drive (300GB)

I started with FCPX, but it only seemed to be capturing video. Video streamed to FCPX, but was not recorded. FCPX can't handle non-timestamped video streams:

• How to "Uncontroll Device" in FCP X: Apple Support Communities. Capture via QuickTime 10 Player and import into FCP. Unfortunately this didn't work for me (see below)
• Setting FCP X to "Non-Controllable...: Apple Support Communities - problem is lack of timestamp.

So, after reading about 10GB import limits for QuickTime Pro, I tried iMovie 8 (iMovie '11). These were the steps I followed:

1. Connected the Hi-8 camcorder via S-Video and RCA to the Optura
2. Set Optura to Passthrough mode (most people now would use a Canopus device or the like to do A/D conversion - workflow is very similar)
3. Turned off Time Machine -- I don't want the 30 GB videos going to my backup drive [2].
4. Start iMovie. It's safest to create the Event (will hold the video, analogous to Events in iPhoto) first so you can choose what drive the event will go to (there's a small icon that lets you see how Events are distributed over media).
5. Start Import in Manual mode and direct to the Event desired. Push Play on camera.
6. iMovie will give you a few seconds to get the video in. It shows the Canon as no tape and displays 'Ready to Record'. If the stream doesn't start iMovie will quit.
7. If recording is working the counter will increment and a small red dot will show. The 'Ready to Record' message never changes.
8. When recording is done iMovie stops (1 minute delay)
9. Then thumbnails are supposed to be created (but see below) ...
10. After all is done, move the Event to an external drive (Cmd-Drag and drop within iMovie.)

I start the when I get home from work then let it run. A 1.5 hour tape produced a 25GB .DV file and 1.5 GB thumbnails.

To export to .m4v (MP4 variant) you first create a Project. I edit my .DV by selecting the parts I don't want and tapping the 'R' (remove) key. Use Shift-Command-A to undo select all; it's easy to select all and not clear how to undo. Study the keyboard shortcuts, I don't like iMovie's standard or context menus. Editing was easier than I'd expected, even for a project this size. I'm just getting rid of the obvious junk.

When done select all and drag to Project window. Then add a title slide.

After the Project is done choose Share and large size. On my iMac compression to 960x540 took about 1 hour and produced a 1.5 G .m4v (mpeg) file; it uses "270% of CPU". I'd prefer a bit less compression, maybe a 3GB output. I'm studying my options. It's ok to save the .m4v to an external drive.

I then move use iMovie to move the entire Event to an external drive for storage (see update, these directions are wrong). Empty trash and restart Time Machine.

The Events, for now, are staying on my external store (1TB+, so should handle about 8-9 of my media. I don't want to keep the .DV files, when I'm satisfied I can produce a high quality .mpeg movie with reasonable compression I'll store that alone. Three 1.0 TB drives should manage all of our video - one onsite, one offsite backup, one rotation [3].

Of course there are bugs ....

• iMovie crashed twice (kernel exception -- it really doesn't like the cheap external drive I'm using until the better ones come along [3]
• iMovie always hangs at the end of thumbnail generation, no matter how short the event. This is a replicable bug. I've tried deleting preferences; this may also be a sign these Project are too big for iMovie. I'm hunting around for an old copy of Final Cute Pro or Final Cut Express. (By Darwin, Apple screwed the video market.) [UPDATE: With my firewire drive unmounted, thumbnail generation does not hang.]

It's early days, but I'm less displeased than I'd expected. I think if iMovie fails I'll try Adobe Premiere Elements next.

[1] When Apple introduced the App Store, they also turned to iOS style licensing. iLife '11 came with my MacBook Air, and was registered with my Apple Store ID. That meant it showed up in "purchases" on my iMac and I could download and install it.

[2] Trick is to remember to turn it on again after the iMovie Event has been moved to an external drive. Streaming capture to my old external firewire enclosure did not work. Not only were frames dropped, but iMovie crashed as well.

[3] Future post pending on the external storage/backup system I'm going to use.

Update: Possible export options (ref)

• MPEG-4
• frame size: largest possible (when I set to be the source size however, the output still omits about 3% of the frame)
• key frames: <10
• data rate: DV data rate is 25Mbit/s for video and 1.5 Mbit/s for audio. The default on .mp4 is about 4Mbit/s.

There's surprisingly little quality information on these settings. Reading the Final Cut Pro 7 documentation makes me envious.

Update 2/17/2012: Apples directions for moving Event's are incorrect. If you drag and drop the Event is copied. To move you have to Cmd-Drag, just as in the file system.

Update 2/18/2012: I realized that, by accident, I was using a low end firewire 400 cable with a 400/800 converter to my external drive. That is a bad idea! I would expect it to cause issues. I switched to my LaCie 400/800 cable. I've also given up on the idea of using .MP4 as my archival format.

I've also experimented with capturing via QuickTime 10 at 'highest qualilty'. That created a .mov file that is almost exactly the same size as the .dv file that iMovie crated from the same source; at heart it's a DV file. There's a trick to get this to work with passthrough video. I had to start the stream, then click the record button. When I move these into an iMovie Event library iMovie simply moves the .mov file then creates thumbnails.

I've read FCPX will import these. I'm experimenting more with this.

Update 2/18/2012b: OK, the QuickTime method doesn't work with this particular video stream. I didn't run into any time limit, but there's no sound on playback -- even though there's sound during the capture process.

Update 2/19/2012: I was able to complete a video capture without the thumbnail hang. Two things were different. My external drive (firewire 400) was not mounted, and I stopped the stream manually rather than let it self-terminate when the video stream stopped. I have another post pending on why I gave up on the idea of .mp4 export. For now I'm collecting 30GB .DV files for each tape. Update: It's not the manual termination. it's that my firewire drive was accidentally unmounted. I've replicated this.

Update 2/20/2012: I tested importing one of the analog to digital projects I created in iMovie 11 into FCPX. FCPX seemed to import the modest project and very large clip without obvious problems.

How should I transcode my home analog video?

Twelve years ago I was keen to digitize my home video tapes - on a 400 MHz Celeron system.

Eight years ago I wrote "The thought of losing the kids' taped videos is not comforting. I need to do this sooner rather than later.". At that time I experiment with transcoding through a Canon digital camera using iMovie and a G3 iMac. I figured my G5 would do the job in 2005, maybe with its capacious 40GB drive and a stack of DVDs.

Now I have a slightly aged 2.7GHz i5, 8GB RAM, and several TB drives -- and my analog tapes are 8 years older. Maybe I'll do it this year, even though I hate digital video [1].

I'm trying to warm to the idea. The big difference over 8 years ago isn't processing power [2], it's cost/MB and the gradual emergence of H.264/MPEG-4 Part 10 (AVC) as a relatively standard format for video. The National Archives recognizes there's no real AV standard, but their list of formats they work with includes MPEG4.

If I do finally walk though this, I don't want to bother with DVDs. My current plan is:

• Purchase two 1TB drives. Digitize everything we have to one of the drives as MPEG4 video and MPEG4 audio.
• Clone that drive and keep one drive on site and one drive offsite, repeat every few years.[3]
• Use my old Canon digital camera with passthrough to digitize my analog tapes, then compress on my iMac and store with a file naming convention to reflect date and time information. (I'll consider the Canopus though.)
• Convert my collection of digital tapes (DV) to H.264 as well

I don't know what software I'll need. Can I do what I need to do with iMovie/QuickTime Pro, or should I buy Final Cut Pro X? I'm not planning to edit, but if I were I'm pretty sure the current fork of iMovie won't do the job.

Update: Apple offers a free download for FCP/X. So I will probably try it out.

See also:

• Good advice on digitizing video (Tidbits) 12/2004
• Home Video Editing started 6/2000! (Pass through with a Canon ZR 60 in 2003)
• Digitizing Video: Tapes - extensive Macintouch archives
• BYTE Media Lab 2004 Imaging Awards: Analog to Digital Converters 4/2004
• Why I hate video: Format, codecs, DRM and m4v vs mp4 7/2011
• Amazon.com: Canopus 77010150100 ADVC110 Converter: Electronics (old tech, good reviews, $218) 2/2012
• REC.VIDEO Google groups/usenet posting: Video degradation with S-Video analog transfer vs. A/V mini-jack connection 3/2005

[1] In Jan of 2010 I discovered a 12 yo WMV formatted video was unplayable; I was able to use Windows Movie Maker to convert the 23MB WMV file to a 311MB DV AVI file. Digital image formats are a friggin' mess, but video is at least ten times worse. No metadata standards, crazy patents, non-standard containers, audio codecs, video codecs - yech.

[2] Yeah, the machines are a hundred times faster. But compression can run overnight.

[3] By 2020 I assume we'll have 100TB main system and backup drives, so storing my old video with my routine data will be trivial. By then a single snapshot will be a GB.

Update 12/14/2012: Emily picked this project for her Valentine gift, so we're off. I found a 300GB drive and LaCie firewire enclosure I'd forgotten about. So far video capture seems to be proceeding well from a 12 yo Hi8 analog tape; there's a thin sliver of artifact at base; I think capture frame is bit taller than the image frame. Setup at the moment:

• SONY Hi-8 TRV65 with S-video and RCA audio out to ...
• Canon Optura 50 with firewire out via secondary port on LaCie to ...
• iMac i5 2.7GHz, 8GB RAM
• Final Cut Pro X (trial version. There's no more academic pricing for Apple products distributed through app store. I expect I'll buy full version. So far this is the big expense) saving project to ...
• External firewire LaCie drive (300GB)

Both cameras are on external power. I needed my old manuals, had the SONY and found the Optura online. Getting passthrough working was balky, FCPX only gave me about 10 seconds to start then it turned off import due to lack of a signal. It can't operate the Optura 50 in pass-through mode, I haven't tried yet with a tape installed. Once I hit the play button fast enough I was ok.

I'll have data on file size tomorrow. FCPX imports using its default Apple ProRes format.

Operating the old SONY feels primeval. It's 15 years old, feels 50. It is weird to see the kid video. Plan is to put a library of MP4 versions (no editing really) on iTunes on a an old G5 with a largely unused drive. We can browse, play from iTunes.

Update 2/15/2012: Not so good. Nothing captured! Looks like I need to read the manual, or perhaps FCPX won't work at all. It certainly showed the video, but it didn't save the event. It also didn't stop recording when the input ended.

Update 2/15/2012b: QuickTime Pro is not an option. I was able to create a @2 hour 25GB .dv file using iMovie 11, but then iMovie 11 got stuck 'generating thumbnails'. I think that bug, however, was triggered by connecting an iPhone to my iMovie machine. Disconnecting the iPhone did not clear iMovie, I had to kill it. On restart my video appears to have been captured and iMovie seems to have stopped input about 2 minutes after the end of tape. Thumbnails are fine - all 1.5GB of them. More on the results and process in a follow-up post. I'd like to take a look at Adobe Premiere Elements.

Update 2/16/2012: Gordon's Tech: iMovie 11 and analog to digital videotape conversion by passthrough.

Update 2/17/2012: I experimented with QT 10 movie capture. The trick with passthrough conversion is to first start the video stream, the click record. Otherwise QT exits. I captured 30 min of video at 9 GB; QT Inspector says the codec inside the saved .mov file is dv.

I've read that this will import into FCPX, so I may try that again. My next experiment will be to see if I can record over 10 GB (1 hour of video). There are many reports of QT recording stopping with a 'size limit' message at varying file sizes, in my case that's what displayed after I stopped the video stream.

Lessons from sharing our team videos

In a burst of foolish optimism, I volunteered to do some videos of our team pitchers and share them.

This turned out to be much harder than I imagined. It's one of those tasks where each step has multiple options, but only a few choices really work.

Along the way I tested and abandoned MobileMe's video gallery [1] and Karelia Sandvox [2]. I briefly considered then discarded Picasa Web album video sharing.

I did figure out a path that works. Two of 'em actually. I'll share the easy one first.

The easy option

Use an iPhone. Take short clips. Don't edit. Upload. Share links.

The much more painful deluxe option

The deluxe option assembling multiple video fragments from a Canon dLSR HD video camera into a one video for each pitcher, then embedding them in a web page.

If I ever do this again, this is what I'll do for the deluxe option.

I. Getting the video

1. Bring a tripod (!) and an external microphone.
2. Have the coach use the external microphone to narrate comments.

II. Use iMovie and share via YouTube hidden links

This was the first time I used the new iMovie. I read a few pages in the surprisingly well done Portable Genius Guide to iLife (see [3]).

1. Each player gets one Project/Movie.
2. Edit in 3:4 ratio -- this is the pitcher we're working on.
3. From iMovie share to YouTube as "private" at the highest available resolution.
4. In YouTube change these to "hidden".

This is time consuming. It took about 10 minutes for each clip to create a movie and upload. An alternative would be to export as .mp4 (NOT default .m4v) then bulk upload overnight [4]

III. Share images using Blogger and MarsEdit or HTML markup

1. I tried a few web page editors, but, as noted above, I didn't have much luck.
2. Instead I used YouTube's embed code (iframe markup) and pasted the embed text into the MarsEdit HTML view for each video. It was tedious but gave good results.

- fn -

[1] I'd not tried it before. Now I see why Apple gave up on the Galleries.
[2] Crashed on me during my video uploading attempts. Could be just bad luck -- pretty much every OS X app I use crashes sooner or later. Almost like 10.6.7 is an unhappy host OS. Still, bad timing.
[3] iMovie notes

• Clip Library is a pool of shared clips that can be included by reference in multiple Projects (movies). Clips can be stored in iPhoto, Aperture 3+ or iMovie. I think Clip processing is smoothest if they live in iMovie. Clips can be split, reorganized, rated, merged. Even deleted, though that's not obvious.
• A "project" is a movie.
• In a clip or a project/movie click to set start point, space to play
• click then drag to create a frame within a single clip (can't span clip): Click into  this frame and drag and drop to the Project area. It took me forever to understand this. I kept thinking I had to edit the clip first.
• Native export is .m4v -> evil, vile, worthless, foul spawn of satan. Want .mp4

[4] The settings to make this work are not obvious. I got decent results when I used Export to Quicktime, MP4, then set data rate to 4096, image size 768x576, Fit within size for crop, and "best quality" encoding mode in video options.

Why I hate video: Format, codecs, DRM and m4v vs mp4

My version of iMovie exports (09), by default, .m4v. (emphases mine)

The M4V file format is a video file format developed by Apple and is very close to MP4 format. The differences are the optional Apple's DRM copyright protection, and the treatment of AC3 (Dolby Digital) audio which is not standardized for MP4 container.

Google's Picasa service doesn't support .m4v ...

Video Upload Requirements : Video - Picasa Help:

... Uploadable Video File Types .3gp, .avi, .asf, .mov, .wmv, .mpg, .mp4, .m2t, .mmv, .m2ts ...

Neither does YouTube ...

Supported YouTube file formats - YouTube Help

WebM files - Vp8 video codec and Vorbis Audio codecs

.MPEG4, 3GPP and MOV files - Typically supporting h264, mpeg4 video codecs, and AAC audio codec

.AVI - Many cameras output this format - typically the video codec is MJPEG and audio is PCM

.MPEGPS - Typically supporting MPEG2 video codec and MP2 audio

.WMV

.FLV - Adobe-FLV1 video codec, MP3 audio

Heavens, but I do hate video data standard issues.

It's the patents, it's the DRM, and above all, it's Apple. Data formats and DRM are at the core of Apple's great flaw -- a deep addiction to data lock [1].

[1] Pro video customers of Apple's Final Cut Pro are learning all about what Apple's data lock means.

Update 7/5/11: Two wikipedia articles on Apple's ProRes 422 and ProRes 4444 and these additional articles help capture the full horror of the 2011 state of video codecs -- and the complexity of the video editing workflow. There is nothing analogous to JPEG or even JPEG 2000. See also ...

• Apple ProRes - MultimediaWiki
• Final Cut Pro 7 - Working with Apple ProRes Codecs - "Apple ProRes is an extremely flexible post-production format. The post-production possibilities with Apple ProRes range from offline editing on a MacBook Pro to advanced “digital negative” workflows for theatrical releases on film."
• Final Cut Pro 7 - Types of Apple ProRes Codecs

FCP X didn't add anything new to ProRes (mercifully). It will do "native editing" on h.264, which sounds interesting.

Facetime connections to elderly parents - a Logitech webcam problem

I mentioned a few weeks ago that I was testing a Facetime videolink to my mother. It's not my first attempt. I'd tried Google Video Chat two years ago, but after months of struggle I gave up; it had, and still has, dismal usability. iChat was even worse. In all cases I've been using the excellent Logitech QuickCAm Vision Pro for Mac. (Still the best webcam ever sold, though I fear it's going away without a true replacement.)

After a few weeks of testing I can report that Facetime is a big usability improvement over Google Video Chat. I configured my mother's machine to auto-answer my calls; I can call from my phone or desktop and her machine will pick up. Facetime doesn't need to be running, OS X 10.6.x will launch it.

There's only one problem.

After I close the call at my end Facetime continues to run on her machine. It doesn't auto-exit (and, at this time, she can't see well enough to reliably quit the app) [1]. This means her webcam stays powered on [3]. Under some conditions, perhaps mostly time, the embedded OS that manages in-camera focus and exposure control crashes. The Webcam still works, but it focuses to infinity and the light levels are very low. If you pull the USB cable, wait a few seconds, then plug it in again, the camera will reset.

I'm considering a few workarounds. Firstly, it would be great if Apple officially supported auto-answer, so FT could then auto-exit on close. Alternatively I could

• Run a cron job to restart her machine once a night - thereby quitting FT and turning off her camera (see update - there's an easier fix - just set this in "Energy Saver"
• Run AppleScript to quit Facetime.app nightly. Unfortunately FT is not scriptable [2], but this article suggests options: How To Create a Security Cam with FaceTime for Mac and AppleScript | Mac|Life
• Resign myself to getting remote control of her computer working again (Windows has a big advantage here, RDP is light years ahead of Apple's VNC technology).

I'm leaning to the nightly restart as the simplest fix, but I should also try remote control -- again!

[1] As her macular degeneration has progressed we've been focusing on her iPad use.
[2] Apple needs to kill AppleScript, but I fear there'd be not replacement. 
[3] The webcam then stays in active mode, so it appears like it's always sharing an image.

See also:

• Gordon's Tech: FaceTime: AutoAnswer, URL, desktop 1 click call

Update:

When a Google search doesn't return much, it's often because the function one is seeking is now a part of the OS.

OS X Energy Saver allows one to schedule a restart. I'll schedule my mothers machine to reboot at 2am daily, that should clear out any dangling FT sessions.

Incidentally, there's a longstanding, perhaps ancient, UI flaw with OS X Energy Save scheduling. Look at this:

It looks like the first option is available for selection, but the second (schedule restart) is unselectable -- it's "grayed out".

Look carefully (it took me a while). The select box (drop down) on the first row is also grayed out. This is standard behavior. The reason the 2nd row is so confusing is that it starts with a drop down -- there's no preceding text to display in normal font. Despite appearances this row is available for selection. Just click the check box.

I deleted a prefs file and did a number of Google searches before I realized what was going on. I found others who made the same mistake ...