Digitizing home video (once again)

I have made (2000) several (2004) tries (2012) at digitizing our family video but my lost best effort only got about 10% done. I figured I’d just hire a local service and then I let it slide.

Apple’s recent codec retirement announcements prompted me to check what was available locally. I found ancient web sites that were internally inconsistent, no noise reduction prior to compression, unclear codec choices … none of it gave me much confidence. (But see [1,2])

So I’m back at it again. This time I might have an accomplice — someone who needs money and would benefit from learning the tech. So maybe we’ll make a better go of it.

The delays may have let to some data loss, but on the other hand the tech is a bit better. My first attempt would have been with a 400MHz Celeron. Yes, that’s an ‘M’. In those days hard drives were measured in tens of GBs. Now the cheapest hard drive I can buy would hold all of our video.

The tech is a bit better, but choice of codec is still an issue. In 2004 I favored H.264/AAC. I ran into an amazing number of headaches with the Apple software I was using.

For the modern era I found three good references:

• DIY VHS Preservation: Planning for Video Digitization at the American University Library
• Audio Visual Digitization Hardware and Software - Marriott Library - The University of Utah
• Building a Video Preservation Rack for In-House Digitization AV CLUB | Issue 1 | Cardinal Tales

They give me a feeling of how tricky it is to do analog video capture well. Time Base Corrector?  BNC terminations? Waveform monitors? CRT monitors?! Yikes.

I did like the sounds of the BlackMagic Intensity digitizer ($240 for T3, $200 for USB 3) used at AUL (Amazon reviews are not great however). It can save output as a lossless file. I want to capture the video as “uncompressed 10-bit 4:2:2” then denoise it, then export as ProRes. Since my accomplice is a student I’ll probably buy the Pro Apps Bundle.

Ideally the process would be automated - capture uncompressed, denoise and incorporate metadata, save as ProRes.

What would I do with this material once it’s digitized? The tech isn’t here yet, but eventually I’d like to incorporate brief (silent) video fragments into my screensaver library. So between showing 10,000 images, show a 30sec of video from our family @ 1995. One day?

- fn- 

[1] From my 2000 page I see Walmart and Target were do video to DVD-R conversions for $35 a tape with YesVideo. They are still around! The price is now $26 for one 2hr tape.  

[2] Pogue years ago recommended Southtree (his screenshot of a VHS tape on a modern laptop screen is remarkable — 333x480 pixels). Their site is impressive; at the moment they’re advertising $57 for up to 3 tapes on 1 thumb drive. I contacted Southtree to ask about denoise/ProRes/etc but they kindly responded that they are consumer-only, so just mp4 compressed.

1Password Logins Notes field blank? Maybe it's the CR.

When I moved to Mojave I had to give up on my ancient FileMaker Pro database. The modern versions of FileMaker are far too expensive; consumer databases have passed into history.

For lack of a better alternative I moved over to 1Password. I liked them when they worked with a local password store, I’m not keen on their current cloud solution. I just don’t trust their tech.

I exported as CSV as I’d done many times before. This time, though, I ran into a problem that was probably always there. The first time I went to look up my ‘secret question answers’ they were nowhere to be seen. The Login Notes field where I’d imported them was empty on both iOS and mac OS.

Later I realized the data was there — but only in edit mode. If the default read-only view the notes field showed as empty.

It took a bit of playing around to realize what’s wrong.

Text fields in my version of FileMaker, probably from 2014 or earlier, uses the old classic Mac OS line separator - the single “CR” code. It doesn’t use the OS X/macOS/Unix standard LF or the old DOS CR/LF. 

When I exported as CSV the output used CR as a separator. When the CSV was imported into 1Password the CR separators went along. They work fine in edit mode but not in view mode.

The proof of the problem was to edit in 1Password, deleting the line feeds and adding new ones. After saving the note displayed correctly.

Update: Per 1Password tech support — turning off markdown formatting causes the notes to appear!

Converting my old video formats to something that might persist

Apple is killing QuickTime 7 (download 22MB Snow Leopard version here)[1]. The announcement has some useful references.

So I’m back to thinking about one of the worst topics in the geek world — video file formats and codecs. Almost all of which are encumbered by walls of vicious patents (AV1 is the great hope).

It’s been about 4 years since I last made stab at this topic. I reinstalled QT Pro then too!

Conversion from old formats is a PITA. In 2015 I converted an old WMV file (I think I used Flip4Mac [2]) to (I think) lossless uncompressed AVI. The original was 23MB, the AVI output was 311MB. Today I used QuickTime Player 10.5 to open the AVI and it created a 52MB (lossy) .MOV file (Info says it’s H.264 for video, AAC for audio). There’s no control on the conversion compression.

To get this done I’m going to have locate my video files (some in iMovie projects, some in the file system, some in Aperture) and apply some kind of batch conversion where needed. Spotlight’s ability to search for codecs might help, I suspect some useful utilities will show up now that this is getting some attention.

What should I convert to? Ideally I’d choose something designed for video editing. The Smithsonian recommends "Motion JPEG 2000, MOV, AVI”; I don’t understand how MOV gets in there, I thought it was just a container. Also MOTION JPEG 2000 is on Apple’s kill list.

Ok, so things are bad. But we knew that.

Apple’s ProRes is one example of what’s known as an “intermediate codec”. Apple seems to be dedicated to it and I gather it’s widely used in the video industry. So let’s see what other choices there are ...

Fifty. That link lists 50 intermediate codecs. Ugh. Maybe ProRes 442 HQ isn’t the worst option.

I’ll probably have to play around and study some more. Some things I’ll look at ...

1. Wondershare is $40 for 1 year. There’s a free demo that converts the first 1/3 of the video. It converted an old .wmv file to an HEVC (H.265, AAC) without blinking an eye. Judging by the 1/3 converted the 23MB WMV file would convert to an HEVC file of similar size.
2. Apple Compressor: $50, or have my college son buy the whole ProApps bundle (Compressor, FinalCut, etc) for eduction for $200.
3. QuickTime Player’s built-in conversion.

See also:

• “Swiss Army knife for QuickTime” Perian turning off the lights 2012
• Why I hate video: Format, codecs, DRM and m4v vs mp4 2011. Still hate ‘em.
• MPEG Streamclip video converter as Compressor alternative 2012. This was last update for Mountain Lion!
• AVI video files in 2014 on the Mac -- you should probably convert them while you can.
• FLIP Video Ultra camcorder: iMovie HD works, iMovie '08 might 2008. FLIP used 3ivx MPEG-4, iMovie ’08 couldn’t use it, but iMovie HD could if the 3ivx coded was installed.
• Video codecs: iPhoto, iMovie HD, iMovie '08 and FLIP Video (2008 again!)
•  . Fun stuff.

- fn -

[1] Yes, that’s MB. Wow. It does indeed run on Mojave! Not only that, but I dimly remembered that I had a registration code for QT 7 Pro. Being a total nerd I still have it on hand. I entered the code and clicked “register”, but the registration server is long gone. It still registered though, I quit and restarted and I have the Pro features back.

[2] Replaced by Switch. For $10 you can convert wmv to mp4, for $200 you can to more output formats. If you search wmv to mov conversion you get a lot of junk, so this is worth paying attention to. I might buy it.

Replacing Sierra's "All My Files" in Mojave

OS X  / macOS had, for years, a Finder Favorite that listed all files by date modified. It was removed in Sierra for unknown reasons.

The closest replacement I’ve found is to save a “This Mac” spotlight search on ‘.’ That is, create a Spotlight search for your Mac and enter a period as search criteria.

This works on Windows 7 too (I don’t have Win 10).

Recovering from a photo crop error when you need to print 4x6

I needed to print our team hockey photo at Walgreens, but somehow I’d saved a cropped version of the team photo instead of the original — and these days Walgreens doesn’t let one print an image without cropping [1]. Part of the image was cut.

My first thought was to custom print from Aperture to PDF then export a JPEG from that (it’s complicated) — but something funny happens with page orientation in Mojave/PDF.

The real answer was to use my old copy of Acorn (still works in Mojave). I resized the canvas to fit the horizontal image and fill in the right 3:2 horizontal:vertical orientation. Then I threw in a title for photo. Looked like it was deliberate.

[1] Once upon a time I think that was an option — just printed with white borders.

Mojave has brought back one of my least favorite macOS behaviors - screen saver bug.

When I used Sierra my screen saver was based on Aperture albums stored in an external drive. When I traveled the screen saver switched the default, but when I reconnected it went back to Aperture.

Mojave can’t use Aperture/iPhoto albums. So Screen Saver points to a share drive. When I travel it switches to default, but when I return it *doesn’t* switch back to my share.

Anyone have a workaround? I wonder about creating some kind of local alias for the remote image folder ...

Update: I created a symlink for the share folder that has my slideshow images using the SymbolicLinker service. Then I pointed ScreenSaver slide show to the symlink. When disconnected ScreenSaver shows black screen. When connected shows my images.

Migrating from Blogger to WordPress ... again ...

I’ve been contemplating migration to WordPress for almost a decade, but Google kept Blogger good enough to keep that headache at bay.

Alas, the days of good enough are ending. Google is removing their photo management API without recourse. They do support posts with images, but only by using their web interface. It’s a concrete and undeniable sign that Blogger is either dead or going to a bad place.

I though I’d migrate first to wordpress.com then to my Dreamhost open source wp install, but via Twitter Daniel Jalkut tells me he got better results using the open source importer directly.

I’ll do a dry run on one of my big blogs first. The URL won’t change but I’m sure feed subscriptions will have to be redone (ugh).

Update 2019/04/06 - results of the pipdig import process

I tested the Dreamhost free version of the pipdig importer from a Dreamhost wordpress (open source) blog. The results can be seen here for the moment, I’ll eventually delete them. I found:

• It doesn't remap internal links. This is a big disappointment. Links continue to direct to blogger, once that account is gone they will be invalid
• There’s no option to migrate images that I can see.
• It missed at least 4 posts from the source blog — specifically from early on. No idea why and it suggests more are missing.
• It does copy drafts over.
• It requires a LOT of access to your Google account! If you use this utility I suggest creating a new google account, give it access to your blog, then after the import destroy it. 
• The paragraph breaks are missing - line feeds vs <p>. This is an ancient Blogger problem with MarsEdit; a legacy of the original sin of English language text formatting end-of-line standards. I think Blogger is mostly to blame.
• Images were not relocated locally, they remain at their original locations.

Pipdig is better than nothing, but I’m going to try wordpress.com’s import tool next. I wonder if a better solution wouldn’t be a static site that I could archive on my personal web server, then do a web server redirect to handle the links. For now I’m still on Blogger. The porting experience reminds me of the impossibility of leaving Apple’s defunct Aperture photo management app.

(As I write this the wordpress import is processing - result should eventually show up at gordontest.tech.blog temporarily, but we’ll see if it works. It’s taking a long time.)

Update 2019/04/06b

Well, that wordpress migration didn’t go so well:

Your site has been suspended from WordPress.com for violating the Terms of Service. If you believe this was done in error, please contact us as soon as possible to have the suspension reviewed….

I sent a contact inquiry, nothing yet.