Tuesday, March 06, 2012

Pay-as-you-go voice and SMS service for a contract-free AT&T iPhone with H2O Wireless

Four months ago AT&T declared war on us. Until then I'd had both children on our family plan, using old out-of-contract (but not unlocked!) iPhones with voice only service. On a day of infamy, AT&T hit us with mandatory $30/month data plans. Our too high monthly mobile bills went much higher.

That's when I fought back. Four weeks later, I declared victory. I'd slashed our monthly phone bills, not least by replacing SMS texting with iMessage (even to the SIM-less iPhone 4) and rare paid SMS. As a side-effect of operation vengeance I even picked up a new iPhone 4S - iMessage and a new subsidy made it cost-effective.

There was only one downside. Number 2 son loved using my iPhone 4 as an iPod Touch, but he no longer had voice service. He really didn't care about that, but we're heading to DC for a family trip. He has a knack for getting lost, so I wanted to be able to phone him.  I decided to follow up on an H2O wireless plan (AT&T reseller) I'd considered last November ...

Giving your old iPhone to your kid: working around AT&T's mandatory data plan

I'm planning to test a H2O Wireless SIM Card (no jailbreak or unlocking for AT&T phones) ... there's a MyH2O app on the App Store. However the H2O wireless cards expire after 30 days, so they're better suited to a heavy voice/data user than to our guys; there's really no saving over our family plan....

I dropped into Best Buy to check out the options. I found two quite different H2O plans:

  • a voice and SMS only plan with 90 day expiration. $10 price includes $7 of talk time.
  • a voice/SMS/data plan with a 5-30 day expiration depending whether payment is per-minute or per-month [1]. Not clear if $10 package includes any services.

I had to make a quick call, and based on the premise that I should only buy what I need now, I went for the voice/SMS only. [2] For you, dear reader, I suggest carefully studying the cost of the new $100 1 year expiration option for the voice/data plan. It's not clear how one switches from voice/SMS to voice/data, I suspect it involves buying a new SIM card and switching the old number (see Help, My Account)

Briefly, it worked. Here's what I did after I bought the card (the procedures for data plan support are slightly more involved, but I'm only writing on what worked for me):

  • Went to a local "World of Wireless" shop and paid them $5 to punch out a micro-SIM from the H2) standard SIM [3]
  • Followed the directions and went to H2OWirelessNow.com/activate
    • Registered, providing my Yahoo.com (junk) email and my Google Voice number. Maybe overly protective, I noticed that the 'spam-me' checkbox was opt-in, which is commendable.
    • After I registered Chrome "sat there". I had to lick 'Activate" on the menu to get to the next step.
    • From Activate I entered the ActFast code, desired area code, and then city.
  • I then picked up my confirmation mail from Yahoo.com account. It said the phone would be ready in 10 minutes and told me my new number.
  • About 6 minutes later I put the micro-SIM in the iPhone and powered it up. On startup it said H2O in the connection bar.
  • I tried phoning out, but nothing happened. So I called into the new phone -- that worked. After that I could call out too.
  • iMessage still worked (yay).
  • Dial *777# send to check account balance and expiration. It said my balance was $6.55 and it would expire on 5/5/12 (60 days, not 90!)
  • Tested MyH2O.app - worked quite well to show balance. I found a very brief call cost about 5 cents or so.
  • I connected to the recharge page and added $10. That brought my total to $16.96 and moved the expiration to 5/4/2012 (90 days)

I'm pleased. I may add $10 every few months, that will likely cover Number 2's use of the iPhone 4 -- at a wee fraction of what AT&T was charging us with a mandatory and unwanted data plan [4].

The plan includes voice mail [6], thought Apple's elegant voice mail won't work without a data plan and accessing voice mail uses up plan minutes. Instead I configured my son's Google Voice number to be the voice mail service [5]. If he misses a call he gets an SMS notification with a message transcription, and an email with a link to the voice file.

At this time, the experiment looks promising.

- fn -

[1] That's what it said on the package. Later, visiting the H2O site, it seemed there were more data plan options than the package suggested -- including a new $100 payment that takes a year to expire -- reminds me of the plan a friend used with his Android phone.

[2] One fringe benefit -- no data plan means less concern about Apple's fake parental controls.

[3] WOW is a different scene from AT&T stores. This is a cash-centric business. Maybe some other sites will cut the micro-SIM for free, or you could try cutting down the card with a razor blade, but Dawayne did it with flair and the card fit my son's iPhone 4 perfectly. Well worth the $5.

[4] It's even cheaper than the $10/month added line phone (plus $4 fees/taxes) we used to have -- and SMS is cheaper. Of course AT&T may terminate this loophole any day now, but they can't force a data rate on us. They can only close off a revenue source; that's hard to do when SMS is going away and desperation is setting in.

[5] Our family has worked from a free Google Apps and family domain for five years. It's trivial now to give each family member a Google Voice account. Within GV there's an option to 'add GV' to any verified phone, makes it the answering service.

[6] The setup directions are poorly written and, I suspect, might be partly in error. I didn't try though.

See also:

Mine

H2O site

Other

at Tuesday, March 06, 2012 0 comments
Labels:

Fixing Siri - When Really Sorry isn't good enough

[Update: before you try resetting network settings, try simply turning Siri off then on again.]

All Siri says to me is "I'm really sorry about this, but I can't take any requests right now." At first I thought she was playing hard to get. Then I figured she'd found another 23 million people.

I'm not the only broken heart, there are 177,000 Google hits on Siri "I'm really sorry about this".

Not so great for Apple's flagship product, so I started in on a pretty good rant. That's how I came across this (marginally good) advice:
Siri says I'm really sorry about this...: Apple Support Communities
... Goto: SETTINGS>GENERAL>RESET>RESET ALL SETTINGS [Don't do this!] Then follow the commands and reset your phone. You'll have to re-enter some info like Wi-Fi settings/passwords.... and then turn Siri and Location services back on.... but thats it and Siri should start to work again...
I gave it a try -- and it worked! Unfortunately, I also lost my custom wallpaper, I had to reconfigure iMessage, I lost my dictionary, and my iPhone restarted as thought it were a new phone (scary!).

The trick is only reset NETWORK settings. You don't need to Reset All. You'll still need to reenable iMessage and you may get the 'new phone' restart. If you do, ignore the 'restore' options, just choose setup as new phone. Everything will be there except iMessage configuration; so review your existing Message setup before you reset.

So why did this bring Siri back to me?

A clue is what you find when you search on "I'm really sorry about this ...". It's what hackers get when they try to enable Siri on an iPhone 4 and they're going through a "bad" proxy server. I was a relatively early 4S user, and I'm guessing when I signed up I was assigned a proxy server that's now overloaded or broken. When I reset, I'm guessing I was assigned a new proxy server.

Presumably in the next iOS update Apple will have some fix for this problem.
at Tuesday, March 06, 2012 1 comments
Labels:

Tuesday, February 28, 2012

Deleting Google: Is Latitude broken?

I'm late to the great Google divorce party -- tomorrow Google promises to exploit and sell everything that they've got on us. (Not sure if that includes Docs and Email -- soon if not now.)

It was easy to delete my extensive search history, but the Google Latitude management page is missing its delete history button. Looks like they forgot that one.

It's sad how fast and far Google's reputation has fallen. I'm at a Silicon Valley (think LA without the beaches or the glamour) tech conference today, and Google gets the disdain Microsoft once owned. I wonder if they've noticed, or if they've figured out that Larry Page is not Steve Jobs.
at Tuesday, February 28, 2012 0 comments
Labels: , , ,

Friday, February 24, 2012

A fix for OS X lousy print sharing - the AirPort Express

I've been struggling with my OS X 10.6 shared Brother HL-2140 ever since I bought it. Whatever I tried, at least once a week I'd get an error message that was supposedly fixed with 10.5.5:
Mac OS X 10.5.5: USB connected printers do not print; printer queue displays "Printer is currently off-line" ... 
... After updating to Mac OS X 10.5.5, you may not be able to print to some USB-based printers. This message may appear in the printer queue: "Printer is currently off-line." 
... Open this link in a web browser:  (http://localhost:631/printers/) Click the "Modify Printer" button for your affected USB printer queue. Click Continue.  You may need to wait a few seconds. From the "Device:" pop-up menu, select "Unknown USB". Click Continue. The "Make:" list should highlight the manufacturer of your printer. Click Continue. Select the printer model that is attached. Click "Modify Printer". Enter an administrator account name and password....
Alas, that 10.5.5 measure didn't work, though it's nice to know that localhost:631 interface to the CUPS printing service. Maybe 10.7 is better, but I'm trying to avoid that mangy beast.

I had past success with the networked Brother MFC-7820N - I only gave up on it when the print mechanism started jamming (old). I figured when my next printer cartridge was due I'd look for something better. Alas, we're WLAN these days, and the reviews for Brother's wireless printers are not good. In fact, there don't seem to be any good reviews for anything. Printing is fading away.

So instead I disconnected the printer from our 10.6 iMac and connected it to an old and underused 802.11G Airport Express. That did the trick. It's been two weeks and the 2140 has been working for all of us. Cheaper (free in our case) and more reliable than a printer WiFi client.
at Friday, February 24, 2012 0 comments
Labels: , ,

Thursday, February 23, 2012

iMovie '11: deleting rejected clips reduces video quality

The great thing about my home video archive project [1] is that there are always more pitfalls. I suppose it's a bit like golf that way.

In the last few days, for example, I discovered that the 10 or so hours of analog video I'd painfully captured had only one audio channel [2]. Prior to that I figured out that the existence of an Event repository on an external firewire drive was preventing thumbnail generation with new event capture.

Yes, it reminds me of the joys DesqView/QEMM DOS memory management. Be glad you don't know what I mean.

Even better, last night I discovered that iMovie '11s "Move Rejected Clips to Trash" feature destructively degrades archival video quality. Without, I might add, any warning.

Like all video pitfalls, this surprised me. I remember QuickTime Pro was very good at excising bits of DV files [3]; select the region, delete, quickly get a smaller file. Alas, iMovie doesn't work this way.

Instead iMovie decompresses (decodes to lossless) the clip, which in the case of my Analog to Digital capture is the entire file. [4] Then it excises the unwanted segments and it re-compresses to back to DV-25, changing the packaging as well (from .DV to .MOV). Since DV-25 is a lossy form of compression [5] there's a significant quality degradation from removing unwanted clips, such as the trailing frames iMovie creates after my camera's DV stream ends.

Oh joy. Now I get to redo that tape [6].

So why does Apple do this [7]? Probably because nobody seems to have noticed. Even when I knew what to search on Google found only one reference from 2007 [8]...

harder & harder to remove unwanted...: Apple Support Communities

... I can still crop bits from my events and send those bits to the trash BUT doing so converts the underlying DV movies into .MOV files and DOES modify the quality of the video. I'm not sure if it's significantly worse quality, but I can do a side by side comparison of the original DV and the MOV file that iMovie 08 insists on making if I delete bits permanently from my events and there IS a difference...

... However, they get "squeezed" for some reason, as you pointed out, so that when played directly (i.e. from Finder in QuickTime) they get reduced from their real/original 853x480 to a smaller, squished 720x480...

Even in that thread the participants missed that DV (MPEG-2) is lossy compression; the reassurances were misplaced.

Someday I'll have to see what Final Cut Pro X does. iMovie's callous recompression doesn't give me warm and fuzzy feelings. (Admittedly, nothing about Apple gives me warm and fuzzy feelings any more.)

[1] For the rest of the current series over the past week or two, see:

[2] I'd swear one of the capture UIs I used appeared to show stereo. I think the problem was that the Canon STV-250N video cable only looks standard; I think it flips an audio and video input. So one audio was going to the video channel, but since I was using an S-video cable I didn't see this. I'd long ago mixed that cable with others, and it took some detective work and web image study to identify it.
[3] DV-25 technically, though I believe that label conceals a wide variety of messiness including variations in codec version, variations in metadata, variations in sound encoding, differences in 'pixel aspects', interlacing variations and so on. 
[4] Maybe it always does all clips in an Event. I don't know. In this case the Event was a single 10GB clip and it was entirely re-encoded. I also don't know if this continues to happen after the initial DV/MOV reencoding, perhaps iMovie can work with some forms of DV more directly.
[5] Quite lossy by still image standards. It feels comparable to 60% compressed JPEG, so a single re-encoding cycle causes noticeable damage. I have a theory that low compression MPEG-4 (30 Mbps+ data rates) is less lossy than DV, but saying this tends to enrage video "experts" in Apple's Discussion groups.
[6] It looks like I'm going to be running through each A to D conversion an average of 2.5 times. If I could travel back in time two weeks I'd send me to a commercial A to D conversion service.
[7] Aperture does something similar, albeit less obnoxious. Even if you don't edit a JPEG image, if you export it as JPEG Aperture always decompresses and recompresses. There's no intelligent export of the original.
[8] Not the point of this post, but this bit of the thread helps explain why video work is so nasty (more on formats some other day) ...

... QuickTime always reports two dimensions, the native format size — 720x480 — and the playback display size, shown in parenthesis, e.g. "(853x480)". The native size is measured in non-square pixels and the display size in square pixels. (Or is it the other way around?)

.. Throughout the history of QuickTime the playback of DV video has evolved from always displaying it as 720x480 — which confused viewers because the pixel shape of DV caused the video to appeared "stretched" — to adjusting the playback display to 640x480 for 4:3 video and 853x480 for 16:9 video. Sometimes QT doesn't choose the display size we expect, but that shouldn't affect the size or quality of the underlying video. (To force the video to use the aspect ratio we want so it looks right, QuickTime Pro lets us adjust the display size, just like we used to adjust all 720x480 DV to play as 640x480.) ..

at Thursday, February 23, 2012 0 comments
Labels: , ,

Tuesday, February 21, 2012

Pinboard and IFTTT to Twitter

A few weeks after Google 2.0 killed Google Reader Social, I decided I'd take the Twitter route.

My Twitter feed was active for about two months, but in time I gave up on Twitter (again). It didn't work as a memory management solution, I don't like the length limit and the lack of structure, and I don't expect Twitter to take better care of my data than Google. Worst of all, like Google, Twitter is free. I can't afford free.

Next I ran through PosterousandTumblr. They both failed, for different reasons. Worst of all, both are too expensive.

Lastly, I turned to Pinboard. I thought I'd use it restore access to my Google Reader JSON exports -- but that didn't work. I could have gotten my $10 back, but I like Pinboard's style, microblog post formats, export formats, import options, ownership, feed-focus, and bookmarklet -- and I really like the Reeder (and, via SendTo, Reader) integration. Most of all, I like paying for it. Oh, and it's Google-Free too [2].

So I've stayed with Pinboard. I tag my (all public) posts with 1 tap codes that are meaningful only to me - s for share, b for blogworthy, y for yammworthy. Each tag has its own stream. The 's' stream is really aimed at Emily, but anyone can consume it. It shows up in her copy of Reeder.app and on her iGoogle page.

No, it's not Google Reader Social. Then again, nothing is. I've come to think of GR Social as a future-fluke, a transient window into a future that might never be. Pinboard isn't GRS, but it's the best microblog/share platform in the mundane world. It beats the heck out of Twitter and (shudder) G+.

Except Twitter, despite my neglect, refuses to die. So I decided to try injecting my Pinboard shares into the Twitter stream. Pinboard doesn't support this, and they don't intend to. However IFTT supports both Pinboard and Twitter, so I now tweet from Pinboard via IFTTT (recipe) [1]

For example:

It mostly seems to work, though I don't know what happens if my Descriptions are too long.

For the moment then, I share the same items in two places:

[1] In my own task I have a colon between Title and Description, otherwise they run together in Twitter.
[2] Is it Google's ambition to be more disliked than AT&T?

at Tuesday, February 21, 2012 0 comments
Labels: , , ,

Monday, February 20, 2012

iTunes Playlist havoc - where did they go?

(The moral of this story - be very careful about following old web recipes for iTunes hackery. The app has changed a great deal in recent releases.)

The original:

It's been a long time since iTunes caused me much trouble, but something's gone wrong recently. Several playlists were missing. I followed the standard procedure for playlist recreation (reference deleted, see [1]). I didn't recover anything, but now most of my many, many Smart Albums are broken.

Be interesting to find out if this is simply my bad luck or something went bad in a recent iTunes update.

Nothing for it but to slowly clean up, wistfully parting with more more hours feeding the beast. I'll try restoring from an a month old playlist XML file via Time Machine and see if I get better results.

I suspect they're not long for the world anyway. Apple is overdue to gut iTunes, and replace it with some simple app that will omit most of what I use.

Update: Oh, great. TIme Machine does not backup iTunes Library.xml. Good thing I have offsite backups using Carbon Copy Cloner. Did I mention Apple gives me hives? In theory we're not supposed to need it, but in this case without that file I have no Playlists at all. Certainly feels like something is broken....

[1] Updateb: It got worse. All my apps were missing too, and all my purchases. At that point I bailed. I pulled my CCC backup to protect it, then told Time Machine to restore iTunes Library, iTunes Library Extras.itdb and iTunes Library Genius.itdb from 1 hour before I started messing around. Things returned to normal; iTunes Library.xml was recreated as it should be. I'm still missing a couple of Playlists, but at least I haven't lost everything.

So I can't say what happened to a few of my ITunes Playlists, but I can say one should be very cautious about trying to recreate a playlist using the method of removing iTunes Library and importing from iTunes Library.xml.

Updatec: Looks like there's more material missing - including several audiobooks! .... Time Machine shows everything was there an hour ago. It's almost as though, when I was trying to restore my playlists, iTunes actually went and removed ... content. I've never seen anything quite like this. I am going to run some disk check software then try restoring my entire iTunes Library from last night's disk clone.

Updated: I think I figured out what happened to the Audiobooks -- and probably other content. It went something like this:

  1. During my ill-fated attempts to restore the wee missing Playlists, I caused iTunes to review all content without any genre settings I'd made.
  2. During the review iTunes decided that some old Audiobooks taken from CD and specified as audiobook, were in fact music. iTunes reorganized them into music (I found them there using Spotlight.)
  3. When I restored the iTunes database it didn't know about the new location, so the content went missing.

I finally restored from a backup of the previous night and brought everything back. I recreated the mysteriously lost playlists. The loss was never explained.

at Monday, February 20, 2012 0 comments
Labels: , ,

Sunday, February 19, 2012

iMovie 11 does not completely import iMovie HD projects

I was infuriated by the data loss during iPhoto to Aperture migration, but that's nothing compared to iMovie HD to iMovie 11 migration [1]. All titles and effects are lost.

This happened years before the better known debacle; Final Cut Pro X does not import Final Cut Pro 7 projects.

How does Apple get away with this?

It's not just a rhetorical question.

[1] Prior versions don't import at all, though you can dig the .DV files out of the Packages and import them by hand. Or open prior projects in iMovie HD, then save them then import.

at Sunday, February 19, 2012 3 comments
Labels: , ,

MPEG Streamclip video converter as Compressor alternative

Squared 5 - MPEG Streamclip video converter for Mac and Windows
... MPEG Streamclip is a powerful free video converter, player, editor for Mac and Windows. It can play many movie files, not only MPEGs; it can convert MPEG files between muxed/demuxed formats for authoring; it can encode movies to many formats, including iPod; it can cut, trim and join movies. MPEG Streamclip can also download videos from YouTube and Google by entering the page URL....

Those are impressive features. The post claimed conversion quality equalled that of Apple's Compressor app.

Reading the release notes it's typical video software - somewhat flaky and tricky to use. Some functionality requires companion apps. It's not fully Lion compatible and OS updates may require app reinstallation. Some details on functionality (emphases mine)...

MPEG Streamclip lets you play and edit QuickTime, DV, AVI, MPEG-4, MPEG-1; MPEG-2 or VOB files or transport streams with MPEG, PCM, or AC3 audio (MPEG-2 playback component required); DivX (with DivX 6) and WMV (with Flip4Mac WMV Player). MPEG Streamclip can export all these formats to QuickTime, DV/DV50, AVI/DivX and MPEG-4 with high quality encoding and even uncompressed or HD video.

Video conversion is performed in the YUV color space; you can choose to scale video to any frame size using a professional 2D-FIR scaler (better than bicubic) but you can also leave it unscaled. Other optional video processing features include a powerful motion-adaptive deinterlacer, a field dominance converter, a chroma reinterlacer and an option to perform interlaced scaling instead of progressive scaling. Audio can be converted to uncompressed or to IMA, AAC, MP2 or AMR using the high-quality MP1/MP2/AC3/PCM built-in decoders of MPEG Streamclip; it is also kept in perfect sync with video using a timekeeping system.

MPEG Streamclip can save edited movies as MOV files, and (when possible) as AVI or MP4 files. Edited MPEG-1 or MPEG-2 files are saved as MPEG or TS files.

MPEG Streamclip (with or without the MPEG-2 Playback Component) can also convert MPEG-2 transport streams into muxed MPEG-2 files, for immediate burning at full quality with Toast 6 or 7 and Sizzle; it can also demux MPEG-1 and MPEG-2 files and transport streams with MPEG, AC3, PCM audio to M2V and AIFF (or M1A or AC3) files, for immediate burning at full quality with DVD Studio Pro or Toast 6 and 7. A special demuxing option is available for Final Cut Pro 4/5: this application does not work well with M2V files, but MPEG Streamclip can write a special "unscaled" M2V file that preserves full video quality when imported into Final Cut Pro.

MPEG Streamclip can handle files and streams larger than 4 GB, split in any number of segments, or with multiple audio tracks, and can also optionally handle timecode breaks. It is compatible with MPEG-1 and MPEG-2 video, MPEG layer 1/2 (MP1/MP2) audio, AC3/A52 audio, and PCM audio.

The player included in MPEG Streamclip lets you preview the files and transport streams before doing the conversion; it also lets you visually set the In and Out points for the conversion so you can convert just the part of the file you are interested in, and also cut commercials and other unwanted parts, or edit the stream and join two streams with Cut/Copy/Paste.

MPEG Streamclip supports batch processing: just drag some files in the batch list, choose a conversion and a folder, click the Go button and MPEG Streamclip will automatically convert all your files.

Reading between the lines I suspect it's sometimes used for non-approved editing of commercial products.

at Sunday, February 19, 2012 0 comments
Labels: ,

Saturday, February 18, 2012

Mac mass storage for video archiving - options

I'm in the midst of a bit of project from heck - assembling and managing our home video recordings. This comes to about 40 hours of analog Hi8 and a comparable amount of 2007 DV technology plus an array of camera files from pretty crummy to modern dSLR HD. I estimate the total archival store will be about 1TB -- mostly using DV-25 files (MPEG-2 codec) and similar acquisition formats. (There are no archival video formats in 2012. More on that in a separate post.)

This is too much data to fit my current backup system - a redundant mixture of Time Capsule and Carbon Copy Cloner with offsite rotation. Once the archive is created I don't expect a rapid expansion rate, so the key backup feature is offsite coverage. I don't think this will be on the same rotation schedule as my current backups.

So I think I'm looking at a completely independent system - at least until we can buy $10TB of storage for under $200 (2016?).

I would like to spend less than $1200. I'd like a RAID-1 solution that would allow me to take a drive off-site, bring the off-site drive onsite, and rebuild the mirror. I'd prefer Thunderbolt though it sounds like Firewire 800 may be good enough for what I want to do. I do want it to be fairly quiet, though in fact it needn't run very often.

This is the list of options I'm coming up with:

I may also opt for basic Firewire 800 drives and do the cloning using CCC (will take a while!)

Any suggestions or comments are welcome ...

at Saturday, February 18, 2012 5 comments
Labels: ,

Video: The state of compression is not good. Blame it on Congress.

The more I study analog to digital video conversion and export compression the more I see I've left civilization.

This is a domain of arcane knowledge and of unknown unknowns -- "experts" who really don't know what they're talking about. I'm sure there are real experts, but they're bored silly by the level of net discussion -- including Apple's own technical notes. It reminds me of my 1981 medical school lectures in renal physiology [1], or my @2002 research into color profiles.

In the absence of publicly accessible expertise I hope I'll have time to do my own experiments, such as creating a range of QuickTime 10.x exports with a span of export parameters. I'm afraid, however, that I don't have that much time.

Which means I probably need to give up on my idea of using .mp4 files the way I use .JPEG -- as an archival storage format [2].

So that means I need to buy more storage -- 2TB should do it. From that point on, given the amount of video we capture, I will just need to upgrade every 2-3 years (4, 8, 16, 32....). And I need to invest in either firewire 800 or thunderbolt.

So now I'm looking at the 1995 era ".dv" format [3]. Should be straightforward. Right?

Yeah, that's what I thought until I read this wikipedia article, which, as is usual for wikipedia, reads like a mixture of a true expert and some confusing amateurs [5] (but is still better than the alternative...):

DV - Wikipedia, the free encyclopedia

... The DV codec was launched in 1995 with joint efforts of leading producers of video camcorders...

... DV video employs interlaced video scanning with the luminance sampling frequency of 13.5 MH...

... One video frame is formed from either 10 or 12 such sequences, depending on scanning rate, which results in a data rate of about 25 Mbit/s for video [4] and an additional 1.5 Mbit/s for audio. This results in a compression rate of 5 to 1...

... All DV variants except for DVCPRO Progressive are recorded to tape within interlaced video stream...

... When video is captured onto a computer it is stored in a container file, which can be either raw DV stream, AVI, WMV or Quicktime. Whichever container is used, the video itself is not re-encoded and represents a complete digital copy of what has been recorded onto tape. If needed, the video can be recorded back to tape to obtain full and lossless copy of the original footage....

So even .DV covers a multitude of sins -- and is a significant compression from the original stream.

Even so, a mixture of .DV and Apple's proprietary professional formats seems the only way to go for now. I hope within a decade we'll have the standards and software to do better.

Or perhaps not. It's worth asking why progress has been so slow over the past decade -- and why the "standards" for still and video images date from the early 1990s. The answer seems to be something Congress and the USPTO did in 1996 ...

USPTO issued Final Computer Related Examination Guidelines stating that "A practical application of a computer-related invention is statutory subject matter. This requirement can be discerned from the variously phrased prohibitions against the patenting of abstract ideas, laws of nature or natural phenomena".

We may need legal reform before we have an archival video format. This is not a technology problem.

[1] After weeks of confident and contradictory statements from researchers and textbooks, I wondered whether they knew they were faking it, or if they'd fooled themselves. I never did find out.
[2] JPEG itself is a non-specific marketing term that hides a very wide range of codec tweaks and metadata "standards". Still, it's a recognized library of congress standard (TIFF is even less standard.). There isn't an image format today that has a comparable archival lifespan, so I use 99% (minimally compressed) JPEG and delete the RAW images -- holding my nose the whole time. Yes, I wish JPEG-2000 had succeeded. Our IP (patent) framework is intensely dysfunctional. 
[4] Now I know what the data rate setting in iMovie 11's QuickTime export parameter means.
[5] In particular there's confusion about DV handling during capture vs. how it's represented during editing/playback and confusion about what containers are. Of course, as a non-expert, I corrected that section.

at Saturday, February 18, 2012 0 comments
Labels: , ,

Wednesday, February 15, 2012

iMovie 11 and analog to digital videotape conversion by passthrough

it's been six years since I've had much good to say about Apple's OS X software (we miss you Avie). So I'm mildly surprised to say something good about iMovie '11.

I haven't paid much attention to iMovie since the iMovie HD debacle, but after Final Cut Pro X and QuickTime Pro failed me I turned to my free copy of iMovie [1]. Short of Adobe Premiere, it was the last tool to use for my family video consolidation project ...

Gordon's Tech: How should I transcode my home analog video?:

... Emily picked this project for her Valentine gift, so we're off. I found a 300GB drive and LaCie firewire enclosure I'd forgotten about. So far video capture seems to be proceeding well from a 12 yo Hi8 analog tape; there's a thin sliver of artifact at base; I think capture frame is bit taller than the image frame. Setup at the moment:

  • SONY Hi-8 TRV65 (1997) with S-video and RCA audio out to ...
  • Canon Optura 50 (2004) with firewire out via secondary port on LaCie to ...
  • iMac i5 2.7GHz, 8GB RAM
  • Final Cut Pro X (trial version. There's no more academic pricing for Apple products distributed through app store. I expect I'll buy full version. So far this is the big expense) saving project to ...
  • External firewire LaCie drive (300GB)

I started with FCPX, but it only seemed to be capturing video. Video streamed to FCPX, but was not recorded. FCPX can't handle non-timestamped video streams:

So, after reading about 10GB import limits for QuickTime Pro, I tried iMovie 8 (iMovie '11). These were the steps I followed:

  1. Connected the Hi-8 camcorder via S-Video and RCA to the Optura
  2. Set Optura to Passthrough mode (most people now would use a Canopus device or the like to do A/D conversion - workflow is very similar)
  3. Turned off Time Machine -- I don't want the 30 GB videos going to my backup drive [2].
  4. Start iMovie. It's safest to create the Event (will hold the video, analogous to Events in iPhoto) first so you can choose what drive the event will go to (there's a small icon that lets you see how Events are distributed over media).
  5. Start Import in Manual mode and direct to the Event desired. Push Play on camera.
  6. iMovie will give you a few seconds to get the video in. It shows the Canon as no tape and displays 'Ready to Record'. If the stream doesn't start iMovie will quit.
  7. If recording is working the counter will increment and a small red dot will show. The 'Ready to Record' message never changes.
  8. When recording is done iMovie stops (1 minute delay)
  9. Then thumbnails are supposed to be created (but see below) ...
  10. After all is done, move the Event to an external drive (Cmd-Drag and drop within iMovie.)

I start the when I get home from work then let it run. A 1.5 hour tape produced a 25GB .DV file and 1.5 GB thumbnails.

To export to .m4v (MP4 variant) you first create a Project. I edit my .DV by selecting the parts I don't want and tapping the 'R' (remove) key. Use Shift-Command-A to undo select all; it's easy to select all and not clear how to undo. Study the keyboard shortcuts, I don't like iMovie's standard or context menus. Editing was easier than I'd expected, even for a project this size. I'm just getting rid of the obvious junk.

When done select all and drag to Project window. Then add a title slide.

After the Project is done choose Share and large size. On my iMac compression to 960x540 took about 1 hour and produced a 1.5 G .m4v (mpeg) file; it uses "270% of CPU". I'd prefer a bit less compression, maybe a 3GB output. I'm studying my options. It's ok to save the .m4v to an external drive.

I then move use iMovie to move the entire Event to an external drive for storage (see update, these directions are wrong). Empty trash and restart Time Machine.

The Events, for now, are staying on my external store (1TB+, so should handle about 8-9 of my media. I don't want to keep the .DV files, when I'm satisfied I can produce a high quality .mpeg movie with reasonable compression I'll store that alone. Three 1.0 TB drives should manage all of our video - one onsite, one offsite backup, one rotation [3].

Of course there are bugs ....

  • iMovie crashed twice (kernel exception -- it really doesn't like the cheap external drive I'm using until the better ones come along [3]
  • iMovie always hangs at the end of thumbnail generation, no matter how short the event. This is a replicable bug. I've tried deleting preferences; this may also be a sign these Project are too big for iMovie. I'm hunting around for an old copy of Final Cute Pro or Final Cut Express. (By Darwin, Apple screwed the video market.) [UPDATE: With my firewire drive unmounted, thumbnail generation does not hang.]

It's early days, but I'm less displeased than I'd expected. I think if iMovie fails I'll try Adobe Premiere Elements next.

[1] When Apple introduced the App Store, they also turned to iOS style licensing. iLife '11 came with my MacBook Air, and was registered with my Apple Store ID. That meant it showed up in "purchases" on my iMac and I could download and install it.

[2] Trick is to remember to turn it on again after the iMovie Event has been moved to an external drive. Streaming capture to my old external firewire enclosure did not work. Not only were frames dropped, but iMovie crashed as well.

[3] Future post pending on the external storage/backup system I'm going to use.

Update: Possible export options (ref)

  • MPEG-4
  • frame size: largest possible (when I set to be the source size however, the output still omits about 3% of the frame)
  • key frames: <10
  • data rate: DV data rate is 25Mbit/s for video and 1.5 Mbit/s for audio. The default on .mp4 is about 4Mbit/s.

There's surprisingly little quality information on these settings. Reading the Final Cut Pro 7 documentation makes me envious.

Update 2/17/2012: Apples directions for moving Event's are incorrect. If you drag and drop the Event is copied. To move you have to Cmd-Drag, just as in the file system.

Update 2/18/2012: I realized that, by accident, I was using a low end firewire 400 cable with a 400/800 converter to my external drive. That is a bad idea! I would expect it to cause issues. I switched to my LaCie 400/800 cable. I've also given up on the idea of using .MP4 as my archival format.

I've also experimented with capturing via QuickTime 10 at 'highest qualilty'. That created a .mov file that is almost exactly the same size as the .dv file that iMovie crated from the same source; at heart it's a DV file. There's a trick to get this to work with passthrough video. I had to start the stream, then click the record button. When I move these into an iMovie Event library iMovie simply moves the .mov file then creates thumbnails.

I've read FCPX will import these. I'm experimenting more with this.

Update 2/18/2012b: OK, the QuickTime method doesn't work with this particular video stream. I didn't run into any time limit, but there's no sound on playback -- even though there's sound during the capture process.

Update 2/19/2012: I was able to complete a video capture without the thumbnail hang. Two things were different. My external drive (firewire 400) was not mounted, and I stopped the stream manually rather than let it self-terminate when the video stream stopped. I have another post pending on why I gave up on the idea of .mp4 export. For now I'm collecting 30GB .DV files for each tape. Update: It's not the manual termination. it's that my firewire drive was accidentally unmounted. I've replicated this.

Update 2/20/2012: I tested importing one of the analog to digital projects I created in iMovie 11 into FCPX. FCPX seemed to import the modest project and very large clip without obvious problems.

at Wednesday, February 15, 2012 6 comments
Labels: , , ,

Monday, February 13, 2012

How should I transcode my home analog video?

Twelve years ago I was keen to digitize my home video tapes - on a 400 MHz Celeron system.

Eight years ago I wrote "The thought of losing the kids' taped videos is not comforting. I need to do this sooner rather than later.". At that time I experiment with transcoding through a Canon digital camera using iMovie and a G3 iMac. I figured my G5 would do the job in 2005, maybe with its capacious 40GB drive and a stack of DVDs.

Now I have a slightly aged 2.7GHz i5, 8GB RAM, and several TB drives -- and my analog tapes are 8 years older. Maybe I'll do it this year, even though I hate digital video [1].

I'm trying to warm to the idea. The big difference over 8 years ago isn't processing power [2], it's cost/MB and the gradual emergence of H.264/MPEG-4 Part 10 (AVC) as a relatively standard format for video. The National Archives recognizes there's no real AV standard, but their list of formats they work with includes MPEG4.

If I do finally walk though this, I don't want to bother with DVDs. My current plan is:

  • Purchase two 1TB drives. Digitize everything we have to one of the drives as MPEG4 video and MPEG4 audio.
  • Clone that drive and keep one drive on site and one drive offsite, repeat every few years.[3]
  • Use my old Canon digital camera with passthrough to digitize my analog tapes, then compress on my iMac and store with a file naming convention to reflect date and time information. (I'll consider the Canopus though.)
  • Convert my collection of digital tapes (DV) to H.264 as well

I don't know what software I'll need. Can I do what I need to do with iMovie/QuickTime Pro, or should I buy Final Cut Pro X? I'm not planning to edit, but if I were I'm pretty sure the current fork of iMovie won't do the job.

Update: Apple offers a free download for FCP/X. So I will probably try it out.

See also:

[1] In Jan of 2010 I discovered a 12 yo WMV formatted video was unplayable; I was able to use Windows Movie Maker to convert the 23MB WMV file to a 311MB DV AVI file. Digital image formats are a friggin' mess, but video is at least ten times worse. No metadata standards, crazy patents, non-standard containers, audio codecs, video codecs - yech.

[2] Yeah, the machines are a hundred times faster. But compression can run overnight.

[3] By 2020 I assume we'll have 100TB main system and backup drives, so storing my old video with my routine data will be trivial. By then a single snapshot will be a GB.

Update 12/14/2012: Emily picked this project for her Valentine gift, so we're off. I found a 300GB drive and LaCie firewire enclosure I'd forgotten about. So far video capture seems to be proceeding well from a 12 yo Hi8 analog tape; there's a thin sliver of artifact at base; I think capture frame is bit taller than the image frame. Setup at the moment:

  • SONY Hi-8 TRV65 with S-video and RCA audio out to ...
  • Canon Optura 50 with firewire out via secondary port on LaCie to ...
  • iMac i5 2.7GHz, 8GB RAM
  • Final Cut Pro X (trial version. There's no more academic pricing for Apple products distributed through app store. I expect I'll buy full version. So far this is the big expense) saving project to ...
  • External firewire LaCie drive (300GB)

Both cameras are on external power. I needed my old manuals, had the SONY and found the Optura online. Getting passthrough working was balky, FCPX only gave me about 10 seconds to start then it turned off import due to lack of a signal. It can't operate the Optura 50 in pass-through mode, I haven't tried yet with a tape installed. Once I hit the play button fast enough I was ok.

I'll have data on file size tomorrow. FCPX imports using its default Apple ProRes format.

Operating the old SONY feels primeval. It's 15 years old, feels 50. It is weird to see the kid video. Plan is to put a library of MP4 versions (no editing really) on iTunes on a an old G5 with a largely unused drive. We can browse, play from iTunes.

Update 2/15/2012: Not so good. Nothing captured! Looks like I need to read the manual, or perhaps FCPX won't work at all. It certainly showed the video, but it didn't save the event. It also didn't stop recording when the input ended.

Update 2/15/2012b: QuickTime Pro is not an option. I was able to create a @2 hour 25GB .dv file using iMovie 11, but then iMovie 11 got stuck 'generating thumbnails'. I think that bug, however, was triggered by connecting an iPhone to my iMovie machine. Disconnecting the iPhone did not clear iMovie, I had to kill it. On restart my video appears to have been captured and iMovie seems to have stopped input about 2 minutes after the end of tape. Thumbnails are fine - all 1.5GB of them. More on the results and process in a follow-up post. I'd like to take a look at Adobe Premiere Elements.

Update 2/16/2012: Gordon's Tech: iMovie 11 and analog to digital videotape conversion by passthrough.

Update 2/17/2012: I experimented with QT 10 movie capture. The trick with passthrough conversion is to first start the video stream, the click record. Otherwise QT exits. I captured 30 min of video at 9 GB; QT Inspector says the codec inside the saved .mov file is dv.

I've read that this will import into FCPX, so I may try that again. My next experiment will be to see if I can record over 10 GB (1 hour of video). There are many reports of QT recording stopping with a 'size limit' message at varying file sizes, in my case that's what displayed after I stopped the video stream.

at Monday, February 13, 2012 11 comments
Labels: ,

Sunday, February 12, 2012

Warning - if you delete your G+ Profile you may turn off sharing from your Picasa albums

A friend of mine complained he was unable to download some wedding pictures I'd taken. Last summer downloads worked, but as of a few months ago they stopped working.

Just around the time I deleted my Google G+ Profile.

My friend told me the original sharing link worked, but the download button was grayed out. Sure enough, when logged out of Google I couldn't download images. It was easy to see why, image downloading was turned off in my Picasa preferences.

I don't remember turning that off. Given the way I share images, that wouldn't make sense. My best guess is that when I deleted my G+ Profile Google reset my sharing preferences to the default settings. if I'd known I could have fixed it immediately. Instead I've been sending share links out for months that weren't working.

I assume most of the people I sent shares too simply gave up in frustration, or ended up downloading a low res thumbnail.

Google Fail.

at Sunday, February 12, 2012 0 comments
Labels: , ,

Saturday, February 11, 2012

Time Machine for Lion supports encrypted backups

In the midst of tweaking a Time Capsule backup, I noticed a grayed out option to encrypt my backup.

Turns out 10.7 enables Time Machine encryption on Lion - but only for directly connected drives.

Reading the MacOS X hints commentary I think they're missing the point. A good backup strategy requires offsite backup, and taking a non-encrypted backup offsite is somewhat risky. I don't take my Time Capsule backups offsite, but I do rotate my Carbon Copy Cloner images/backups -- and they are encrypted.

This is a good feature if used wisely.

at Saturday, February 11, 2012 4 comments
Labels: , ,
Subscribe to: Posts (Atom)