Manipulating JSON data in a traditional relational database (Microsoft Access, FileMaker Pro, Converters)

While I wait to see if Pinboard can fix their Google Reader JSON (JavaScript Object Notation) import, and while I consider Google Reader Share JSON import into WordPress, I'm also exploring JSON import/export tools. If, for example, I could import JSON into FileMaker Pro or other data management tool I might be able to manipulate the archive and produce a more useful WordPress import.

StackOverflow and its kin have a good set of references on this topic. Note that CSV can manage only very simple JSON; we really want native importers similar to what Microsoft Access tried with XML [1]. I suspect one approach might be to convert JSON to XML then use Microsoft Access 2010 import.

Incidentally, this topic veered off unexpectedly into something that's actually relevant to my work life and a Strata conference I'm attending in a few weeks.

As of today here are some of my pointers ...

• What is “JSON” and what does it have to do with distributed computing? « DivConq: great reference (see below for DivConq series on Access)
• JSON - Wikipedia, the free encyclopedia: I liked the comparison to XML [1]
• JSON.org: points to libraries
• 18.2. json — JSON encoder and decoder — Python v2.7.2 documentation: If I have to use a programming library to manipulate Google Reader json files my choices are probably Python or Javascript and I prefer Python.
• conversion - Convert JSON format to CSV format (for MS Excel) - Stack Overflow: JavaScript
• convert from json to csv using python - Stack Overflow
• Download JSON data and convert it to CSV using Python - Stack Overflow
• python - Convert JSON to CSV - Stack Overflow
• javascript - Python - convert csv file to JSON - Stack Overflow

For me this DivConq series was particularly useful because it placed JSON nosql processing in a familiar context - Microsoft Access.

• What is “JSON” and what does it have to do with distributed computing? « DivConq: best single reference for my purposes
• Export a Microsoft Access Database to JSON (Northwind Example) « DivConq
• Why Cassandra « DivConq
• Installing and Running the Cassandra Database « DivConq
• Migrate a Relational Database Structure into a NoSQL Cassandra Structure (Part I) « DivConq
• Migrate a Relational Database into Cassandra (Part II – Northwind Planning) « DivConq
• Migrate a Relational Database into Cassandra (Part III – Northwind Conversion) « DivConq
• Migrate a Relational Database into Cassandra (Part IV – Northwind Import) « DivConq

Maybe I should start using Apache Cassandra to manipulate my Google Reader JSON archive and prepare it for WordPress processing. For example ... Cassandra Development Environment in Mac OS Snow Leopard « BigDiver.

[1] I doubt JSON has truly significant advantages over XML as a data interchange format (see JSON Example and wikipedia xml/json). Alas, nobody asked me. Fashion is more powerful than geeks imagine.

Pinboard imports Google Reader JSON exports

Pinboard is the first service I know of that will import a Google Reader Social (shared item) JSON file:

Pinboard: howto page

Google Reader Click the gear icon in the upper right corner of the page. Select the Import/Export tab. Choose either items you have starred or items you have shared and click the Reader JSON link (the rightmost column)

When I stumble unexpectedly over something I've been looking for, I look for who else found it. Then I add them to my reading list. Google gave me only these references:

• Google Reader Shared Items -> Pinboard, FeedDemon, Send to Pinboard… « cmiles – blog 11/5/2011
• Pinboard Blog - Nov 2011 - "... Added importers from Firefox JSON format and Google Reader. Wrote scripts for tarring up user archives in human-readable form so people can download them. Removed Google Reader support (RIP, Google Reader, I'll mourn you til I join you)...

Pinboard has a feed, I don't know if importing will trigger feed actions (probably not)

See also:

• Gordon's Tech: Recovering Shared Reader items: JSON import into Wordpress
• Gordon's Tech: Sharing and annotation: Instapaper's supporting apps

Update: I paid my $10 and imported by Google Reader shared item JSON file. I have 3 days to cancel. I used Amazon payments.

Here are the results; as of today the most recent post is 7 weeks old. I may also try importing the JSON for my Reader shared items, which may produce some duplicates.

• http://pinboard.in/u:jgordon - my pinboard collection - really my Google Reader shared items. Note my user name is a part of the URL, so it's nice that 'jgordon' was available. Posts show a title, a bookmark, and an excerpt. I think my GR annotations precede the excerpt. It's more like Google Reader Social than I'd expected.
• http://feeds.pinboard.in/rss/u:jgordon/ - the public feed for my collection. I viewed this in Google Reader; gave me a real sense of deja vu. Alas, GR only pulled in 44 items.

I'm still studying the results. So far Pinboard is only showing a fraction of the JSON file, there are not tags, and every item shows with date of '9 weeks ago'. I don't see a convenient way to navigate across the entire collection.

Update 12/31/11: Pinboard has now imported 2 months of Reader shares - about 1100 items or roughly 1% of the total.

Sharing and annotation: Instapaper's supporting apps

I haven't found a replacement for the rough annotation-share-feed ecosystem that had grown up around Google Reader Social (RIP). I've given up, for example, on using Twitter as a Reader Social replacement.

Yes, I miss Google 1.0. I even miss Microsoft these days.

So I'm continuing to explore the pieces of the post-Google world; trying to see where this micro-market may go. This is poorly tracked territory, but today I came across an unexpected guide in the Instapaper: Supporting iPhone and iPad Apps page.

Instapaper has an ecosystem, and although it doesn't have a feed, it will post to Tumblr, Twitter and Pinboard. Tumblr has a feed (barely), Twitter can be turned into a feed (awkwardly) and Pinboard has a feed (and, mercifully, it's not free).

So what can I do with these pieces? Can I archive the output of Pinboard as WordPress posts?

I'll find out.

See also:

• Gordon's Tech: Pinboard imports Google Reader JSON exports

Update

• I tried Instapaper's bookmarklet, but it hangs in Chrome with a "saving" status in the tab.

Recovering Shared Reader items: JSON import into Wordpress

Google amputated a portion of my distributed memory, but they left me a frozen json remnant.

(Yes, we are living in a cyberpunk novel. Sigh.)

Across the net there are unanswered questions about what to do with these json archives. Google has been silent. I believe the Google humans who might help are ashamed or demoralized or fearful. Google 2.0 is not a happy place for them.

I want to represent my JSON archives as posts in a WordPress blog, perhaps with some kind of synthetic title. Then they will be available to search and link. Eventually I hope to add new annotations and shares to that archive, though there will be a gap of several months that will be difficult to fill.

This feels doable, but so far Google (the search engine) hasn't told me how. This is what I have found so far. When I do find an answer, I'm going to answer some of the dangling questions across the net ...

• Dariel: wrote a Python script to import shares into ReadItLater
• What are the good ways to import Google Reader Notes (.json) to Evernote? - Quora - an unanswered question
• battis.net » Transmogrifying those Google Reader JSON dumps into something useful - Seth Battis wrote a "PHP script that converts (at least my) Google Reader JSON dump into an XML file that WordPress can import as a list of posts. With the tags and annotations converted over. In fact, with all of the data in the JSON dump embedded in the XML file (although WordPress doesn’t read all of it)."
• Importing .json archives (Wordpress) - Stack Overflow - an unanswered question
• osx - Json viewer for mac - Super User: a way to at least view JSON documents.
• How can I Import Google Reader JSON archives into WordPress - Super User: I ask on SuperUser
• A public question I asked on Google+: I ask on G+

I'll update this post as I learn more. Seth's contribution suggests a fix is close; he needs to tweak some of his code.

Update 12/25/2011: Seth writes that he won't have time to work on this further but he recommends downloading his php file from his linked zip. I'll have to learn how to run PHP scripts from my Dreamhost account, but I don't think that's too hard.

Update 5/3/2012: Coping With Google Reader Changes | Much Ado About IT - accessing the lost items.

The cost of repairing a Mac is less than expected

When my iMac 11,1's 2yo 1TB Seagate drive developed metastatic blockitis I was most unhappy.

It wasn't that the drive is dying young, just two weeks after my AMEX extended warranty lapsed. Two years is short for a drive, but this machine runs all the time and goes through two full disk backups every single night. The drive has had a hard life.

The fact Apple's diagnostics missed my drive's unmasked bad blocks is annoying. There's no magic to a disk scan; I shouldn't have had to buy TechTools Pro to make a diagnosis. Windows diagnostics have managed this for twenty years.

Worse though, is the cost of the repair. FirstTech, a well regarded local shop, gave me a $625 quote to install a 2TB replacement. (They can't get 1TB drives.). I was amazed, I'm used to paying $150 or so for a drive and doing my own installation. That's what I did when my old fully serviceable G5 drive died.

The problem is that the lovely 27" iMac is not user serviceable. Elegant quite design with special thermal sensor cables turns out to have a high post-purchase price. That's why I wrote ...

Gordon's Tech: Mac drive diagnostics: TechTools Pro and Drive Genius find problems OS X missed

... When you consider that iMac 27" hard drives are NOT user serviceable, the iMac is more expensive than it seems. The iMac G5 was entirely user serviceable. Design has its price....

I was wrong though. Today i checked what the cost would be for an Apple store repair. They quoted me about $200 for a 1TB drive replacement. (They don't do upgrades, only like-for-like replacements.)

How do they do that? Apple has a flat $40 service fee, regardless of the complexity of the repair. Apple offsets the ownership cost of their elegant designs by subsidizing repair. (In this case they have another advantage -- they have an inventory of 1TB drives with bundled thermal cables even as the world runs short of hard drives.)

I still prefer my G5 iMac's design -- but that was a hot and noisy machine. My 11,1 (i5, 27" 2009) iMac is quiet and cool most of the time. Apple's subsidy of post-warrany repairs makes that tradeoff more palatable - at least if you live near an Apple store!

Update 1/6/12: One warning: they will want to keep the hard drive. This fits with their out-of-warrantee repair following their in warrantee process. Apple should make this better known. It means if you bring your machine in for an Apple Store repair, you need to do a secure wipe first. Some additional tips:

• Create an admin account with no password that Apple can use for testing. I didn't think of this, and my machine has guest account disabled.
• They will want to recreate the problem -- even though I have to pay for the repair. Again, their out-of-warranty repair is basically in-warranty with a parts-charge and subsidized labor.

Update 9/2/2012: In comments Alex points to a 1TB Seagate drive replacement program! Alas, my serial number wasn't accepted. I bet this is what hit me though. I wonder if Apple will eventually extend the program. I'm W8946HAH5PJ.

Update 8/3/2013: Sometime in the past year Apple sent me a check for the costs of this drive. They did eventually extend the replacement program.

Mac drive diagnostics: TechTools Pro and Drive Genius find problems OS X missed

I've been having suspicious application crashes lately. In the Mac world, that suggests a hardware problem. (Once upon a time software was the cause, but these days it's hardware.)

I knew from some backup issues that I had 3 unreadable files. That suggests my 24+ month old 1TB iMac drive is dying youngish. It was time for some diagnostics, so I plugged in my old Apple Keyboard and mouse. (Most non-trivial diagnostic work requires a wired keyboard and mouse; Apple's bluetooth keyboard/mouse drivers may be unavailable when needed.)

After I deleted the bad (non-critical happily) files I ran Disk Utility - but the drive passed. Then I ran my Apple Hardware Test - extended, and loop mode. Still no problems.

I didn't believe it. Something had to be wrong. So I checked out Disk Warrior, Disk Genius and TechTool Pro - 3 reputable diagnostic apps. They're all $100. Disk Warrior has a good reputation, but Disk Genius has a trial version. It found about 58 bad blocks -- out of 1.8 billion. That seems a modest number, but DG said I needed to replace the drive. (Incidentally, Disk Genius has a built in uninstall feature -- very nice. Yes, the Mac needs an OS level uninstaller.)

I decided to get a second opinion. Andy M clued me to a MacUpdate bundle, so I got TechTool Pro 6 for $50 (plust a bunch of other apps I don't care about). It one-upped Disk Genius; as it found bad blocks it told me which of them had files (none in this case).

TTP also found bad blocks - 56 (so two less than Disk Genius, but I don't make much of that either way).

I wasn't sure what to make of this. After all, 56 out of 1.8 billion is minuscule. Unfortunately, a modern SATA drive shouldn't have any bad blocks. The excellent TTP manual explains why ...

... TechTool Pro should not normally report bad blocks for these types of drives. The drive controller in them automatically tries to map out bad blocks as they are encountered. It will do this unless either the bad block is in a critical area that cannot be mapped out at the moment or the bad block table is full. If this occurs, TechTool Pro will report a bad block and you will ultimately need to do a low level reinitialization of the drive. When the drive is reinitialized, the entire platter is accessible so that bad blocks can be mapped out if possible no matter where they occur...

.. You can use Apple's Disk Utility to reinitialize your drive. Be sure to choose the Security Option to "zero out data." Choosing this option will map out bad blocks, if possible, during the reinitialization. This may take several hours (depending on the size of your drive). If the reinitialization is successful, the drive should be fine at that point. We suggest, however, that you do a Surface Scan a few times in the next month or two just to be sure no new bad blocks are developing. If they are, then the drive is probably failing and you should consider replacing it. If a low level reinitialization fails, this indicates the drive is faulty and needs to be replaced...

So the bad blocks I see now are probably a small fraction of the number that have already been mapped out. I'm seeing the overflow, including blocks that went bad after they'd been written to.

My i5 iMac is 24 months and 2 weeks old - so it's past even my AMEX extended warranty (by two weeks!). If the drive were user serviceable (like my old G5 iMac!) I'd simply replace it. Since it's not a user serviceable I'll probably bring a new drive and the machine to FirstTech in Minneapolis for a $200 24 hour turnaround replacement. I'll make a bootable clone before I do that. (My usual Carbon Copy Cloner backups are to an encrypted image for offsite transfer, so not bootable.)

If it doesn't I'll try the reinitialization. (See Update - this drive is on death row.)

Update 12/21/2011: Various notes and reflections the day after ...

• 16 months is a short lifespan for a hard drive. I bought this machine early in its lifecycle, I wonder if there will be more failures in this product line.
• Modern drives don't write to bad blocks. Based on the dates of the files that were involved the involved blocks went bad in the past month. That fits with Carbon Copy Cloner not complaining until recently. (See my backup issue post for a twist to this story.)
• I'm glad I bought TechTools Pro - I think I'll get good use of it. From what I know now though, I didn't really need it. In a modern drive a single bad block in a file, especially a relatively recently written file, means replacement. Carbon Copy Cloner told me 3 files were bad.
• The TTP manual suggests reformatting. I suspect that might work if there was an initial formatting problem, but in this case I know existing blocks are going bad. This drive is on death row.
• Carbon Copy Cloner complained about bad sectors in files during backup, but Time Machine didn't. That may be because Time Machine only reads files that have changed?
• Most of the bad sectors are in unused parts of the drive. I suspect they were randomly distributed but were hidden by the drive OS as they were discovered in the parts of the drive that have been used (about 500GB of 1TB).
• Drive Utility and Hardware Test didn't find any problems, but both Drive Genius 3 and Tech Tools Pro failed the drive and Carbon Copy Cloner complained too. The SMART diagnostics still pass the drive - even today! I'm a bit surprised; this isn't rocket science. Apple could do better. TechTools Pro gives more SMART diagnostics than Disk Utility -- my drive was complaining about heat (it's cold in this room!)
• It's clearly worth running TechTools Pro or equivalent drive scan on a new drive then every few months. (11/23/11: TTP just crashed during a routine drive scan. I'm not impressed.)
• I think Windows scandisk/chkdsk are superior diagnostic tools to Disk Utility.
• TechTools Pro DVD includes an image for burning a PPC DVD. Nice touch. I still have an old PPC.
• It's good to know the drive is dying before it dies. I have time to do extra backups and to move selected files to other machines -- including my Aperture and iPhoto Libraries and perhaps iTunes.
• When you consider that iMac 27" hard drives are NOT user serviceable, the iMac is more expensive than it seems. The iMac G5 was entirely user serviceable. Design has its price.
• Since I know the drive is dying I've disconnected my clone backup. It's my known good repository. I'll take it to my office then create a new clone, then disconnect that clone. I won't be saving data to this machine, I'll be treating it as a "guest" machine until I get it serviced. I may turn off Time Machine too.
• OWC (Other World Computing - great Mac shop) showed me how to find my Model Identifier (System Profiler), it's iMac11,1. I can only go to 2TB of storage. I'm not confident that their options are correct however.

See also:

• How do you upgrade the hard drive in the "Late 2009," "Mid-2010," "Mid-2011" and "Late 2011" (21.5-Inch and 27-Inch) Aluminum iMac models? - I'm glad I didn't consider doing this myself. I wonder if this is almost an Apple-only replacement.
• Proprietary Cable can put the brakes on upgrading Late '09 iMacs. | Other World Computing Blog: I have an ST31000528ASQ - so Seagate. OWC says these drives will fit my computer's thermal cable: 
  • 1.0TB Seagate Barracuda 7200.12
  • 1.5TB Seagate Barracuda 7200.11
  • 2.0TB Seagate Barracuda LP

Backups - why you need two methods and abundant paranoia

I can't say I feel good about my backups. I believe data wants to die; it wants to be free of the burden of order. Against the despair of data, even the best backup is barely adequate.

Consider tonight, when everything almost failed - Time Capsule and Carbon Copy Cloner alike.

The Time Capsule serves all the machines in our home over a wireless network. I was surprised at first that backup would work over wireless, but it does. Each machine has its own unencrypted disk image; one on the TC's old internal 500 GB drive, two others have images on an external 2TB drive. The TC sits in a closet upstairs;  it's unlikely to be stolen but fire would destroy it. I have done 1-2 file Time Machine restores from that image, so I know it can work. The only test of a backup, of course, is a restore.

I don't trust Time Machine as much as old-time DantzRetrospect, but it seems Apple has gotten most of the bugs out.

I trust Carbon Copy Cloner [3] more. Each day it clones my server, on which all the important data lives. It's more than a cloner; CCC keeps copies of changed or deleted files in "_CCC Archives". I've configured CCC to use an encrypted image it automatically mounts every night. Since that backup is encrypted I can take it offsite, which I do every few weeks. Ok, every month or two. Offsite rotation relies on me, so it's prone to failure. Still, even if the house burns, I am unlikely to lose more than a month of images and videos. I can live with that.

So I have two backup methods, both fully automated, both relatively independent [2]. If each is 95% reliable each day, then the chance both fail on a given day is 1/400. If the daily chance of a server drive failure is 1/1000, the odds of all three failing on the same day are about 1/400,000 [2], [4]

Tonight though, my data got within a few miles of the cliff it wants to meet.

My server has been having worrisome memory exception (EXC_BAD_ACCESS) crashes, and a TV show I  recently downloaded had a file error [1]. There's something wrong on my 2yo i5 iMac; I need to run Apple Hardware Test (again). So I know my server data is at risk.

Time Capsule has had problems too -- it's reporting a "communications error" periodically. I think that error message is  a scarlet herring related to the iMac issues, but clearly I can't trust that backup.

Happily there's good old CCC -- but when I restarted my server for the first time in weeks it reported a problem. The backup drive didn't mount. That was easy to diagnose -- I'd unplugged it. Probably when I was debugging my Aperture crash 3 weeks ago. Why didn't CCC report the error? Maybe it had crashed.

I wasn't that close to data loss -- but I was in a bad neighborhood. As paranoid as I am, I'm almost not paranoid enough.

It's good to have two fully automatic and completely independent backup methods. Data wants to die, and backup is still an unsolved problem.

-fn-

[1] Incidentally, you can't easily report a purchase problem to Apple until they process a charge, and to reduce transaction costs they wait a few days before they process. This is very annoying! Also, the UI for reporting a purchase problem is suspiciously clumsy. More on that experience when I see what they do.
[2] In reality they common failure points of course - me, computer memory, etc. There is the older offsite backup though, so complete and total data loss is probably less than 1/1,000,000.
[3] Donationware. I donated. I wish donation ware apps would let us set a 'reminder' so I could donate yearly. I suppose I should just make donationware donations every year on my birthday against the apps I use.
[4] I'd love to have automated offsite backup too, but I've never foundan offsite vendor I trusted and I expect ISPs to eventually charge for bandwidth use.

See also:

• CrashPlan (or JungleDisk) instead of Retrospect 8? (1/2010)
• Online backup – the security problem (it’s not the encryption)
• backup (50 or so posts on backup)
• Retrospect 8.2 for OS X fails my latest review -- because it's been abandoned (11/2010 - I checked under the latest owner but it felt the same)
• Freeing up Time Capsule space – and documentation for Time Machine and Time Capsule (12/2009 - TM/TC are better quality now, but the documentation still sucks).
• Time Machine, Time Capsule and offsite backup (2/2010)
• My 10.6 iMac is crashing - a debugging exercise (2/2010 - using hardware test)

Update 12/21/2011: I was closer to the cliff than I realized.

Twitter to WordPress via ifttt - limitations

Weeks after Google's Day of Infamy i'm still failing to fully replace Google Reader Shares. Recently I gave up on Tumblr, Posterous, the zombie version of Google Reader, and some screen-scraping attempts to turn G+ streams into feeds.

Lately I've been focusing on my @jgordonshare tweets and tonight I tried using ifttt to create a WordPress feed-equipped archive of tweets.

It was easy to setup the ifttt task to turn the tweets into WP posts. I used a "1 button install" Dreamhost [1] WordPress instance I've been testing. I had to turn on XML-RPC publishing (used by Windows Live Writer, MarsEdit, etc) and provide a WordPress username and password [2].

The ifttt doesn't trigger immediately after tweet creation. I assume it checks the Twitter stream every 15-30 minutes. I manually triggered a check from the ifttt dashboard.

Here's an example of what I got

Just testing iftt tweeting to wp (sorry). http://t.co/GbR8Qdud

... Just testing iftt tweeting to wp (sorry). http://t.co/GbR8Qdud...

Yeah,  not to impressive. The problem is a tweet is simply a string, it has no special structure, no way to distinguish URL from my commentary from page title from annotation (not that there's room for all that). Tweets are much simpler entities than old-style Google Reader shares.

The experiment did work, but the result isn't terribly interesting.

So the quest goes on ...

[1] Use the code "KATEVA" or this link and you are supposed to get 50% off your 1st year costs and I get an equal saving as credit.
[2] Obviously you should create a user for this purpose and create a unique password. IFTTT has to know your credentials.

See also:

• Dreamhost - apparently the kickbacks work 6/2010
• Google reader shared items to Facebook 3/2009 - Google's glory days - before the Fall
• Loving Google Reader - Shared post feed 9/2008 - Google's apex
• My reader shares are back for now - thanks to Keakon 11/2011 - sweet hack, but a bit of a security risk
• Social wrestling: Google Reader, Twitterfeed, Blogger and Facebook 4/2010 - twitterfeed is still around
• Filtering my Google Reader Share with Yahoo Pipes 9/2008 - never quite worked, but an interesting example
• Using Bloggers undocumented label (category) feeds and Yahoo Pipes to create a tech opinion feed out of Gordon's Notes - 2/09 - fiddling
• Tweeting Google Reader Shares and Notes via feedburner 10/2010 - we assume Google will axe feedburner in 2012
• FreeMyFeed - Getting Twitter feed to Google Reader - turning twitter into rss 4/2009
• Microblogging and Google Reader: Tumblr Fails 11/2011
• After the fall of Google Reader: Posterous, Tumblr and Zootool with Twitter on the side 11/2011. I haven't tried Zootool yet.

How to learn what your current AT&T mobile contracted services are

AT&T has the lowest customer service rating of American mobile phone companies. Of course that's like asking what's worse - Ebola or Rabies?

This is not because of their retail staff. They must give them powerful drugs, because, despite working for a moderately evil corporation, they're remarkably cheerful.

Their web site though, that's part of what makes them "Rabies" rather than just "Ebola".

For example, for the past few days I've been trying to follow up on some extensive bill slashing changes. In particular I've been trying to find a current contract summary for our family plan. I think I've found the best that ATT offers, but they have one of the worst web sites I've ever come across. [1]

As of Nov 2011 try this. Don't click on the tabs, but mouse over to see the substructures. Note you may have to authenticate repeatedly.

• Go to your AT&T mobile account page. Look at the top menu structure. It will say myAT&T, with "tabs" like "Overview", "Bill & Payments" and so on. Depending on the services you use some are not useful, but they will still appear. The tabs that are useful for a mobile-only customer are
  
  • Bill & Payments: see current bill statement
  • Wireless: usage and recent activity
  • Profile: user information
• From user information look for "Contract Information". Click Customer Service Summary and Contract. Now you get a popup window. In there you find several options including two that, despite their names, both show a similar PDF (these links may actually work as shortcuts once you're authenticated):
  
  • Wireless customer agreement: This is the real deal. CSS plus six pages that summarize your true contract [2]
  • Customer service summary (wireless): Just the CSS
  • Alternatively, AT&T's "Your Phone CSS" email provides this link with goes to a screen I can't find when I navigate the site, in fact it seems to be a outside of the tabs they define and possibly a separate web site: https://www.wireless.att.com/olam/loginAction.olamexecute?target=CSS. There's a trick here. Unless you read carefully, you'll hit the "continue" button -- that will just take you back to the main site. Instead, look for the link under the wireless number drop down and click that. You get the PDF contract summary.

Note that under the Wireless tab is a "Rate Plan" link, but it only shows voice plan.

So, in summary, to determine your actually currently contracted services for a family plan you need to print/view a PDF for each individual family member and do the sums to produce an integrated view.

I wonder if there's a medical term for the psychosis induced by dealing with AT&T?

Next up: How to track SMS use.

[1] The primary flaw is that AT&T mixes marketing with service. There are other reasons, but that's the primary dysfunction. What I want to know is mixed with what they want to sell me. A secondary reason, is that they choose not to invest in areas that allow customers to see what their contracts are. Those investments have a low return on investment and will not contribute to someone making VP. It's just a happy accident that it works this way.

[2] Update: AT different times I got different PDFs for different users. Only my wife, a secondary number, produced the full six page report when I requested a "wireless customer agreement".

Update 12/11/11: I've studied the PDFs in more detail. They are quite hard to interpret, particularly for a family plan, but, with some study, they do show the current contract. I think the "wireless customer agreement" is a copy of the original contract, but the "customer service summary" is the current contract.

Deleting Google Profile breaks picasa share links

I  recently deleted my Google Profile. The link explains why.

I expected consequences and I am not disappointed.

Today many of my share links to my Picasa albums have broken. The albums are still gone, but the share links don't work.

It's easy to see why. Here's a link I shared last week to one of my recent albums. I've bolded the problem (omitting the auth key)

https://picasaweb.google.com/113810027503326386174/WildcatVsEagleNov2011

Here's today's link to the same album ...

https://picasaweb.google.com/jfaughnan/WildcatVsEagleNov2011

When I deleted my profile I removed my 2007 identity: 113810027503326386174. With the advent of G+ that identifier was the basis of shared image links. When I removed G+, those links broke.

Fortunately I hadn't shared that many albums recently. I got singed, but not burned.

I feel like I escaped the burning house of G+ just in time. Future exits will be far more difficult -- if not impossible.

Google doc share links cannot be changed

Google uses the secret share link feature in a few places. In several places the share link can be changed. I think of that as an essential feature; a way to correct a share mistake or deal with an escaped link.

Google Apps shared item links, however, cannot be changed.

Nobody talks about this.

Sometimes I think I'm the only person in the world who actually uses Google Docs.

Blogger is dying more quickly than expected

I've been expecting Blogger to die, but since Google has been porting it to the "new look" I thought it had a year or two left.

Today when I search in Blogger I'm getting results sorted from oldest to newest, and search ends at 2006. I'm seeing this in both the old and new UI. In the new UI, of course, search is very limited; in particular you can't navigate large numbers of results readily.

Blogger is showing more of these inconsistent and failed behaviors. If Blogger were human, I'd say it had an untreatable cancer.

The good news is that WordPress.com's Blogger Import works fairly well -- though it omits all draft posts.

The bad news is that blogging is tracing the same trajectory I remember with web site authoring. In the 90s web site authoring tools were accessible to relatively non-technical amateurs. By the 00s that market had gone away, and quality "content management systems" were aimed at professionals and businesses. I'm seeing the same thing happening with blogging -- it's becoming a professional activity dominated rather than an amateur form of communication. The number of interesting blogs expressing personal opinions rather than muted convention is also diminishing.

These things are easier to understand if you live in a climate that has ice and snow and darkness.

Lion: Only 3.5 stars on Apple's App Store

I have no compelling reason to move our family accounts from MobileMe to iCloud, though Find My Friends might be nice. Still, I'd probably do it -- except that I want to keep my iPhone and OS X Contacts in sync. Moving to iCloud for my iPhone means I need to move to iCloud on the desktop -- and i have only one Lion machine. Two others are Snow Leopard and a fourth won't run Lion at all.

I'd need to put Lion on the dual core MacBook, where it will run slowly even with encryption off. It should be fine on my i5 memory loaded desktop.

The bigger problem is that I'll need to upgrade software. FileMaker for sure, and probably a few other odd apps need Rosetta. The experience is guaranteed to be painful and expensive.

Apple's App Store is not encouraging. Lion gets 3.5 stars there, and the negative comments (only ones I bother with) are persuasive. That is, they complain about issues I know are real (such as software upgrades).

Meh. I'll wait until March or so. Maybe Lion and iCloud will both look better then.

Standard feeds for G+ Profile streams

When G+ Profiles first appeared, I recall that public posts had feeds.

Those feeds disappeared. Now G+ is more of a walled garden than Facebook.

Russell Beattle created an RSS feed app for G+, but then Google's AppEngine price increases put him out of business.

Now Jeff Turner has one ...

Google+ to RSS Feed

This site is still in beta so it might go down from time to time. If you have any issues please submit a issue to Github Issues. I had to up the cache to 60 minutes so we don't run out of API calls in a single day.

Here's my (100% public) John Gordon feed via nodester: http://googleplusrss.nodester.com/107785880910936077757.

It renders in GR. Next I'll try using http://ifttt.com/  and Feedburner to turn my G+ posts into something I can own and others can consume. I hope Jeff can make a business of this somehow.

Markets route around aberrations, and the Net is routing around Evil+.

Update: I tried a feedburner version of Jeff's converter but IFTTT didn't like it. It complained "Feed has items without valid urls".

On the other hand, ifttt will share G+ Posts to Tweets using yet another G+ to feed service - Plu.Sr. Looks like there are more than a few of these! Talk about routing around evil.

So here I am on Plu.Sr, where first sentence becomes feed title: http://plu.sr/feed.php?plusr=107785880910936077757&short=true.

From this ifttt creates posts on my unused Posterous blog: http://jgordonshare.posterous.com/ -- but only if autopost is turned off.

I tried this one with Feedburner, but it complained: "The URL does not appear to reference a valid XML file. We encountered the following problem: Error on line 77: The reference to entity "T" must end with the ';' delimiter."

It's just for play at the moment, but it looks like we'll have some options. At least until Google terminates my John Gordon Profile.

I've also set up a ifttt action to http://twitter.com/#!/jgordonshare from my G+ shares.

Next step will be to get my wordpress microblog working. Then IFTTT will create wordpress posts, and that will in turn have a feed and a twitter stream.

iOS 5 on a 3GS - not bad

I wiped a 3GS, installed iOS 5, and restored my son's apps and data.

I did it primarily to get iMessage - part of our assault on AT&T.

It's better than I thought. I expected more UI problems. I'm sure there are problems, but I expect those will improve with later versions of iOS 5.

If you have a 3GS I think it's worthwhile, but I'd wait until 5.1.

iMessage use on an AT&T iPhone without a SIM card (iPod Touch mode)

iMessage is a very intriguing product. It's available as part of iOS 5 for iPod Touch, iPad and iPhone 3Gs, 4 or 4S.

On non-phone iOS devices iMessage provides non-SMS (iMessage) texting services to other iMessage users over either WiFi or, if supported, 3G services. That's like WhatsApp.app, but WhatsApp only works on a iPhone with an active voice service!

On iPhones iMessage has two modes.

In standard mode it supports SMS/MMS messaging as well as iMessage texting. iMessaging is the default when it's supported by the receiving device; you can see what will be used before you compose a message.

In an optional mode you can disable SMS/MMS messaging and go purely iMessage. You may want to do this, for example, if you choose not to pay for AT&T's extortionary "unlimited" plans. You will still receive SMS messages (20 cents each, including spam text), but at least you won't send any. (You can tell AT&T to turn off all but 'administrative texting' if you want to avoid spam SMS and spam SMS fees.)

In a world where SMS fees exceed AT&T's mandatory minimal $15/month 200MB/mo data plans, iMessage is subversive [1]. For our family, discontinuing our $30 month texting plan and using a combination of iMessage, Facebook Messenger and Google Voice/Text more than pays for my son's data plan and old 3GS.

Siri is nice (more on that in Gordon's Notes, soon), but iMessage is the biggest thing in iOS 5. I would love to know what AT&T thinks of it, and whether those thoughts are printable in a family blog.

Alas, not everything is quite perfect in iMessage and iOS 5.01. Apple's Discussion groups have many complaints about "waiting for activation". For example:

iMessage waiting for activation: Apple Support Communities

... To update on my iPhone-off contract, even though it says iMessage is waiting for activation, I can still iMessage my friend in Australia (and I am in the USA) So I don't know how it's working, but it's working great!   Also, another one of my USA friends has an iPod touch with iMessage. It is working flawlessly..

We had no trouble at all with 3 iPhones with functional SIMs. In an SIM-free iPhone 4 in use as an iPod Touch, however, we got stuck at "waiting for activation".

The first time I used the device I think it sent messages, despite the notice. The next day, however, it could not send. I tried various tricks to no avail, including:

• reboot phone
• remove and restore my son's iCloud credentials and account.
• play with location and time zone settings
• create a contact card in iCloud with his migrated iCloud ID (@me.com) and specify that in iMessage

Nothing worked. A day later, however, his phone could again send and receive messages -- despite showing "waiting for activation".

I don't know how long it will keep working. Apple doesn't truly support use of a SIM-less iPhone as an iPod Touch, which further reduces the (suprisingly) low value of a used iPhone. I'm somewhat optimistic, however, that the current flaky behavior is a bug or a reflection of overloaded systems. I'll update this post as I learn more.

[1] If Apple integrates it with iChat on OS X, and provides a Windows 7 client ... hmm.

See also:

• Gordon's Tech: Giving your old iPhone to your kid: working around AT&T's mandatory data plan 11/3/2011
• Gordon's Notes: AT&T and the mandatory iPhone tax - even out of contract phones must pay 11/3/2011
• Gordon's Tech: AT&T Smart Limits for Wireless is almost worthless 11/4/2011
• Gordon's Tech: Turning an iPhone into an iPod touch - keep the original SIM! 11/28/2011
• Gordon's Tech: Porting a mobile number to Google Voice 11/21/2011
• Gordon's Tech: Testing Facebook Messenger as a texting alternative (4 and 3G) 11/5/2011
• Gordon's Tech: iPhone alternatives to AT&T's texting fees 11/11/2011
• Gordon's Notes: AT&T’s secret Nov 2009 mobile contract change – Elegant Evil 6/2010
• Gordon's Notes: The fear that's driving AT&T's smartphone data plan policies 11/7/2011
• Gordon's Notes: iPhone micro: How SMS pricing is accelerating the smartphone transition 11/24/2011

Martin's WordPress plugins

I think my microblogging experiments will likely end with WordPress. It's a complex world to learn however. For one thing, there are hundreds of "plugins" available for WordPress, and no obvious way to tell the useful from the disastrous.

So I'm grateful to Martin Steiger for sharing the ones he uses on different sites ...

Essential:

- Chunk Urls for WordPress
- Germanix URL (probably not necessary for you)
- WordPress Database Backup
- WP Super Cache

Essential for sites with comments on:

- Akismet (default WordPress plugin)
- Antispam Bee (in addition to Akismet)
- Subscribe To 'Double-Opt-In' Comments (if you provide comment subscriptions)

Useful / additional features:

- Better Delete Revision
- Better WordPress Recent Comments
- Comment Form Quicktags
- Comment Whitelist
- Contact Form 7
- Country Filter
- Exclude Pages from Navigation
- Get Recent Comments (depreciated)
- Google XML Sitemaps
- Intypo
- Limit Login Attempts
- No Self Pings (maybe no longer necessary)
- Optimize DB
- Posts By Tag
- WP Minify
- WordPress Popular Posts
- Yet Another Related Post Plugin

On the way out:

- AbsoluteRSS (no longer necessary)
- AntiVirus (too many false positives)
- Flattr
- Google Analyticator
- Google News Sitemap
- PubSubHubbub
- Really simple Facebook Twitter share buttons (sharing buttons slow down any site)
- Save Post. Check Links.
- RSS Cloud
- WP SUP

On my latest WordPress-based site, steigerlegal.ch, I use as few plugins as possible. Some are essential and some are needed if you wish certain features. In any case, I carefully read the reviews on WordPress.org and I try to use only plugins in active development. I have trust in some reputable developers such as Sergej Müller. I cannot review each plugin from A to Z but I never get plugins from dubious sources. Up to now, I have never had a security problem with any of my WordPress-based sites (although being careful cannot rule such problems out but it's a start).

iPhone cables and like things - Monoprice

I advertised AT&T's $6.50 iPhone cables on a corporate social site, and a colleague responded with an even better deal from Monoprice: only $3.55 each when QTY 50 purchased. They're a big $4 for less than 10. Monoprice sells a $7 wall charger too.

These aren't no-name or counterfeit cables, Monoprice specializes in this sort of product:

Monoprice, (DBA. MonoPrice.com) Inc. is an eCommerce leader specializing in high quality cables, components and accessories for computer and consumer electronics. Established in 2002, we have built our reputation by the word of mouth of our customers. The Monoprice brand's greatest claim to fame is our consistent ability to deliver premium quality products on par with the best known national brands at prices far below the retail average along with unmatched speed and service...

I'd be willing to try them. I'm told Monoprice's thunderbolt cables are less reliable, but those seem hard for everyone to make. I'll be looking at them for future purchases.

iOS 5 updates and lost game data: wait for iOS 5.1

I don't know how widespread this is, but it happened to us despite having a robust backup ...

Lost game data: Apple Support Communities

I just migrated my son from a 3GS to an iPhone 4 running iOS 5.01. I wiped the iPhone 4, reactivated it, then restored from a complete backup of his 3GS. His Harry Potter Lego saved game setttings were not preserved. He was, understandably, disappointed. I haven't evaluated his other games. In general iOS 5 migrations have been difficult, with more than expected work to reinstall apps, delete crashing apps and resume them, and restore accounts.

I suspect the problem is some combination of ...

• a bug (of course), perhaps related to Apple's confused identity management infrastructure (Cloud ID, AppleID, StoreID)
• a change in how applications are allowed to store data. If an app is not updated prior to updating, and the app stored data in a fashion that's no longer supported, then data will be lost.

I've moved three users from iOS 3 or iOS 4 to iOS 5, and it's been painful each time. Most credentials have to be recreated. ActiveSync account migration doesn't work at all; accounts have to deleted and recreated. Migrating Google Authenticator was particularly difficult (that's Google's fault though). Worst of all, for my son, is this data loss.

I don't know if Apple will fix this, but I recommend waiting for iOS 5.1 if there's no driving necessity to upgrade to iOS 5.

iOS 5 -- What Apple didn't fix

My personal list of things that are still broken in iOS 5 (to be expanded):

• Migration problems: 
• ActiveSync accounts must be removed completely, then restored when updating from iOS 4 to iOS 5 (this may be a new bug)
• My son's Harry Potter Lego game data was not saved when I migrated him a 3GS to iOS 5
• Alerts: Now there is a birthday alert, but it's a 1 week alarm. We need 2 week alarms for all events.
• Colors on Calendars: Still can't control color assignment, still get colliding colors with ActiveSync calendars, color assignment algorithm still doesn't look across multiple calendars

Migrating MobileMe family accounts to iCloud

I've started migrating family accounts to iCloud. This explanation from Apple Discussions is helpful ...

With a Family Pack, the master account holder and the sub-account holders can each migrate to an individual iCloud account by going to http://me.com/move and entering their email address and password (not www.icloud.com). The order they do this in is immaterial; if the master account moves first it can no longer administrate the sub-accounts. If a sub-account moves first the master account cannot create a new sub-account to replace it. Once migrated each account becomes a full iCloud account entirely separate from the others. The master account holder will get the 20GB storage upgrade free until June 30th 2012; the sub account holders will not, and will have only the basic 5GB.

I began with #2, currently using a SIM-less iPhone 4. He's got almost no data to lose.

I want him to continue to use my Apple ID however ...

... iPhone, iPad, and iPod touch must have iOS 5 or later. Note: When you are asked to provide your Apple ID during iOS setup, use your MobileMe email address and password. To use a different Apple ID for iTunes and iCloud, just go to Settings > Store on your iOS device after you've finished the iOS setup assistant...

... Inactive and expired MobileMe accounts do not need to move to iCloud. Simply use your inactive or expired account to sign up for an iCloud account and follow the onscreen instructions. See this article for more information...

and

When you first set up your iOS 5 device, enter the Apple ID you want to use with iCloud. If you skipped the setup assistant, sign in to Settings > iCloud and enter the Apple ID you’d like to use with iCloud.In Settings > Store, sign in with the Apple ID you want to use for store purchases (including iTunes in the Cloud and iTunes Match). You may need to sign out first to change the Apple ID.

Following this path I learned that #2 will still have MobileMe Gallery, iDisk and iWeb Publishing through 6/30/2012 -- even after moving to iCloud. Better than I'd expected. No keychain sync though. I hope that comes back in some form.

I had to enter his iCloud (same as MobileMe) credentials to setup Facetime and iMessage on his SIM-less device. Although the phone doesn't need a SIM for this, it can't be in Airplane mode.

So far, it has gone better than expected.

Update: Thinking this over, I realize I need to update my machines to Lion before I move Emily and I. So this will take a while ...

Turning an iPhone into an iPod touch - keep the original SIM!

The kids had used old iPhones as IPod touches without trouble, but apparently I'd never done a factory reset.

If you do that, the phone can't be used -- until it's activated.

You need a SIM card that was either originally used to activate the phone, or that works with a currently active phone:

Using an iPhone without a wireless service plan

Follow these steps to use your iPhone without a wireless service plan:

Insert the SIM card from your new, activated iPhone or one that was previously used to activate the original iPhone.

Connect the iPhone to iTunes on a computer connected to the Internet.Once iTunes activates the device, you're free to use the iPhone as if it were an iPod touch.

... It may be necessary to repeat this process after updating or restoring the device.

Yikes. If I'd known that then, when I got my 4S, I'd have asked to keep the old SIM card and had them give me a new SIM. Instead we just swapped the old SIM. (Not sure if AT&T will do this, it's an advantage of ordering you iPhone from Apple for home delivery.)

Lesson: When you upgrade your iPhone, keep the SIM card that was previously used to activate the device.

Update: On the brighter side, iOS 5 allows use of FaceTime and iMessage without a SIM card. You can't use them in airplane mode however.

Update 11/30/2011: iMessage behavior is flaky in a SIM-Less iPhone. More on that later. In a separate experiment on another phone I found a SIM card that had previously been used in a different iPhone worked to get pasts the iTunes check even though the number no longer existed and the SIM card was no longer valid.

Update 3/5/2012: iMessage was much better by mid-December and has worked well since.

Google's 2-step verification is (almost) the spawn of Satan - iPhone upgrade edition

Two months ago I decided Google's 2-step verification was an incomplete mess unsuitable for use by non-geeks.

Today I decided it's the spawn of Satan.

This is what happens if you refresh an iPhone running Authenticator - either a new phone or restore from backup

1. Restore iPhone from backup
2. Authenticator settings gone
3. Per directions to account page for 2-step verification settings.
4. Discover, despite 30 day authorization, my computer wants authenticator token today.
5. Fortunately, I have my old phone. That works.
6. Realize that there's no support for authenticating a new phone. Ok, I'll just turn off the iPhone ...
7. Get the QR code. That works ... but
8. All the friggin' application specific passwords are gone -- all revoked.

Do you know how friggin' long it takes to enter all those application specific passwords across multiple machines and operating systems?! Can I scream now?!

Friends don't let friends use 2-step verification.

Update: A few minutes later and, for now. I see my (not) application-specific passwords and they still seem to work. So only almost the spawn of Satan. Google needs a workflow to support migrating from one iPhone to another.

See also:

• Gordon's Tech: Implementing Google's two factor authentication - early days
• Gordon's Tech: The massive security hole in Google two factor authentication - problems
• Gordon's Tech: Making the most of Google's alternative "2-step" verification model - more problems
• Gordon's Tech: Google Chrome sync does not work with 2-step verification - still more problems

 

Good habit: review sites with access to google services

Through your Google Accounts page you can edit "connected sites" and "application-specific" passwords (they aren't btw).

I did this tonight. I was surprised at how long my list was. How did Facebook ever get access to my Google Contacts? They are diabolical.

I deleted most of  the items on the list. This is one practice I want to make a habit.

AT&T sells iPhone cables for $10 with 3 for 2

I've never come across an AT&T store bargain - until today.

Turns out AT&T sells their own iPhone/iPod/iPad charge/sync cable - for $10. As of today they also have a "buy two get one free promotion", so I bought 3 for $20.

Excluding obviously counterfeit $2-$3 cables (caveat emptor), I've not seen these cables on Amazon for less than $20. Griffin's powerblock/cable combo for $23 or so has been the best deal.

Since Apple's last two years of cables have been absolute garbage (separation at iPod end), my family needs some extra cables. Now we're flush.

ifttt, Google Reader Share, and Wordpress

ifttt has a WordPress channel, including a post action.

Google Reader's share RSS feed is still active, and can be exposed with the Keakon extension.

So using iftt could I blog to Wordpress by clicking the Keakon-Share link in Google Reader?

Be a fun experiment anyway.

Google Chrome sync does not work with 2-step verification

As best I can tell Google's two-factor ("2-step") verification is incompatible with Chrome sync. There are two ways it fails:

1. During initial authentication you are required to enter a full access password, the Authenticator token won't work. (Laughably, Google calls these 'application-specific' passwords. That's a lie. I wish they'd stop repeating it.) This defeats the value of the Authenticator's keystroke-logger protection.

2. You can't use your Google account authentication to encrypt your sync store. Maybe it uses the 'application specific password'. When I try this, Sync hangs - but tells me it has succeeded. Using a separate sync password works.

There are lots of similar bugs with use of two-factor. It's really not finished; I wonder if it's one of the projects that Page has terminated. I still use it, but I don't recommend it to anyone else. The illusion of security may be worse than no security.

Microblogging and Google Reader: Tumblr Fails

I got pretty far along on the Tumblralternative to Reader Shares. I even created a kateva.org subdomain with a Tumblr IP A Record to give my Tumblr blog a kateva.org URL.

I tested with Google Reader Mobile and desktop, but the workflow for micro-sharing was too awkward. That looked promising, but all the posts were going to my original Tumblr blog, not the full powered one I'd setup for sharing.

What was up with that?

That's when I discovered the weird world of Tumblr primary blogs ...

Help Center | Tumblr

... What is my “primary” blog? Your primary blog is the one created when you sign up for Tumblr. It represents you (with its name, link, and avatar) when you follow or like other Tumblr blogs. You can read about blog management to understand the differences between your primary blog and additional blogs. Can I switch my primary blog? It’s not currently possible to switch or move your primary blog to another account. You can read about blog management to understand the differences between your primary blog and additional blogs....

Cue the ominous music. I could see where this was going ...

Blog Management | Tumblr

Each Tumblr account comes with a primary blog. A primary blog can fully use all of Tumblr’s social features including Follow, Like, Reply, Ask, and Submit. But, a primary blog cannot be password-protected and cannot be multi-user.You can also create additional blogs on your Tumblr account. An additional blog can be password protected for privacy and security and can be multi-user. But, an additional blog cannot fully use all of the Tumblr’s social features...

The primary blog that is made when you create your Tumblr account will always be the primary blog for the account. It is not possible to reassign which blog on your account is the primary blog. And, due the way in which Tumblr is architected, it is unlikely that we will be able to support reassignment of the primary blog in the foreseeable future. It is also unlikely that we will be able to support password protection on primary blogs....

Yes, it's all about that d*mn closed-world money-making social stuff.

The primary blog is the one that receives my shared items from Feedly. It won't do for what I want.

So now, like Posterous, Tumblr has failed.

Only Twitter and, perhaps, WordPress microblogging, remain.

Oh, and, of course, HiveMined.

No. Please. Not ... not ... Blogger!

See also:

• Tumblr - a Google Reader social replacement?
• Gordon's Tech: Posterous - a Google Reader social replacement?
• Gordon's Notes: Dapocalypse now: Google's day of infamy
• Gordon's Notes: How to replace Google Reader
• Gordon's Notes: The end of Google Reader shares and the rebirth Gordon's twitter feed

Aperture 3.2.1 recurrent crash - EXC_BAD_ACCESS (SIGBUS)

This ends well - because I have backups. Other than the necessity of backups, the main lessons of this post are;

1. Aperture 3.2.1 Libraries can become corrupted.
2. Aperture 3.2.1 has some wicked bugs.
3. A few debugging tips.
4. It is unwise to edit in Aperture until all import processing is completed.
5. How to manage when disaster strikes

This morning I did a routine import of 30 or so CR2 images from my Canon T2i using Aperture Import. Import completed normally. As images were post-processed I worked on some red-eye editing. I mis-clicked somewhere, and the spinning pizza of death appeared. About a minute later Aperture crashed, leaving this crash log ...

Process:         Aperture [75650]
Path:            /Applications/Aperture.app/Contents/MacOS/Aperture
Identifier:      com.apple.ApertureVersion:         3.2.1 (3.2.1)
Build Info:      Aperture-201094000000000~2
Code Type:       X86-64 (Native)
Parent Process:  launchd [41554]
Date/Time:       2011-11-25 09:15:19.689 -0600
OS Version:      Mac OS X 10.6.8 (10K549)
Report Version:  6
Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: 0x000000000000000a, 0x0000000138d4d000
Crashed Thread:  23

I tried five more times. Each time Aperture opened normally, and about 30 seconds later, presumably as it resumed image processing, it crashed with the same error.

I knew I had backups, so I didn't have to panic. In fact, I have three backups:

• Encrypted image with Carbon Copy Cloner to local drive nightly. This is a drive clone of course, but CCC also saves changed and deleted files in a 'just-in-case' folder. So it's a Clone+.
• Encrypted image with Carbon Copy Cloner in my office that's 3 weeks old (offsite).
• Time Capsule backup to a 2TB drive upstairs.

It was easy to demonstrate that the problem was in the Library. I option-launched Aperture and created a new Library. It was fine. I also removed my Aperture Preference files and Aperture still crashed on Library launch.

As an experiment I used EasyFind to locate the images I'd imported. I'd renamed them on import ("_Thanksgiving_") so it was "easy to find" them using EasyFind's package search option. I moved them out of the Aperture Library. This time it launched normally.

Whatever was wrong with Aperture, it was something related to processing the images I'd imported that morning -- or a problem with the images themselves.

I then used ImageCapture [1] to bring in the suspect images from my SD card. There were no problems. I added these to an empty Aperture Library. No problems.

By then my 45GB restore had completed so I put the images into the restored Aperture Library. No problems. Just to finish the cycle off, I again performed the red eye correction associated with the big crash. No problems.

One experiment I didn't do was use Aperture First Aid to repair or rebuild the Aperture database. Speaking of that kb article, I suspect this paragraph is relevant to my problem ...

...If Aperture does not open, it may help to defer creation of previews. Press the Shift key immediately after you start Aperture to prevent preview generation for that session. If a damaged image in your library is preventing normal opening, this may allow you to start Aperture...

i don't think the original images in my Library were damaged, but I do think something went wrong with Aperture's preview creation. Actually, I have a hunch that the problem is related to Aperture's face processing -- the EasyFInd recovery process I experimented with returned a number of face "thumbnails' -- but only a number. As though face thumbnail processing was not completed.

I hope the fixes I've outlined here will help others. The bottom line - be sure you have a full backup of your Aperture Library. I prefer two fully automated backup methods that have almost nothing in common. Backups are too unreliable to trust a single backup method.

[1] This app has its own really annoying bug. When used it stores the path it last saved images to. If you change drives it keeps asking for the old drive -- and tries to mount it. Dumb. Easy to delete preferences to fix.

Update: Terence Devlin - an Apple guru, also suspects the Faces process.

Update 12/20/2011: I suspect it was due to bit rot (bad sectors) on my drive.

WordPress.com - Blogger Import

I'm continuing to examine exit strategies from Google 2.0 as I prepare to delete my TrueName G+ Profile [1].

Other than Gmail and Google Apps, my strongest Google Connection post Reader Social is Blogger. It hosts this post, for example.

I've considered Posterous and Tumblr, but both have fallen short. I think I'm going to have to learn the WordPress world, starting with hosted at WordPress.com and migrating to self-hosted.

I suspect that will take a while, but I'm starting to experiment, starting with an experimental import of tech.kateva. org to WordPress.com. It's not a complete import, draft posts are not included. For example, import dialog shows 3738 Gordon's Tech posts, but that omits 467 draft posts. For Gordon's Notes it shows 5754, so 1277 draft posts are omitted.

Wordpress also supports import of a Blogger Atom export file. I don't know if that will include Drafts, but I may experiment with it.

I can live without the draft posts, especially since I'll still have access to those drafts post-migration. I can plumb those that look interesting and abandon the others.

I don't expect any quick changes. In any case I own the domain kateva.org, so the changeover ought to be relatively invisible to my visitors (though I bet feeds will need to be revised).

If you'd like to inspect the results of the import, here's Gordon's Tech at WordPress.com - http://gordonstech.wordpress.com/. It's pubic but not-indexed with a temporary URL. The import went more smoothly than I'd expected, but it uncovered two surprises, both of which make me keen to move sooner than later ...

• The "New" Blogger has no UI control to move to the earliest post. The "Old" Blogger UI has this.
• Old Blogger doesn't show my 2003 Blog posts, it starts with 2004. WordPress imported them.

[1] I'll keep my John Gordon profile -- until Google deletes it as a TrueName violation.

Google's Omnibox and implementing Google search alternatives

Since I'm distancing myself from Google 2.0, I was receptive to Phil Bradley's alternative engine advice. I'm testing blekko (spam free) and Duck Duck Go (no tracking). I believe both are wrappers that enhance Google search.

Naturally, this experimentation works best with Chrome's Omnibox. It's very easy to add search engines; Chrome 'detects' an engine during a search site visit and adds them to it's collection. You can make any a default, and define a text shortcut. Type the shortcut in the Omnibox, hit spacebar, and your custom search is ready.

I made Blekko my default, and assigned it the letter 'b'. Google gets 'g', etc.

While I was at it, I defined a search string for one a Google custom search engine that searches my own content (web pages, blog posts): <http://www.google.com/cse?cx=009911250981951822495%3Aphjhjp-tdfa&ie=UTF-8&q=%s>. I already had one for searching my dev team's Rally project.

I can distrust Google and still appreciate Chrome ... right?

• Using the address bar (omnibox) - Google Chrome Help
• Managing search engine options - Google Chrome Help

Update 11/24/2011: I ran a search on moving from blogger to wordpress on both Blekko and Google. Blekko wasn't just a bit better. It was immensely better.

Porting a mobile number to Google Voice

We had two kids on our AT&T family plan when AT&T hit us with their smartphone tax. The cost of each kid phone went from @$16/month to $32/month (estimated real costs) and that includes a data plan we don't want.

I think we know why AT&T is making these desperate price hikes, but we're not playing along. After reviewing several options we expect to save about $600 over the next two years with these 3 steps [1]:

1. Cancel the phone account for our youngest. Turns out he's not interested in either phoning or texting. He'd prefer we put a fraction of his phone bill into games and movies. In retrospect he could have waited another year. So we've canceled his account -- with a twist. We're porting his AT&T number to Google Voice.
2. Replace our $30/month family texting plan with a combination of Google Voice, WhatsApp and Facebook Messenger (yeah, it's still messy).
3. Canceled a now useless SmartLimits account.
4. Use DataMan Pro to help keep my son under his 200MB/month data limits. We disable Safari and YouTube on his phone, so iTunes is the main data drain. So far he's doing well.

When we canceled #2's phone I decided to salvage the phone number. It's a memorable local number and when I recently added a GV line to my business number there were no local numbers available. I ended up choosing an area code from my former home town - Escanaba Michigan. Since I was going to cancel his phone with a memorable local number, I decided to make it mine.

I followed Google's minimalist number porting directions. Unfortunately you can only do this with a mobile number [2] - at least for now. Here's how it went:

1. In Voice Settings:Phones I clicked the Change/Port link.
2. Checked lots of warning checkboxes.
3. I tried following the phone confirmation directions on the old Nokia that currently holds my son's SIM card. (We pulled it from his iPhone when we got hit with the rate increase.) It didn't work -- Google couldn't recognize the touch tones!
4. I put his SIM into his 3GS -- Google recognized those tones.
5. I filled out my AT&T account information.
6. I paid $20 into a new Google Wallet account (on my work account)

In theory the transfer will be done in 2-3 days and my old GV business number will work for about 2 months before dying.

In theory my son's AT&T number will disappear and that account will close. I'll check in 2-3 days. Of course since this is a family account things will probably be messier. He's not under any contract (no subsidized phone, he "brought his own phone"), but we'll see what happens. I'll update this post.

john

[1] Since we're paying for a data plan (200MB/month). [2] It's convoluted, but I assume you can move a landline number to a mobile number, then from there to Google Voice. I'm thinking about that ...
[2] Presumably we could move it to a family mobile account, then from there to Google Voice. Bit expensive and a hassle.

See also:

• Testing Facebook Messenger as a texting alternative (4 and 3G)
• iPhone alternatives to AT&T's texting fees
• iPhone on a budget: The AT&T GoPhone PayGo Option
• Giving your old iPhone to your kid: working around AT&T's mandatory data plan

Update 11/22/2011: The next day my AT&T account showed #2 was gone. I tested the number and it called Google Voice. I called AT&T and was told the account had been cancelled. Since AT&T bills monthly service in advance (who new?) I've already paid for the billing cycle that ends in 6 days. There's no rebate on that $12 fee. Any expenses incurred this month (overage fees, long distance) will show up on next month's bill. So this went smoothly.