iCloud group replication resolved by AppleScript

It's been a month since my disastrous MobileMe to iCloud migration. Over the course of a week I figured out a convoluted solution for iCloud's CRLF/LF end of line bug, but I was left with "metastatic group replication" in my primary iCloud account/quaternary AppleId [1]. Each of about five Address Book groups had been replicated about 50-100 times (only one of the replicants has associated contacts).

 

The Groups were in iCloud - not Address Book. So even deleting my Address Book Library files didn't remove them. Bento couldn't see them. Mountain Lion didn't help. The only option appeared to be tedious manual deletion. My iCloud account was poisoned.

 

Today I figured out a solution. I had to learn a bit of AppleScript [2] but I'll recover. It was worth it (in ML Address Book has been renamed Contacts):

tell application "Contacts"
repeat 50 times
set theGroup to group "MyGroupName"
delete theGroup
end repeat
save
end tell

It's a crude hack. I had to change "MyGroupName" five times, and when the replicant count dropped I had to reduce the repeat number so it would execute. (One could drop the count  parameter and see if the repeat loop stops when it errors with "missing value"; that would be much faster. I was being cautious.)

At one point Contacts didn't respond very well. I quit and restarted and completed the process.

When I was done I confirmed iCloud's web view was also Group free. Then I purged all the Contacts in that account [3], it's now pristine.

I'm sure an AppleScripter could turn this into a general script for removing all groups all at once. Or could get very clever, and only remove groups that had no contacts (so one could salvage the group/contact relationships).

I didn't need that. My problem has been fixed.

Incidentally, I think iCloud sync works a lot better in ML than Lion. iCloud updated very quickly. 

[1] I have 4 AppleIDs. Two or three of them have iCloud accounts. One has all of my iTunes and AppStore purchases. One was my MobileMe account. A third, which I've never used, acquired, apparently by email transfer, all of my Apple device records. Does Apple have any idea how screwed up their AppleID/iCloud accounts are?

[2] Of which Jamie Zawinski wrote: "I used to think that PHP was the biggest, stinkiest dump that the computer industry had taken on my life in a decade. Then I started needing to do things that could only be accomplished in AppleScript."

[3] They're safe in Snow Leopard until I move my primary machine to Mountain -- which might be sooner than expected. ML seems to be everything Lion should have been.

Mountain Lion - my experience

I've never installed an OS on its debut. I generally wait 5-6 months for the .4 release.

This time was different. I really don't like Lion (Apple removed it rather quickly from the App Store, didn't they?). I also have a MacBook Air that I really don't rely on -- so I could sacrifice it.

Notes so far ...

• I backed up the installer prior to installation. (I did the same thing for Lion. Glad I did since it's now gone from the App Store. I'll have it if someone needs it.)
• In addition to the 4+GB installer there were GB or so of iWork and iLife and firmware updates to apply.
• The Guest account was reenabled. I removed it.
• I was able to add a keyboard shortcut of cmd-opt-S for Save As... (instead of cmd-opt-shift-S). It's good to have Save As back. I also changed Settings:General so I'm asked if I want to save changes on exit (I may change it back to default).
• Unsurprisingly, my broken iCloud Contacts database (hundreds of replicated groups) is still broken. Contacts sucks less than it did though.
• It doesn't feel any faster or slower than Lion.
• I'd read Notational Velocity didn't work. Works fine for me. I run as a non-admin user fwiw. Sync to SimpleNote worked to. That's a relief.
• There's supposed to be a LaunchBar bug with Contacts, but mine was fine. I have iCloud disabled so maybe that helps.
• My Time Capsule backup was gone after the upgrade. Good thing I wasn't relying on it. When I ran TM it showed no backups. I used the option-click trick to get TM to validate my backup and now the backups are all accessible through Time Capsule. I think this may be related to the 7.6.1 firmware bug with older Time Capsules.
• I enabled voice dictation; that's not automatic. (Lately it's working on my iPhone, but I doubt it works as well on the desktop. Unrelated to this, but Siri is actually working again.)
• None of my apps were moved into an 'incompatible application folder'. This machine has always run Lion, so that's not surprising.
• Chrome seems fine, I'd heard it had problems. Maybe Google fixed 'em. I had to make it my default browser again.
• There are new Parental Controls for the Game Center. I didn't see any other changes there.
• I think I'll like Messages. It does remind me though that I'm going to have to come up with some way to fix my broken iCloud account. I don't think Apple will be much help.
• What's with Video Chat being in both Messages and FaceTime? Isn't that a bit odd?
• I like the unification of App Store and system updates.
• For kicks I tried using Google's m.google.com ActiveSync service as an Exchange Account. Didn't work - couldn't contact server.

Overall it feels like a less broken version of Lion. Maybe I'll upgrade my main machine in September rather than February.

My Apple purchase records jumped from true AppleID to one I've never used. This is what I think happened.

I made a rare "Genius" consultation today. The visit confirmed that there's no "safe" Apple-approved way to fix broken glass on an AT&T carrier-unlocked iPhone [1].

That was bad, but along the way we tried checking the devices associated with my Apple ID. That would include several iPhones, iMac i5 27", MacBook Air, iPad, iPods, etc, etc.

Except there were NO devices associated with my Apple ID. All of my registration information was gone. Vanished.

Fortunately the 400+ iTunes purchases associated with that Apple ID appear to be intact [3]. So what happened?

I contacted Apple support who, of course, had no idea what had happened. (I'm sure they thought I was merely senile, but the rep was very polite.) However Apple support's reverse number lookup on the phone I was using brought another of my unwanted four Apple IDs. That AppleID had "lots and lots of purchases".

Except I've never used that Apple ID when doing any kind of purchase or authorization. I always use the (.mac) Apple ID associated with my iTunes account. In fact, I only discovered that AppleID existed about 2 months ago!

So I checked what devices were associated with each of my Apple IDs. I went to Contact Apple Support and chose the Your Products option. This is what I found:

• AppleID (used for all iTunes purchases and for all product registrations and authorizations): 0 items.
• AppleID associated with an old developer account: 18 products going back to 2002 or so.
• AppleID associated with an abandoned MobileMe account: 0 items
• AppleID associated with a now broken iCloud account (formerly working MobileMe account): 0 items

So everything was now under a single AppleID, but it was an oddball AppleID that I've never used anywhere. How the heck did that happen? Why choose that AppleID rather than a complete strangers?

I don't know of course, but the most likely explanation is that the Apple's IT systems are kludged together.

I'm guessing that Apple had one IT systems that was used by iTunes and that an older system used for product registration. At some point in the past the "product registration system's" true "Key" was either a phone number or (more likely) an email address. Recently Apple lashed together the two systems, perhaps attempting to "join" on the email address.

That's where my May 2012 MobileMe/AppleID bug came in

... I've figured out the bug. It arose as a side-effect of changes to the way Apple IDs work, and it only impacts people who are still on MobileMe accounts and who have the same email address associated with two Apple accounts prior to the time Apple made that illegal...

I won't repeat the details of the bug here, but the workaround was to remove an authenticated email address from my primary AppleID and associate it with ... yes... that oddball developer account AppleID. 

I bet, in database terms, that email address was the "Foreign Key" that linked my iTunes controlled AppleID with my Mac purchase records. Moving the email address from one AppleID to another causes the database query to associate purchases with the AppleID I'd moved it to [2]. 

I could try moving the email BACK to my primary AppleID, but I'm afraid I'd lose my purchase associations altogether.

[1] Apple's only approved fix is a refurb substitution. Unfortunately, Apple will replace an AT&T-unlocked iPhone with a locked iPhone, the Apple product database doesn't correctly track AT&T changes to a phone's lock status. After replacement the IMEI does not match AT&T records, so the new phone cannot be unlocked.
[2] Reading my blog post from May I even noticed the problem then, but I was too tired to f/u on it: "Apple's Support Profile is supposed to show the products associated with my Apple ID. I think it used to. I don't see them any more. It says my home number is associated with a different Apple ID..." So maybe my home number was also associated with the moved email.
[3] Apple's iTunes group manages AppleIDs, not the Mac group. 

Mobile broadband hope: Walmart, TruConnect, Netzero, Sprint, Amazon and why I'm waiting on my next iPhone

I recently reviewed the state of American mobile broadband, including mifi, tethering and iPad hotspots. Bottom  line - the Verizon/AT&T duopoly means the market is bleak and getting worse. I can believe Srinivasan's claim that "Mobile service providers are moving towards just one bundle of voice and data at $100 per month.

That's a lot of money, even for the 2%. It's ridiculously expensive for the mass market. If Verizon and AT&T succeed, they'll put immense pressure on Apple's margins -- and wipe out Google's margins completely.

Unless Google acquires T-mobile there's only one option in play - Sprint. The weak, wounded, third rate carrier that desperately needs friends.

Except the option isn't really Sprint, but rather MVNOs [2] that have negotiated access to Sprint's aging CDMA network.

Sprint, and its motley allies like TruConnect Mobile MiFi. It's a "mobile virtual network operator (MVNO) that sells 3G mobile data service for laptops and tablets such as the Apple iPad and the Kindle Fire via the Sprint wireless network" (wikipedia). They've recently partnered with Walmart to deliver Internet on the Go.

'Internet on the Go' seems promising -- buy a data block with, practically speaking, no expiration. There's only one glitch. The web site doesn't mention price.

Yeah, that's weird, but you can find the price listed on Walmart [4]. The current price seems to be $45 for 1 GB [1] That's a lot more than the US-standard monthly charge of $50 for 2GB. The big difference though, is that the 1GB doesn't expire. My wife, for example, does quite a bit on her iPhone and is often below 200MB/month (no video, no software installs, no purchases, etc). With similar usage I wonder if I could stretch that 1GB out to 3-4 months -- at which point it's relatively economical. [3]

Walmart is a potential disruptor. You know that Verizon and AT&T are keeping one eye on them, even as they put their other eyes on Google and Apple (they have lots of eyes).

Amazon is another potential disruptor, it's already a covert MVNO player. 

Then there are the small guys like NetZero offering a $35/month 2GB plan over the Clearwire WiMAX 4G Network  (alas, this Engadget review doesn't match current pricing). Ok, that's no better than Virgin, but the difference is their $10/month 500 MB plan and $20/month 1GB plan - if you can use ClearWire. [5]. Most importantly, NetZero doesn't have overage fees ...

A. With NetZero 4G Mobile Broadband there are no overage charges. Customers cannot accidently exceed their monthly data allotment. We alert our customers when they reach their data limit and, if they want to continue accessing the Internet, they can either quickly and easily update their data plan, or buy extra data "Top Ups" that are good through the end of their billing cycle.

That's big. NetZero's hard stop puts them in the same league as Walmart's data block plan, and moves then well ahead of TruConnect's capless per MB fee.

So things are more interesting in the US market than I'd thought. I'm going to be researching both Walmart and NetZero; I'm likely to try one of them with my MacBook Air.

This also contributes to a decision to hold off on getting a new AT&T iPhone. I want to wait a few months and see what happens in this rapidly changing world. I'd love to buy an unlocked device ($600-$700) and bet able to, say, get voice and data with tethering for, say, $50/month.

- fn -

[1] The recommended MiFi (Mobile Hotspot) costs $120. I wonder if the Virgin Mobile Broadband2Go MiFi 2200 would work, but I haven't researched that.

[2] It's forgotten now, but prior to Apple's AT&T announcement there was widespread speculation that the iPhone would run on an Apple MVNO. Alas, that was not to be. Watch this US MVNO list for potential disruptors.

[3] We have good Sprint service in most of MSP. Note that when one is paying for each bit fetched, Chrome's prefetch is a bug, not a feature.

[4] TruConnect's own package costs $40 per GB but $5/month. So if one assumes use of 4GB data per year Walmart's and TruConnect's cost are both $180/year. The catch is that that a data accident could run up a large TruConnect charge, but a Walmart credit provides a hard stop. The Walmart plan is a much better deal. 

[5] MSP can. So NetZero's cost could be as low as $120 (plus taxes/fees) for 6GB of data compared to Walmart at $180 for 4GB of data with equal overage prevention. NetZero relies on the quirky Clearwire network, Walmart uses the slowish Sprint network.

Update: I decided to gamble $50 and try NetZero using their 'free' 200MB option. My first experience was using their $#!$@ insane web site registration form, which does not appear to have been tested with Chrome. That led me to do a bit more research, which found sites like this. I couldn't find any reviews from people I trust, just a few 1 star user comments on a PC Magazine review.

On the other hand, while the Internet on the Go site said my area was covered, the Walmart.com web site mifi order form said it wasn't. (I think the first is correct.)

I think I'll give this a bit more time to settle out.

Update 7/28/12: A few similar Clearwire-based potential disruptors...

• FreedomPop: Similar to NetZero, but less costly. They were supposed to launch in "mid-2012", but are still struggling. They got a wee bit of VC money a month ago.
• TIng offers a Huawei Hotspot but their monthly rates aren't competitive.

Carbon Copy Cloner is now commercial. Good change, bad execution.

[See update: The lack of notification was a screw-up.]

Carbon Copy Cloner, one of my best loved apps, is no longer donationware. It is now a commercial product with a 30 day trial period.

That's great -- except the contract transition occurred during what seemed like a routine (albeit big) upgrade. I didn't see any warning that this was an unusual upgrade; but I've been a CCC user (I donated before there were post-donation registration codes) for a long time. Maybe there was a warning, but it was way too subtle for me. I don't see any notice on the front page of CCC's web site.

This is what I wrote Mike Bombich, the author of CCC.

I absolutely don't mind paying for CCC; in fact I'm glad to have that opportunity. I donated to CCC before, but that was before you tracked donations. Time for me to pay up again. That's not my problem.

I'm shocked, however, by a CCC update that made a contract switch without warning.

What you should have done was to do an update first that warned users that CCC was going to end-of-life on the donation model. Then the infrastructure would be in place to notify users that the next update would cost.

I used to use SuperDuper. I didn't mind paying for it, I used CCC because it was better. Now I have to decide whether to continue with CCC, or go through the pain of transitioning back to an inferior product because of how you handled this.

An apology to your customers would help a lot.

Again, I fully support your transitioning to a commercial model. I'm glad you did that. I'd have happily paid if you'd only warned me prior to the update. Of course I would have updated.

But you didn't warn.

I'll see how he responds before I switch back to SuperDuper.

Update: Mike Bombich responded with a sincere apology. He's amended the splash screen and release note with this warning. This was one of those bad-day mistakes; something that was missed in beta testing. He has a record of prior donations; he'll grant a license even if a donation was made prior to the 'registration' feature. In my case I'm happy to pay, I got my $10 worth a long time ago.

American MIFI - priced for a limited and shrinking market

I priced US cellular 3G/LTE data access recently. I was looking for something like $10/GB with 12 month expiration, I could then spend something like $60 every 2-4 months.

I didn't find anything like this. Disregarding device cost, most vendors came in at $50/month for 2GB [1]. That included DataJack, T-Mobile and AT&T iPhone tethering.

Virgin Mobile (Sprint reseller subsidiary) seemed to be $35 a month, so it had a clear price advantage. 

The best data access was an iPad at $30/month (no contract), but I assume a good part of the LTE iPad's price premium is an up-front payment to the carriers. So this pricing is probably equivalent to the Virgin Mobile pricing. A March 2012 article  claimed Verizon allowed LTE iPad tethering, but that seems quite unlike them. I haven't found any discussions of this; I assume it's false.

More recently AT&T and Verizon have "coincidentally" created very similar shared data plans that are effective price increases for most customers -- their "peculiar" coordination makes it unlikely that either will compete on price (or anything).

The bottom line was a significant price advantage with the Virgin/Sprint MiFi, but after reviewing the market no solution appealed. My impressions are:

• The MiFi market is going away. Carriers don't like them, and they're too geeky for most consumers. The US market is being driven to device-specific LTE access.
• Tethering is going away. Carriers don't like it; even T-Mobile doesn't seem serious.
• AT&T and Verizon have carved up a nice duopoly in the US. They may weather the end of SMS better than most of us would hope. Sooner or later, we're going to have to sign on to their shared data plans.

I'm putting my MiFi plans on hold for now. It will be interesting to watch how this market evolves, and what kind of tethering options will be available with the 8" iPad [2]. I'll be looking into the mifi market again in November.

[1] Of course they rarely price things so clearly; I just looked at what would provide me with 2GB over 30 days. This is contract-free pricing. There are often hidden fees  and caps tacked on to mobile charges; these corporations have mastered the profitable art of concealed pricing.

[2] The most anticipated product in recent Apple history. I suspect it will be $300 without a Retina display. 

Update: Virgin Mobile USA is a Sprint subsidiary, no relation to Virgin any more.

Update 7/22/12: I managed to overwrite this post with an older version. Oops. Here's roughly what I removed:

• Enon tells me in comments that the Verizon 3G/LTE iPad provides hotspot service at no extra fee - at least under his contract. The current contract may be different, but this is worth looking into. This would be a significant advantage over the AT&T 3G/LTE iPad! The monthly data fee for an iPad is $30 (cheaper than Virgin Mobile), but I suspect the purchase price of an iPad includes a carrier payment. So the real cost is probably about $35/month and thus comparable to a mifi. Even so, this is/was probably the best hotspot deal on the US market.
• Sprint iPhone has no tethering advantage, it requires a separate payment plan.
• See also: Benedict Evans • iPhone pricing and US market share. Interesting examples of how subsidy agreements in different markets changes iOS value.

Update 7/22/12b: Things aren't quite as bleak as I'd though. Walmart and others are trying to spoil VerizATT's party.

OS X Limitations: working around deletion of large numbers of files (time machine image)

I've run into problems like this, when I was unable to delete an OS X sparsebundle image on a Time Capsule.

I think I was able to use rm -r, but in some cases event that might not work. This Mac OS X hints discussion has several fixes: Script to delete huge sparsebundle images.

They include:

• AppleScript invocation of rm -rv
• unix commands including use of bands, xargs, head, find and so on.

The article includes related references

• osx - OS X - argument list too long for rm -rf *, 4000 files - Super User
• Perl to the rescue: case study of deleting a large directory | Randal L. Schwartz [blogs.perl.org] (millions of files to delete - some good discussion there too)

Fixing Outlook's Ctrl-D usability bug - with AutoHotkey

The letter S is next to the letter D - on my keyboard anyway.

That means when my lifelong q10sec Ctrl-S twitch hits, there's a 1/200 chance I hit Ctrl-D instead.

In Outlook Ctrl-S saves an email, but Ctrl-D deletes it - without warning. In the midst of my work my email vanishes.

This is annoying, but I usually catch it. I make a trip to the Deleted Items folder and drag the email back to Drafts. The other day, however, I was multitasking and didn't notice I'd deleted my email. It wasn't in the UI, it wasn't in Drafts, I probably assumed I'd sent it. Later I emptied my Deleted Items folder (because, by default, Microsoft's slightly daft Search tool includes Outlook Deleted Items, I keep that folder empty).

The email was gone. Minor panic ensued the next day. Fortunately I realized the email was lost, so I responded with an apology rather than being obnoxious.

I resolved to fix the bug. I found very little on the web, really just this one unanswered question: Ctrl-D del adjacent to Ctrl-S save. Either this afflicts very few people, or most don't realize what's happening.

So I asked on our corporate social network (Yammer) and a colleague gave me the answer. He wrote me an AutoHotkey macro that swallows the Ctrl-D key when Outlook is running.

;-------------------------------------------------------------------------------------
#ifwinactive ahk_class rctrl_renwnd32
^d::
return

It works!

Since I grew up on TSRs (if you don't know what that means then you are blessed in more ways than one) so I try to avoid system hacks like this, but Outlook's Ctrl-D bug has broken me. Since I've signed up, I now need to find other ways to use this utility...

Pinboard and IFTTT - blog, task, share

Despite concerns about too many moving parts, I'm still using Reeder, Pinboard and IFTTT as my core information process workflow. I just added two more IFTT actions based on single character Pinboard tags. The current set is:

• s: Reeder/Pinboard shares with the tag s are "shared" via IFTTT to Twitter and WordPress (it's a public tag, so there's a Pinboard feed too).
• t: Reeder/Pinboard shares with the tag t generate a Gmail email to a 'secret' toodledo address that creates a task. This is most often a f/u task on a product I'm interested in.
• b: Reeder/Pinboard shares with the tag b generate a Gmail email to a 'secret' toodledo address that creates a draft Blogger post. (I prefer this to IFTT's limited Blogger channel). These are subjects I would like to write about.

I enter the one letter space separated tags when I share to Pinboard from Reeder.app for iOS and OS X, or from Google Reader. Many posts have two or three tags.

Password security: what if your desktop were stolen?

For various reasons, in addition to using 1Password (always encrypted), I keep a comprehensive set of family user credentials in a FileMaker database that began life over 15 years ago.

I need it to be accessible to Emily or my estate, particularly if I'm dead or incapacitated. I don't like making it too accessible though; recently a neighbors home was vandalized and some computers were taken. Losing hardware is always a problem, even if you have homeowner's insurance and good backups [1], but losing an unencrypted comprehensive password file is a bigger problem.

On the other hand, I don't need to defend against the NSA. The chances is low that anyone stealing my Mac would even bother to Google ways to browse the file system, or would know how to use FileMaker. It's even less likely that they'd scrape deleted or cache data. I just need need decent security.

Whole disk encryption would be more than decent [4], but I'm running Snowie [2] on the main machine. So I do that just on my MacBook Air. Instead this is what I do for the password file and for similar data:

• Used Disk Utility to create a 500 MB 256bit encrypted sparseimage in Users/Shared and moved my files to it. These images ignore permissions by default, that's what I want.
• I set permissions on both the sparseimage file and the mounted disk image to give r/w to Emily and I and nothing to anyone else. (You can't set permissions for the mounted image on the sidebar, you have to navigate the Finder to view the mount)
• In both of our user accounts I mounted the image and allowed OS X to store the password in the user keychain (so anyone knowing our login pw can get it).
• In both of our user accounts I added the image to the User Accounts login items.

With these settings every user can see a folder called 'confidential' (hint) but only Emily and I can open it.

 

It's just enough security so that if our home server were stolen I'd be able to revise our passwords over days rather than hours. [3]

 

-fn-

[1] I'm a believer in using two relatively unrelated techniques to do at least daily backup. For me thats Carbon Copy Cloner and Time Capsule/Time Machine with offsite rotation of 3 disks containing clones of varying age.

[2] Not only is Lion a memory/perfomance slug, it's also tied to iCloud -- which has been a bit of a disaster for me. I'm hoping to skip to Mountain Lion; maybe I'll get some features to go with the bloatware and the iCloud malware.

[3] The way I configure CCC deleted files are kept in archives. So after moving the database to the disk image I deleted it on CCC. That's not too bad a risk however because my clones are stored on an encrypted image (for offsite backup). So even if they take my backup disk they won't get much of benefit. If thieves took my Time Capsule I'd be in trouble with copies of the unencrypted file. So I opened TM and told it to delete ALL backups of the file (made me nervous to do that!). So I feel relatively covered, at least to the level of a typical burglar/vandal [4]  But wouldn't Time Machine backup the data to an unencrypted store anyway? Maybe a disk image is a better idea?

Update 7/18/2012: There's an odd permissions bug with creating documents on the shared image that limits this somewhat. Regardless of permissions on either the disk image or the mounted disk, whichever user mounts the image first has the "most" access. That is both users can open and edit existing documents/files, but only the first user to mount the image can create new documents. The second user gets this error message:

The operation can't be completed because you don't have permission to access some of the items.

This happens even though the image is configured to ignore permissions with both journaled and non-journaled MacOS formatted images and with sparseimage and standard image.

The workaround, paradoxically, is to turn off "ignore permissions" for the mounted image. With permissions enabled both users can read and write to the mounted image.

 

Don't try converting a MacBook Core Duo to Lion

We converted our 2006 MacBook Core 2 Duo dual USB to Lion a few weeks ago.

Don't bother; the MacBook isn't up to it. It's too slow.

I suspect more memory or an SSD drive would help, but, really, six years is a long time for a laptop.

I think we're due for a new machine. When we get one I'll wipe this machine and revert it to Snow Leopard. That would mean no more iCloud, which is, at this point, a feature.

Incidentally, it has the flaking plastic border problem that's common with older plastic MacBooks. It started doing it four years ago and has been stable since; we use scotch tape to close the gaps. A friend tells me Apple will replace the broken plastic for free, even on quite old devices. The hassle isn't worth it for us, but it would be nice if that were true.

The Outlook 2007 blocked sender list import file function is flaky: how to fix it.

This summary is not available. Please click here to view the post.

Hotel iCloud

"… Welcome to the Hotel California
Such a lovely place (Such a lovely place)
Such a lovely face
Plenty of room at the Hotel California

"We are programmed to receive.
You can check-out any time you like,
But you can never leave…"

Eagles, Hotel California

I'm in recovery from a tedious multi-update post on my failed iCloud transition, and still digesting all I've learned. I'd like to pass on one conclusion.

iCloud really is different from MobileMe. 

MobileMe was a peer-to-peer synchronization service. My data lived on my machines, as well as on MobileMe.

iCloud is a server solution with local caching. The data moves to iCloud, it is no longer On My Mac.

Yes, I know it seems to be in the usual AddressBook folder, but as I discovered the real source of truth is remote. When there is a clash of opinions on the state of data, iCloud wins. If your data goes bad, you need to fix things on iCloud. Good luck with that.

Perhaps even more importantly, this is a one way trip. Yes, if you know the ins and outs you can copy Contacts back to your Mac and remove them from iCloud. You cannot, however, get your Group relationships back. That data has checked into Hotel iCloud.

With iCloud, OS X Mail and Address Book/Contacts are much more like Google Mail/Contacts -- which also has an offline mode. Apple's data lock is stronger than ever, and I have a very bad feeling about the next version of iTunes.

Chrome broken in OS X due to certificate change: Issue 108238 - Canary asks for access to every keychain item

I got hit with this bug on my Air: Issue 108238 - chromium - Canary asks for access to every keychain item.

We had to change the certificate we use to sign Chrome.

Transition code was included in (stable channel) Chrome 19.0.1084.53 and 19.0.1084.54 to migrate your Keychain items. If you’re on the stable channel but did not launch one of these two versions, upon update to 19.0.1084.56, your Keychain items were “lost” and need to be reauthorized manually.

There is a mitigation in place for 19.0.1084.56 and newer: when Chrome automatically updates itself, the updater will perform the transition. Unfortunately, this is only effective if the updater has permission to do this, and that’s generally only the case when you’re on a “user ticket,” and not when “Set Up Automatic Updates for All Users” has been used.

The correct thing to do when such a dialog appears is to click “Always Allow”. We don’t anticipate any further Keychain changes like this for a very long time, and we hope that in the future, if we do need to make such changes, to have a longer time in which to let the transition run.

It's an interaction bug between Chrome Sync, OS X Keychain, Lion, and a certificate signing change with a short overlap period:

The at-update reauthorization is intended to handle the reauthorization for users who rarely restart Chrome and might miss out on the at-launch step during the window where Chrome is signed by the old certificate but has the new reauthorization code in place.

It hit a machine I rarely use where Chrome was probably running, so when I restarted it the old cert had expired.

There's no plausible workaround, the Google support forum thread on this topic has quite a few deleted messages. The fix seems to be to switch to Safari or Firefox. In a case like mine, where there are hundreds or even thousands of passwords in the Keychain, tapping "Always Allow" repeatedly can take a very long time. Chrome is unusable for now on my Air.

I'm curious as to why Google made this fairly urgent change to their Chrome code signing cert. My Google searches didn't turn anything up. I assume it's related to a significant security breach.  The silence is curious.

Update: Since this was not my main machine I applied the drastic "fix" of deleting my entire #$#$ user login keychain because Google was supposed to 'regenerate it'. Which it sort of did, but the process turned into a slog of bugs and issues far too tedious to list (including a password displayed in a font that rendered a zero as an o. Even after the Google sync was done I am still out many passwords on that machine that are used by other apps. I really don't recommend this approach. It's easier to switch to Firefox or Safari.

So far tonight Google and Apple are causing me similar amounts of pain.

iCloud transition went as expected - disastrously

I was sure Apple would screw up my iCloud transition.

So I did everyone else on our family account first. I backed up my Contacts. I did a final MobileMe sync on all machines. I logged out of MobileMe on all but my Lion MacBook Air (fully updated).

Then I migrated. All seemed well until the Preference Pane opened. And crashed.

Address Book synchronization failed.

My Address Book now has one entry instead of 1,824 entries.

Apple, your competence is not complete.

Happily, I have backups.

I ended up disconnecting the Air from iCloud and moving my Contacts from my iPhone.

Did I mention that iCloud doesn't render well with Chrome?

And Apple thinks they can do Maps?

Update: 

• My old MacBook, barely able to run Lion, managed to synchronize its Address Book correctly (at least counts and samples match) to iCloud.
• I signed out of iCloud completely on the MacBook Air and logged out. When I logged in again my Address Book entries were back. Looks like they weren't removed, rather the sync failure left Address Book in an unhappy state. I logged out of iCloud completely then logged back in and that restarted iCloud synchronization. It appeared to complete successfully.
• You can't enter an admin un/pw to active Find My Mac, you have to activate it from an admin account (so same iCloud settings have to be active in two accounts.)
• On the plus side, the web version of Contacts is pretty good in Safari. I'll be using that until I convert my iMac to (yech) Lion. I may see if I can wait long enough to go to Mountain Lion directly.

Update 6/27/2012

• There are 1831 contacts on the iCloud site, 1828 on my MacBook Air and MacBook, 1831 in Snow Leopard (no longer synchronizing but at this point should still be in sync) and 1829 on my iPhone. I tried removing all contacts from my iPhone and then restoring from iCloud, that gave me 1828. So there are 3 on iCloud that aren't anywhere else. 
• My MacBook Air (the one that crashed on initial configuration) was hanging on iCloud login, so I tried restarting it in safe mode then tried connecting to iCloud again. I got a curious error message, it seemed as though the machine was in a mixed state part way between MobileMe and iCloud state. From the 'Mail, Contacts and Calendars'  Preference Pane I removed a residual mail account that seemed a 'mix' of MobileMe and iCloud. Then I learned that I was entering an incorrect password, which perhaps didn't fail as it should have because of a partial MobileMe connection. After this I was able to login to MobileMe easily, and I got a System Preferences Directory Services configuration modification notice. Subsequently my Address Book slowly updated without any UI indication of ongoing synchronization. Although my MacBook is now behaving better, it still has 1828 Contacts (Cards) compared to 1831 in iCloud. So those numbers aren't changing.
• After the initial synchronization, I realized updates were not propagating to my OS X Address Books. I unsubscribed from iCloud Contacts, removed all contacts, and resubscribe. At the moment they are updating. (See Apple's very popular iCloud Contacts troubleshooting article for more on this.)
• I'm trying Contact Cleaner to look for problems on one of my Lion machines.
• Looks like I have another update for my Synchronization is Hell blog post. MobileMe, over the past few months at least, was actually working reasonably well.
• Apple Discussion on this post.

Update 6/27/2012: Disaster diagnosed

It appears that each time iCloud synchronizes it duplicates the Notes in many of my Contacts. MobileMe and Snow Leopard Address Book didn't do this.

I've abandoned iCloud. I'm now synchronizing the same way I did my old PalmPilot -- by cable. It works quite well. After some Contact Cleanup I've 1821 contacts on Snow Leopard and 1821 on my phone.

I then copied my Snowie Address Book archive to a thumb drive and found that Lion would import it. Now I have 1821 contacts on my machines. These sync to iCloud still, I'll see how badly that goes and I'll experiment with dropbox distribution.

Update 6/27/2012: I find the extra 3, and a theory on the iCloud bug

I figured out how to find the 3 extra iCloud Contacts. I deleted all the contacts from an Address Book on one of my machines, then I looked at what was left in iCloud.com. There were 3 duplicate addresses left, all dupes of a single good address. So I removed them.

My theory on this bug is that it's one of the oldest bugs in the Mac world -- the CR/LF vs. CR vs. LF bug. Many of the entries in my Address Book started in Outlook on Windows, and moved over via MobileMe Windows. They probably have a line termination issue. This was ok with MobileMe and with Snow Leopard, but iCloud can't handle it.

I'm reasonably sure this is part of the problem, because when I move my Contacts to iCloud I see extra line spaces, even when the notes aren't duplicated.

As I wrote in my Apple Discussion post:

I think I've figured out the bug.

It's the oldest bug in the book -- line termination.

My Address Book contains contacts that started out in Outlook, then went via MobileMe Windows Control Panel to live in MobileMe, and then into my desktop machine.

MobileMe could handle the LF (Mac) vs CRLF (Windows) difference. So can Address Book in Lion and in Snow Leopard.

iCloud can't.

I'm seeing extra line spaces in all of my contacts with notes that started out life on Windows.

Each time iCloud does its sync with desktop it finds a mismatch -- because OS X and iCloud handle CR and CR/LF differently.

So it replicates the Note.

The Note grows eternally, until everything crumps.

Nasty bug! I hope Apple figures this one out soon.

There's an option to specify a text encoding on Address Book import. I'm experimenting with exporting an archive, then reimporting as UTF-8 rather than "automatic".

Additionally, I figured a way to crash iCloud Contacts reliably. Just try deleting 1800 contacts from the web UI.

Update 6/29/12: Ongoing experiments

• I tried importing an archive into Address Book using Western (Mac OS Roman) text encoding, but that didn't help.
• When I export a VCF file for the contacts that show extra line spaces, the vCard shows \n\n\n\n where, based on the Notes display, I'd expect \n\n.
• I installed a trial version of Bento, opened a contact and inspected the text in TextWrangler then pasted it back in from TW. I also migrated to a Bento Contacts view. Then I exported a vCard again and this time it only showed \n\n. The Card synchronized to iCloud and the extra line feeds disappeared there. 
• I get the same results just using Address Book, pasting into Text Wranger, then pasting back into Address Book. The round trip through Text Wrangler solves the EOL problem. I wonder if that's something I could write an Apple script to do.
• Mac OS X hints has the best information on doing this conversion. From 2003.
• I found a Service that I could enable in Service Preferences called "Macintosh Line Endings". It may have come with Devon Technologies WordService. Highlight Address Book then applying this service removed the Windows CRLF pairs and left Mac OS X LF line feeds. It fixed the problem. So if I can figure a way to walk my Contact Notes and apply this Service the problem will be fixed.
• iCloud updates are unpredictable. Sometimes fairly fast, sometimes I give up.

Update 6/30/2012: vCard options

i've decided there's an Uncanny Valley for software; products that look almost like real software, but don't quite make it. Bento and Automator are like that.

Bento didn't turn out to be very useful, but it showed me there were 1000s of notes to clean up. Too many to brute force, even after creating kb shortcuts for the "Macintosh Line Ending" WordService.

I looked at scripting Bento, but it's not scriptable. Address Book is very scriptable, but there didn't seem to be a way to walk through a list of addresses applying the Line Ending utility to each note in turn.

vCard is more promising though. I can use TextWrangler to change \n\n to \n and remove the dangling \n at the end of many vCard Note strings. On import then Address Book will turn the \n into a LF -- no more CRLF problem.

As expected, however, Apple's Address Book import is unable to process the %^$#! CATEGORY strings that Apple Address Book Export $^@%# writes! Grrr. #$#@!$% Apple.

I tried using the 'merge' function, but, like iCloud, that merely replicated the Notes. I'm looking at ways to put the CATEGORY strings into the Notes field (Append), so perhaps I can use smart groups to recreate my Group assignments. Maybe Stefan Kelley'rs Export Address Book utility will help. I need to think on this one.

Update 7/1/2012: Hex Editor won't help

It occurred to me that I could use a hex editor to try to fix the problem. Alas, .abbu is just a Pacakage, there's no single file I can try editing.

So I think I'll be doing the vCard export/import and I'm experimenting with ways to speed group membership recreation on import. Some useful references:

• vCard import group 9/2008
• Difficulty importing Categories into 2/2009
• HT4489 Importing vCards to iCloud from... 5/2012
• VCard grayed out when attempting to... 4/2012
• Custom fields in Mac OS X Address Book « Diaries, Triumphs, Failures, and Rants
• Address Book Power Tips | Mac|Life

Update 7/2/2012: Exploring more options, with a focus on somehow retaining Group membership

Lion's Address Book 6.0 has more import/export behaviors, including restoring 10.4's option-export individual vCard. That makes its continued inability import Categories as groups, or import "custom fields", all the more exasperating.

• Address Book 6.x: Resolve dulicate cards when importing cards: After first review can apply to all.
• Address Book 6.x: Import contacts
• Address Book 6.x: Export and import vCards

And LDIF is perhaps an option ...

• Importing an ldif address book?

Some potentially useful tips ...

• 5 handy Address Book tips -- Reality Distortion: Macs, Mac OS X, and Apple stuff
• Mac Address Book tips: 5 more handy hints
• Hawk Wings - Plug-ins for Address Book (and other) (there used to be an advanced importer plugin, but it appears to be gone - Address Book importer)

I suspect vCard import/export includes the Department field. I don't often use Department, so I could probably copy the Category strings into Department, and then use Smart Groups to create subsets from which to create Groups.

Update 7/3/2012: vCard export/import won't work, but a hybrid hack looks promising

Many fields are omitted from Address Book's vCard option. So I can't do clean up and restore from there, I'd lose too much. Worse, Address Book Import is also quite weak. Instead on a test machine running Lion and Address Book 6.0 I tried this convoluted process on a copy of my Snow Leopard Address Book.

1. I selected all contacts and exported Group vCard (1820 contacts)
2. I used Bento to delete all Notes (select column, hit delete, exit Bento)
3. Working on the vCard export, I used TextWrangler to replace \n\n with \n everywhere and remove a trailing \n at the end of the NOTES string. This removed some paragraph definitions but I didn't mind that. Simpler to do.
4. I then imported the vCard file back into Address Book. Address Book said it had found 1818 duplicates and merged those in; I ended up with 1822 Cards. So there were two duplicates.

The results seemed good, I no longer found duplicate notes. The Address Book synchronized to iCloud far more quickly than before, and iCloud also appeared fine and without duplicate Notes.

I considered using Address Book merge to resolve the two duplicate Cards, but it is too aggressive and would merge addresses I wish to keep separate. So I used Contacts Cleaner. It found 3 duplicates and a (new) 5-6 duplicate addresses. Those were quickly resolved and I ended up with 1819 cards, but iCloud did not update; it still showed 1822 Cards. I suspect that because Contact Cleaner is altering records "behind Address Book's back" that it doesn't know what to sync to iCloud.

Unfortunately Contacts Cleaner still refers to MobileMe in the UI and the App Store support link goes to a 1 page contact-free FAQ that is also the entire product documentation.

I suspect the only way to force iCloud updating is one outlined for a different product; Address Book Cleaner. It requires deletion of all contacts on iCloud to enable synchronization. Since I've shown previously that Safari will crash if you try to delete 1,800 Contacts this way, you have to do it using Address Book and some Archive shuffling (if you are reading this you are enough of a geek to figure that one out on your own).

This approach, though inelegant, appears to work. I'll do some more testing before I try it on my real Address Book. (I haven't tested yet to see if Snowie will open a Lion Address Book. I suspect not, but the process I used will probably work on Snowie. As long as one backups up the Archive, it's easy to restore an Address Book.

*OOPS* Late update: Even if I delete all entries, iCloud/Address Book now has large numbers of replicated Categories/Groups. This is quite comical!

Update 7/4/2012: Every iCloud client Address Book irrevocably poisoned by metastatic Groups

This was when I gave up. I'd solved my Notes duplication issue, and I'd gotten the Contacts Cleaned. I'd even figured out how to clear out iCloud (I thought!) and how to push changes from one machine to others. Then I noticed this:

That's a portion of the hundreds of replications of the Personal Group as shown in iCloud.com. I see the same list in every Address Book synchronized with this iCloud account, even though all the Contacts/Cards have been removed.

Wait, it gets worse. On these test machines I tried removing the entire contents of Library/Application Support/AddressBook. (I found I had to quite LaunchBar to empty the trash -- it holds onto this data. Which means LaunchBar is a minor suspect in this crime.) It didn't work; they came back (I could see them being added to Library/Application Support/AddressBook/Metadata as Address Book launched).  They are immortal and they're spreading to all of my machines. These iCloud Groups are the metastatic cancer of OS X Lion.

How can this be? Clearly they don't only live in Address Book. I suspect that after one syncs with iCloud, regardless of whatever one does later including signing out of iCloud, the group definition lives in iCloud. On a disconnected machine it's somewhere in an iCloud offline cache, and they get synchronized into Address Book. To delete them one would need to clean out the iCloud local data store. There's no known way to do that (no iCloud equivalent of  reset SyncServices).), just as, incidentally, there's no way to delete an iCloud.com account. 

Of course there are still options. Given enough time, I could delete them one at a time in Address book.app or iCloud Contacts. Maybe I could track down the cache and remove it. Maybe I could see if creating a different iCloud account removed them.

I could do a lot of things ... but ... really, I'm done. I've established to my own satisfaction that iCloud and Lion are broken.

I'm done. My Address Book is in Snow Leopard, and it works -- even with the EOL bug. I sync now to my iPhone via the iTunes cable. Maybe I'll sync them to a special Google Apps account as well -- using iTunes. One that's setup just to provide access to my Contacts.

I'll try again when Mountain Lion is out. (Update: Mountain Lion helped a bit, but AppleScript was the fix. See below.)

Update 7/4/2012b: More broken iCloud (I can't resist).

I have a mac.com Apple ID. For kicks I tried creating an iCloud account from it. Turns out you can -- except iCloud hangs. So you get part of an account I think. Did Apple outsource all their iCloud development? I mean, this thing is made of cheap crystal.

Also, when I did this OS X told me that this new account wasn't my primary iCloud account. So you might ask - how does one change or configure the primary iCloud account? It's not clear, but in some experimentation involving removing accounts, logging in and out, etc. I think I did.

One more discovery. In the iCloud world data really lives on the Cloud. My local data seems like a cached version of Cloud data. So, once you make the Cloud transition, Apple owns your Contacts. I don't see a way to get them back. I have noticed that the Address Book archives are bound to a .me account, not to my machine.

Update 7/4/2012c: Postscript, in which all 1819 contacts move to a new iCloud account with the EOL problem fixed.

I couldn't quite leave it alone. In further experiments I discovered:

• Sometimes my Address Book 6.x Import option would be grayed out. Deleting everything in AddressBook fixed this. Seems to be a permissions problem related to iCloud.
• It's iCloud that's writing the Groups back into Address Book. Indeed, I discovered when iCloud is active it's impossible to truly delete all files from AddressBook. It grabs hold of some on startup. When all iCloud accounts are moved both from System Preferences and Address Book Preferences you can delete the files.
• You can drag and drop Contacts from "iCloud" to "On My Mac" and create a local set -- but you can't move or copy Groups. Groups live only on iCloud. Makes you wonder what Apple wants to do with groups.
• I was able to make my mac.com account my Primary iCloud account. It told me if I wanted email to work I'd have to pick a me.com email address. I ignored that.
• I deleted everything from iCloud on my test Mac (I've figured out how to do that). Then I copied my Address Book archive from Snowie. Then I went through the  EOL cleanup process I described above. Then I set up an iCloud connection to my mac.com identity. This time it moved everything over (so what I see on my Mac is only a cached version). With the EOL bug fixed I had no duplicate notes and no metastatic Groups, and the same count on both.

And now I wait for Mountain Lion. At least I know the EOL fix works; I'll be sure to test that before I try Mountain Lion. At some point I may take my machine to an Apple "Genius" and see if they can get the original iCloud account cleaned up.

Update 7/26/2012: I fixed the Group Replication problem - deleted the Groups with AppleScript.

Update 7/27/2012: Nigel Garvey of MacScripter writes an AppleScript to fix the line terminations (!)