Showing posts with label Contacts. Show all posts
Showing posts with label Contacts. Show all posts

Sunday, December 17, 2017

Sending multiple Contacts to an iOS user from iCloud Contacts

Found this works

  • Select multiple in iCloud
  • Click gear bottom left to export VCF
  • Email VCF to iOS device
  • Click on attachment - get option to Add all members

I’m sure there are other means. Note this does seem to require a computer. I imagine works same from Mail.macos.

Saturday, December 02, 2017

Spotlight bugs in iOS 11.2: Empty contact on search.

I think the spotlight/contact bug was introduced in 11.1. It’s not fixed in 11.2.

If I search in spotlight for a Contact name, or search on Contacts field of phone app, I get an empty (Null) Contact as a result. It has no content.

Turning Search (spotlight) off then on for Contacts restores search, but only transiently. I have 2,162 Contact Cards, I did find one that was empty and deleted it.

I’ve now removed all Contacts from my phone (iCloud) and restored them. I’ll see if that works.

Sunday, September 03, 2017

Annals of iOS inconsistency: Contacts vs Notes vs Reminders - backup and sharing

https://www.icloud.com/#settings currently shows an “Advanced” subsection for restoring Contacts. It provides options to restore an iCloud data set “archive” from iCloud (not to be confused with restoring an entire iOS device backup):

Screen Shot 2017 09 03 at 11 38 36 AM

Notes aren’t on the list though. They have their own note-specific backup restore option, but it’s at the level of an individual note and there’s no version restore, only the ability to undo a delete for 30 days by restoring a Note from “recently deleted”. (BTW, if you Share a Note only the Owner can “delete” — but anyone with Edit privileges can remove all content — and since there’s no version undo that means anyone who can edit a Note can delete it without a recovery option.)

Screen Shot 2017 09 03 at 11 43 45 AM

Sharing is another area of odd inconsistency. Notes must be shared one at time, but multiple Reminders can belong to a set of People.

I’d like to see Notes add Google-style Note-specific version save/restore and share by container (folder) as well as Note, but there’s no rumor of that in iOS 11. I’d pay for a third party solution for iCloud, similar to what CloudPull does for Google App docs, but I fear the demand is too small (for example). An Apple iCloud Drive folder view of Notes [1] would be a big help; I’d then be able to restore an individual Note from a Time Machine or Carbon Copy Cloner backup …

Anyone have an AppleScript to create a local daily snapshot of Notes? (There is this, but in Sierra Apple omitted AppleScript dictionary support for PDF creation).

The world moves in unexpected ways. We seem to be converging on a form of backup that’s a regression for people like me, but a big improvement for most. There’s probably some kind of futurist principle there — the good-enough mass solution will drive out the elite ideal …

- fn -

[1] The main reason I’m still on Simplenote is that nvAlt on my Mac maintains a synchronized file store that works just like this. Perfect data freedom — but almost nobody appreciates this …

[2] As of Sierra at least some parts of Notes are in /Users/[username]/Library/Group Containers/group.com.apple.notes. This location has changed a few times. Note content is distributed between media files (PDF, etc) and text in a sqlite database, so recreating an individual Note document as, say, an RTF file, is a non-trivial task. For example (sqlite browser):

Screen Shot 2017 09 03 at 12 20 57 PM

I suppose Time Machine backups of this folder might be a kind of ‘restore all notes’ option, but restoring a version of an individual Note would be tricky…. (There’s something deep here about the ways in which we assemble bits to create something our brains perceive and our tools manipulate, but it’s beyond my ken. Once upon a time a BYTE article would have traced the roots of the Notes sqlite store back to database file systems of the 1980s…)

Wednesday, April 05, 2017

Migrate Contacts from Outlook/Exchange server to OS X Contacts

I last wrote about migrating contacts from Outlook/Exchange server to OS X in 2011.

Back then one of the issues with migration then was that Exchange uses X400 format email addresses. One way to make Outlook convert to internet is to drag contacts into an email — that produces .vcf files (undocumented) with internet email addresses. In 2011 I wrote that dragging into a local PST did the same thing. Not sure that still works.

Here’s what I did recently …

  1. Drag and drop from Outlook/Exchange into email (undocumented export)
  2. From email save to desktop.
  3. Drag and drop the .vcf files into OS X Contacts (undocumented import.

From OS X contacts they can go to iCloud as usual.

Incidentally, i couldn’t see any way to readily import a set of single contact .vcf files into Outlook 2016. It only wanted to import them one a time. Drag and drop of the files into Outlook resulted simply in Outlook opening each one until Windows 10 die.

Wednesday, June 10, 2015

Mavericks bug: enabling iCloud Keychain duplicates Contacts if use .me domain

This is a truly weird bug.

If I enabled iCloud Keychain sync in Mavericks all of my Contacts would duplicate. One set would show up with the header of 'iCloud', the second (new) set would have the heading 'iCloud jf" where jf are my (true) initials.

The two sets appeared truly identical. Edits to one appeared in another. My iPhone Contacts were not affected.

If I turned off iCloud Keychain sync I could unclick Contacts in iCloud preferences and watch one set disappear. Then it would check itself back on. I unchecked again and a second set disappeared. Then I rechecked it and the first set of Contacts returned Lastly I reenabled iCloud Keychain and the second set of Contacts reappeared.

I think this bug is related in part to the .me to .iCloud transition. I say that because my iCloud account used the old '.me' domain. It's supposed to be interoperable with iCloud, but I guess it really isn't. I signed out of everything on my iMac and left iCloud, then restored using the '.icloud' domain. Now I have one set of Contacts and I have Keychain sync enabled.

Curiously, even after the fix, the Accounts setting in Contacts shows the .me suffix. I can't edit that, it claims to be inheriting from Preferences. iCloud support in Mavericks is a real hack. I probably can't wait for El Cap, will need to accept Yellowstone after next bugfix release.

Wednesday, May 20, 2015

How I moved my daughters iOS Notes from school to personal iCloud account

Please don’t put data into iOS Notes. Really, it’s quite horrid [1]

If you do you may find it’s stuck there. We ran into this when my daughter’s school iPad had to be wiped and we needed to rescue Notes that were only stored on her iPad (school iCloud account doesn’t support Notes, neither does school Google Apps).

I did this:

  1. Used AirDrop to move notes one at a time from her iPad to my iPhone 6 (her 4s doesn’t support AirDrop).
  2. Since my iPhone 6 is configured to use iCloud, each time a note arrived via AirDrop it went via iCloud to Notes.app on Mavericks [2]
  3. On Mavericks the notes appeared in my iCloud account. I added my daughters Google Account to my Mavericks User account. I could then select ALL notes (yay) and drag and drop them to the Google account.
  4. Then I created an OS X User account and associated it with her personal iCloud Account and added her personal Google Account. Then I did the drag and drop from her Google to her iCloud. 
I did something similar with her school Contacts. Interestingly the account drag and drop behaves differently — Notes are moved, but Contacts are copied.
 
- fn -

[1] I took a look at the folder where Apple stores Notes.app data: ~/Library/Containers/com.apple.Notes/Data/Library/Notes. (Yes, the organization is bizarre). The data is in NotesV2.storedata-wal. I inspected the binary file in Mavericks [2] and found it contains text of Notes I deleted long ago. So if you had sensitive data in Notes.app deleting it won’t remove it from your Mac. It seems the file is never purged.

More — in Mavericks, it only looks like you can drag and drop notes to the desktop. It doesn’t actually work.

More — Notes can sort of hold images and rich text, in some Apple OS but not in others. Definitely not in Google IMAP.

More — Notes was implemented unsung an oddball IMAP hack. It’s like nothing else.

More — Like Contacts Notes can have Groups / Folders in OS X, but in iOS you can’t do anything with these.

There’s still more…

[2] Yeah, I’m still on Mavericks. Yosemite has … issues. I’m waiting for a fix for the crazy network problem.

Thursday, November 13, 2014

Force Chrome to remember iCloud password - how I ended up with LastPass

I don’t use iCloud for much - Apple’s cloud functionality is almost as lacking as its reliability. Our family has used Google Calendar for 7 years [1], I use Simplenote and nvAlt [3] as an information store, Gmail for email [4] and Toodledo/Todo.app for tasks. [5]

That leaves Contacts in Apple’s badlands — they’re too tightly coupled to iOS and OS X to readily migrate. So Contacts are the only bit of iCloud I use; I have to admit that they have been relatively reliable.

Hang on — this does get to the part about forcing Chrome to remember an iCloud password. But first I need a bit of setup. I have to explain something about corporate life and personal data.

Fifteen years ago many employees mixed their personal and corporate data on business laptops and workstations. It wasn’t unusual to use a single email for both work and personal use. Ever since then the two worlds have been dividing - driven by legal and security concerns. Even thumb drives are encrypted on insertion now; data on the increasingly locked down corporate laptop belongs to the corporation.

Which is fine for email and work documents - they should belong to my employer. Contacts though — they’re a problem. They don’t divide neatly between work and personal — and my work Contacts are pretty important for my future employment and family food. So, when it came time to decide where my Contacts should live, I moved them entirely into the personal sphere.

Which is why I need to use a corporate browser (Chrome [6]) to access iCloud — that’s where all of my Contacts live. I need ‘em when I work. 

Ok, so we’ve established I need to use Chrome to access iCloud. Now the problem — it makes me enter my password way too often. And my passwords aren’t easy to type or remember. There are extensions that once forced Chrome to store this password, but they don’t work any more.

So today I broke down. LastPass has a freemium model for online credential storage; the web app and Chrome extension are free. (LastPass charges for mobile services.) Unlike 1Password, which I use on iOS and OS X, there’s no need to buy a Windows client — and I don’t want to put all my credentials in the Cloud anyway. So I signed up for free LastPass, and created an account with a single stored credential - my iCloud ID and password.

It works fine. So one of my longstanding annoyances has been fixed — I can quickly bring up iCloud Contacts.

- fn - 

[1] Calendars 5.app is essential for the Google Calendar power user — we have it on every phone [2]. My native iOS Calendar syncs to my corporate calendar, Calendars 5 reads the iOS calendar database so it appears inline with my other 17 calendars, including 1 for each family member and one ‘all family’ calendar. Our family grandfathered into free Google Apps accounts, but if we didn’t I’d probably pay for the business service. Free has been nice though.

[2] It’s $7 a user. Since it’s not funded by in app purchases I believe iOS family sharing would allow one purchase to support five users. If it did use in app purchases that wouldn’t work. FWIW we still share a single Apple Store only AppleID, I use a different AppleID with iCloud.

[3] nvAlt is in maintenance mode for now — but so is Notational Velocity. Brett has a commercial replacement in the queue, as of today “it’s amazing and will probably be released”.

[4] Google’s broke Gmail usability with their last UI redo. I use Mail.app on iOS and sometimes Airmail on Mac, but mostly I gnash my teeth and weep and use Gmail.

[5] Nobody would do this by choice. Is a legacy choice.  Works and I hate to change things that work.

[6] The corporate standard is IE 9 — thanks to legacy apps. So IE for corporate apps, Chrome elsewhere.

Wednesday, May 08, 2013

Contaqs.app - A contacts.app replacement - with a flaw since fixed

I have about 2,200 Contacts, and if they could speak they'd all complain about iOS Contacts.app. The bizarre Groups implementation, the limited search options, the crummy search result display ...

Frustrating.

So I've been looking for an alternative. Most of the App Store alternatives failed the smell test, but Clark pointed me to a relatively good one - Contaqs.app. It looked good enough that I bought it.
My initial impression was pretty good. Search results show the company name. You can search on names, company names and phone numbers. You can edit group relationships (!). There's a smart-tag and geolocation function and you can create new tags. Seems fast.

Then I tried searching on my wife and didn't get any hits.

This test explains the problem:
Contact: John Paul Jones
Search in Contacts.app on John Jones: 1
Search in Contaqs.app on John Jones: 0
Search in Contaqs.app for John Pau: 1
The sound you here is my head repeatedly hitting the wall.

Instead of what's called 'starts with word search' the authors of Contaqs.app implemented phrase search. This means that you need to include the middle name if your Contact has one.

I'm sure they had some reason for this, but it wasn't the right choice. If Contaqs.app were to implement 'starts with word search' they'd be a 4-5 star app. As it is I'm tempted to rate them 2 stars, but that's frustration speaking. I'll give 'em 3 for now. I might use it to clean up Group relationships, but it doesn't replace Contacts.app.

Darn it.

Update 6/11/13: They've fixed the search problem. Confusingly they call the new search 'phrase search', but it's starts with word search. The middle name problem is gone. Performance is a bit sluggish on my i5, probably why this feature is only for newer devices. Still, it's a great improvement.

Tuesday, March 12, 2013

Apple's iCloud control panel enables iCloud Contacts within corporate Outlook

iCloud support is limited to relatively new Macs able to at least run Lion. Practically speaking, Mountain Lion.

Apple is kinder to the Windows world. The iCloud Control Panel will run on Outlook 2007 or later on any old OS.

I suppose they don't have the same sales incentives on the Windows platform. In any event, it seems to work very much like the old MobileMe Control Panel, at least when it comes to Contacts. I installed it on a Win 7 laptop running Outlook 2007 that syncs to Exchange Server and I now have full access to my iPhone/Mac Contacts.

It works by creating a new account, separate from the Exchange account. There are few install options -- you have to sync Calendar, Contacts and Reminders (but not Notes). Just like MobileMe [2] (I suspect some shared code). It also creates an IMAP account.

I had some glitches on installation and had to fiddle with restarting both the Control Panel and Outlook 2007, but now it seems stable. I didn't want the IMAP service so I canceled out of the credentials prompt and deleted the IMAP account.

There's no way to turn off Reminders, so I now sync my iPhone reminders in some puzzling ways -- basically via ActiveSync/Exchange for one set of reminders, via this method for the iCloud set. Weird.

Contacts is what I was interested in and it seems to work. Since Outlook allows only one Group per Contact, and iCloud allows many [1] there are potential problems related to Group assignment. In the case of MobileMe this didn't seem to break anything.

It's great to have all my Contacts at hand, and to able to quickly add to them. As an extra bennie, I get to use Outlook's Contact views. They are old and complex, but they are far more powerful than anything Apple gives me.

[1] Group relationships are kind of messed up between iOS and OS X, but Apple has bigger problems.
[2] At one time MobileMe Control Panel could work with Outlook/Exchange, but then it coulnd't -- which made things hard. This version can, perhaps because it creates a new local account.

Saturday, February 16, 2013

Pre-Lion Macs: Go iCloud or Google?

It's been a while since I reviewed the PIM world [1], but a post on CardDav sync between MacOS Mountain Lion (10.8) and Google Contacts brought me up to speed. In particular, I thought about the special case of a pre-Lion Mac user (Snow Leopard and Leopard especially - 10.6 and 10.5). I'll assume for the purposes of this post that every Mac users also has an iOS device.[2]

Every Mac/iOS user has to choose between Google and Apple for Calendar and Contacts (and thus Email). On Mountain Lion (or Lion) the choice is more difficult; those operating systems can work directly with Apple's iCloud. So it's not a slam dunk decision [3].

On pre-Lion Macs however, the choice is fairly easy. Older Macs can't use iCloud sync or data services at all; they can only access the web interface to iCloud Contacts and Calendars. On the other hand, even old Mail.app can use Google's (troublesome) IMAP services, and old versions of iCal can more or less interoperate with Google Calendar. Google's web interface to Contacts is crude, but their Calendar web services are much more robust than iCloud calendar. If there were any doubt, Gmail and Google Docs would seal the deal [7].

So, if you're an iOS/Snow Leopard user you can (mercifully) forget about iCloud. You need to use Google; for maximal Google compatibility SL users can even install Chrome [5].

You still have one choice though. You can:

  • pay for Google Apps ($50/year) and use ActiveSync [4] on your iPhone and web UI on your Mac with optional user of iCal (CalDAV) and Mail (IMAP) on the desktop [6]
  • use ad-supported Google Services and use web UI on Mac and the standard sync services for iPhone or Mac: CalDAV (calendar), IMAP (Mail), CardDAV (iPhone, not Desktop alas).

I don't think ActiveSync is necessarily superior to CalDAV/IMAP/CardDAV any longer -- but it is simpler to setup on the iPhone. Still, that's not enough to choose between paid and ad-supported. The main thing you get for your $50/year is freedom from ads and phone support. The last isn't a small thing.

Snow Leopard users used to feel abandoned and left out. No longer! Sometimes lack of choice is a feature. Apple has spurned you, Google awaits with open arms. What are you waiting for?

--

[1] PIM: Personal Information Management. Always includes Calendar and Contacts; since PalmOS usually tasks and notes (or equivalents). In the 90s it was mostly PalmOS and Palm Desktop for me, in 2013 it's a bit more complex:

  • Contacts: iOS/iCloud/MacOS/Win7OutlookExchange via ActiveSync + Google/MacOS via CardDAV
  • Tasks: Todo.app/Toodledo via proprietary + iOS/Win7OutlookExchange via ActiveSync
  • Notes; Simplenote/Notational Velocity
  • Calendar: iOS/Google via ActiveSync + iOS/Win7OutlookExchange via ActiveSync

I've simplified a few of those connections. I think we're starting to hit the boundaries of human cognition.

[2] If you're Mac/Android you have to Go Google. Mac/RIM or Mac/Microsoft -- you're on your own and you're used to being hard core.

[3] I've largely gone Google myself.

[4] I don't think Google offers free versions of Google Apps any longer. I'm from the early generation when it wasn't hard to end up with a half-dozen App/Domain combinations all of which were grandfathered in to ActiveSync support.

[5] Real old Mac users may want to try Camino.

[6] It's not clear to me that Google Apps users can use CardDAV services.

[7] iCloud can still be used for iDevice backup and media services, but trusted friends tell me the backup is unreliable.

Google Contact Sync on Mountain Lion and iOS - CardDAV (Project Contacts 2013)

I'm rather late to this party [6], but around the time Google ended ActiveSync support for unpaid accounts [1] they added vCard 3 (1998) based CardDAV support [3]. Up until then one could sync iOS and OS X mail and Calendars via IMAP and CalDAV, but Contact sync was only via Active Sync. [2]

TUAW has some details and Google's iOS CardDAV instructions are here.

As we all know, synchronization is hell. Even if Apple and Google actually used the same vCard spec (probably vCard 3, 1998), and even if they agreed on newline characters, they still have to deal with distinctions between their internal data models and vCard. Some of those data model gaps are non-computable. [5]. Even thought Google and Apple have similar 'Group' models that relationship metadata is ignored [8].

That said, at least CardDAV is relatively Apple friendly. Most importantly, this standard means, for the very first time I know of [7], there's a somewhat supported way to synchronize Contacts between Google and OS X Contacts and iOS contacts.

I've begun using it on Mountain Lion. Since I use Google two factor authentication I had to use one of their totally-defeats-the-purpose-giant-security-hole-not-single-use-god-i-hate-passwords-humans-are-losing-the-battle alternative passwords. There was some kind of transient authentication glitch but on a second try it worked. After a few minutes I had all 634 of my Google "My Contacts" on my Mac. Because of the long history of my Google/Apple Contact sync efforts (See Also, below) these largely duplicate my Address Book/iCloud contacts -- but Mountain Lion Contacts.app seems to merge the duplicates when I search [9].

This looks promising in a terrifying sort of way. You can drag and drop contacts between the iCloud and Google Contact lists; they'll be copied.

So what about Snow Leopard? Many fine machines run Snowie (and should not upgrade to Lion even if they theoretically can). They're (more or less) cut off from iCloud. Does CardDAV sync work with Google now? I wonder about adopting Hsiaio's technique, but so far the answer is no (same with Yahoo CardDAV). I didn't find much research on this question, but I suspect it's a combination of 10.6 bugs and SSL requirements. [9]. There is some limited Google Contacts sync on Snow Leopard that some still use.

- fn - lots

[1] Google grandfathered my half-dozen Google Apps accounts and recently extended the termination date for non-Apps users.

[2] Google says Google Apps accounts should use "Google Sync", which is their implementation of Microsoft's Exchange Server derived ActiveSync (with different bugs one assumes).

[3] vCard over HTTP/WebDAV. vCard has been Apple's Contact/Address Book export option for as long as I can recall, so it may be a good match to their Contact data model. CardDAV is also used by Apple's ancient Address Book Server. vCard 4 2011 has old-style and XML representations, but I suspect Apple, like Google, is more or less on 1998's vCard 3.

Google's vCard implementation has limits; worse the only partly implement the 1998 vCard 3 spec: "why did Google move the ADR extended address "Oak and Pine" into the street address component? And worse still, why did it separate the two using an '\r\n' (CRLF) sequence? The VCard 3.0 specification clearly states that CRLF must be escaped by the single escape sequence '\n', not by '\r\n'".[4]

[4] Reading the last line of [3], by the way, a penny dropped. Apple changed the way they handled CRLF and vCard between MobileMe and iCloud, with near-disastrous consequences for my OS X Contacts. By Darwin, I hate the ancient DOS/unix/mac line newline debacle.

[5] Meaning data loss is inevitable, transferring a Contact between systems won't return the original. Incidentally, under my TrueName, I've spent 15 years dealing with interoperability issues in healthcare - from HL7 to TermInfo to SNOMED/RxNORM and back again via cCCD and the like. If you understand why CardDAV is hard, then you have a glimmer of a smidgen of a hint of an idea why we don't have true interoperability between EpicCare and Cerner -- and why we won't for decades.

I think the people who trudge away on vCardX are heroic figures who should be carried upon our shoulders. Forget Ive, these people do the work I rely on.

[6] The marketing/interest problem is unsolved -- largely because of the spam/advertising counter-current. I'm very interested in productivity software and I follow a number of related feeds; but I still miss news like this.

[7] Check out "See also" - lots of history here.

[8] Looking at Apple's braindead implementation of iOS Contacts and Groups I'd pin this failure on Cupertino and Apple's corporate silos.

[9] A friend of mine has a 10.6 MacBook and needs to use Google's world (as do we all). I think the easiest way to manager her Contract migration will be to get them to iOS/Google and then use Gmail web on Mac. I may put them on an account on my ML machine to help with cleanup.

[9] So which gets edited when one clicks "Edit" in Mac Contacts.app? Don't ask those questions. OS X doesn't say.

See also - via Gordon's Tech

Yeah, I've been fighting this war a long time. February 2009, almost exactly 4 years ago, was a big series of battles, but the 2008 PalmOS to iOS migration was probably the worst (later there were tools). I've read recently that Apple is an intensely siloed company -- that explains why iOS and OS X contract integration is so very bad.

Saturday, January 12, 2013

Removing 3500+ iCloud Contacts with no content (No Name) and Contacts.app use of 25GB of virtual memory

I suspect many people's iOS Contacts have been trashed by iCloud, but only a small percentage realize how bad things are. It may explain some of the mysterious data use and battery life problems some see.

I can't prove this of course; I'm not about to study a random sample of 100 devices. I can only go by my own disastrous experience and Emily's recent malfunction.

In my case the bug had to do with a change between MobileMe and iCloud in the handling of line endings resulting in large numbers of 'Groups' metastasizing between my devices. It was very hard to diagnose and fix that bug; since then things have seemed to go well with my use.

Emily's malfunctions seemed simple at first. I noticed she had both addresses on her device and in iCloud. I didn't expect that, but I figured it would be easy to clean up. [1]. Then I discovered about 3,500 entries in Contacts that showed up only as 'No Name'. This isn't a new problem and I don't know how long Emily had it. She never noticed, I suspect most non-geeks ignore that sort of thing. I wonder if it explains some battery life issues she's had, and even Spotlight and disk access issues on some of our older machines as these malformed entries propagated. (As noted below, during the removal process the OS X Contacts app used 25 GB of virtual memory and locked up my 27" iMac.)

Removing these malformed null-value entries was much harder than expected. Something about processing the deletes using Mountain Lion Contacts.app consumed vast amounts of virtual memory -- almost completely paralyzing my 2 yo iMac with the usual Spinning Beach Ball of Death (SPOD, SBOD, spinning wait cursor). (See: Getting control of a Mac when rogue software consumes virtual memory.)

In retrospect I think I wonder if I should have disabled my network connection, made the changes, and then reconnected and let iCloud process the cached change list. In any event, this is more or less what I did to delete them; in our home we have 1 iOS device (Emily iPhone) and two OS X devices (Mountain Lion Contacts.app and older Lion-only machine running Address Book.app) that synchronize to her iCloud account.

  1. Work from our primary Mountain Lion machine, and create a local backup using the Contacts Archive function.
  2. Turn off iCloud synchronization from the other iOS and OS X devices and delete all local contacts. On iOS the OS will offer the delete option, on OS X you have to delete them manually. (With iCloud sync off delete is easy.)
  3. Open Safari to view changes there.
  4. Because of the problems I ran into I suggest turning off Time Machine and, if you know how to do it, disable Spotlight disk indexing. At least uncheck Contacts from the Spotlight Search Results in the Pref Pane. You'll add them back in after you are done.
  5. Close all other apps and log out all other users.
  6. Do this incrementally - delete a few hundred, then quit and restart Contacts. When I skipped some of the above steps and tried to delete all 3,500 at once my machine locked me out. After some geek gymnastics I found Contacts.app was using 25GB of Virtual Memory -- enough to store about 1,000,000 contacts. Very little CPU was being used, but that much VM use is fatal.
  7. When you're done run Contacts Cleaner if you own it.
  8. Restart OS X, reenable Time Machine, Spotlight, etc.
  9. On iOS (iPhone for example) reenable iCloud account. (In Emily's case this brought up the merge with local dialog -- the original small problem. See [1] for the easy fix for that, it does mean letting changes sync back to desktop as well).
  10. Once that's settled down, reenable iCloud accounts on other devices. Let Spotlight catch up indexing them. You can disable then reenable Contacts search if you like
It will be interesting to see if her battery lifespan and Spotlight behaviors improve.

[1] If that's your only problem here's the way to clean it up:

  1. Remove iCloud account from phone (you'll put it back).
  2. Make sure only local device Contact remain.
  3. Add iCloud back -- you'll get offer to merge them. Merge.
  4. Clean up duplicates, easiest to do that on a desktop account using Contacts Cleaner (with my latest version I didn't disable network connectivity when doing this) or the dangerous fully automatic built-in Mac desktop duplicate remove.

See also:

More on the No Name bug:

Saturday, November 17, 2012

iCloud transition - Contacts take 2

Four months ago, when MobileMe was turned off, an attempt to move my OS X Lion Contacts to iCloud transition was a first class debacle. I still have PTSD from that disaster; it took several consultations over a week and some AppleScript intervention to undo the mess.

Since then I've maintained a test set of Contacts on iCloud while I sync my primary Contacts from iPhone to Desktop via iTunes. Now my primary machine is on Mountain Lion, iCloud presumably has fewer bugs, and my iPhone is on iOS 6.01. So I tried again, following this procedure:

  1. Create a backup copy of my contacts on my main mac - prior to iCloud migration.
  2. MacBook Air - remove all contacts. (Cloud empty)
  3. MacBook - sync, confirm empty.
  4. OS X: set up iCloud -- everything goes over.
    1. 1833 cards on desktop
    2. 1833 cards in web view of iCloud (a friggin miracle that they match) - thanks to Nigel Garvey's CR/LF cleanup script.  I did see an issue - the web view seemed to be hanging mid-way through the letter S. I edited the note for that contact, quit and tried again. Everything showed up. Don't know what that means.
  5. iPhone -- need to clear out existing Contacts (was synchronizing via iTunes). Can't just turn on iCloud contacts because it will try to merge with iCloud. i'm sure that would be a disaster.
    1. Turn on iPhone sync with my Google Contacts (usually that's off). iPhone offers to delete local. I accept.
    2. Turn off Google Contact sync
    3. Confirm no contacts left on phone
  6. iPhone - turn on iCloud contacts
    1. 1833 cards on iPhone (note that matches the above counts. Another flaming miracle.)
  7. MacBook Air - turn on iCloud contacts
    1. Yes, again, 1833. Whoopee.
  8. MacBook running Lion …
    1. Here I chickened out. I really don't use that machine much any more and I don't really need to replicate my Contacts there. Why take the risk of adding another OS in the mix, especially Lion? In fact, I plan to revert that machine back to Snowie with a fresh install. Then my house will be Snowie and ML - two decent versions of OS X.

So I'm back on iCloud again for Contacts, and I think a few bugs have been worked out.

Friday, July 27, 2012

An AppleScript to prevent the iCloud CRLF bug - normalize line endings

Yesterday I wrote perhaps the fifth AppleScript of my life, and I purged an iCloud paralyzing cloud of "Groups" created as a side-effect of iCloud's CRLF bug.

In the course of writing that brief AppleScript, I benefited from the extraordinary community built known as MacScripter. As I basked in my small victory against the uncaring forces of Appledom, I thought once again of fixing my Address Book's mixed CRLF/LF line terminations with AppleScript. I joined MacScripter and asked for advice.

In response,  Nigel Garvey wrote me a script. I suppose for him it was easier to write the solution than to try to explain it!

I ran it on my Snow Leopard primary address book (of course I made an archive backup). After a few seconds it returned "missing value". I've seen that when scripts complete, though it doesn't seem to be an ideal response.

In any case, I didn't see any evidence of windows style CRLF line terminators. When I exported test addresses as vCard they didn't show unexpected numbers of \n characters; though I continue to have some extra line feeds at the end of notes (but they seem to have LF terminations, not CRLF).

I took an exported archive and imported it into my Mountain Lion machine, then sent it to iCloud. I then viewed on my Lion machine. On all devices I see 1824 contacts and no duplicated notes. The only problem I came across were the extra LF at the end of notes, but they did not appear to be replicating. I think they are benign LF life feeds, not dangerous CRLF pairs.

I believe Mr Garvey's script worked. If you are one of the very few people who ever used MobilMe to sync Windows/Outlook and OS X/Address Book you should back up your Address Book, run this AppleScript, then confirm. Then consider migrating to the Cloud.

Thursday, July 26, 2012

iCloud group replication resolved by AppleScript

It's been a month since my disastrous MobileMe to iCloud migration. Over the course of a week I figured out a convoluted solution for iCloud's CRLF/LF end of line bug, but I was left with "metastatic group replication" in my primary iCloud account/quaternary AppleId [1]. Each of about five Address Book groups had been replicated about 50-100 times (only one of the replicants has associated contacts).
 
The Groups were in iCloud - not Address Book. So even deleting my Address Book Library files didn't remove them. Bento couldn't see them. Mountain Lion didn't help. The only option appeared to be tedious manual deletion. My iCloud account was poisoned.
 
Today I figured out a solution. I had to learn a bit of AppleScript [2] but I'll recover. It was worth it (in ML Address Book has been renamed Contacts):
tell application "Contacts"
repeat 50 times
set theGroup to group "MyGroupName"
delete theGroup
end repeat
save
end tell

It's a crude hack. I had to change "MyGroupName" five times, and when the replicant count dropped I had to reduce the repeat number so it would execute. (One could drop the count  parameter and see if the repeat loop stops when it errors with "missing value"; that would be much faster. I was being cautious.)

At one point Contacts didn't respond very well. I quit and restarted and completed the process.

When I was done I confirmed iCloud's web view was also Group free. Then I purged all the Contacts in that account [3], it's now pristine.

I'm sure an AppleScripter could turn this into a general script for removing all groups all at once. Or could get very clever, and only remove groups that had no contacts (so one could salvage the group/contact relationships).

I didn't need that. My problem has been fixed.

Incidentally, I think iCloud sync works a lot better in ML than Lion. iCloud updated very quickly. 

[1] I have 4 AppleIDs. Two or three of them have iCloud accounts. One has all of my iTunes and AppStore purchases. One was my MobileMe account. A third, which I've never used, acquired, apparently by email transfer, all of my Apple device records. Does Apple have any idea how screwed up their AppleID/iCloud accounts are?

[2] Of which Jamie Zawinski wrote: "I used to think that PHP was the biggest, stinkiest dump that the computer industry had taken on my life in a decade. Then I started needing to do things that could only be accomplished in AppleScript."

[3] They're safe in Snow Leopard until I move my primary machine to Mountain -- which might be sooner than expected. ML seems to be everything Lion should have been.

Wednesday, July 04, 2012

Hotel iCloud

"… Welcome to the Hotel California
Such a lovely place (Such a lovely place)
Such a lovely face
Plenty of room at the Hotel California

"We are programmed to receive.
You can check-out any time you like,
But you can never leave…"

Eagles, Hotel California

I'm in recovery from a tedious multi-update post on my failed iCloud transition, and still digesting all I've learned. I'd like to pass on one conclusion.

iCloud really is different from MobileMe. 

MobileMe was a peer-to-peer synchronization service. My data lived on my machines, as well as on MobileMe.

iCloud is a server solution with local caching. The data moves to iCloud, it is no longer On My Mac.

Yes, I know it seems to be in the usual AddressBook folder, but as I discovered the real source of truth is remote. When there is a clash of opinions on the state of data, iCloud wins. If your data goes bad, you need to fix things on iCloud. Good luck with that.

Perhaps even more importantly, this is a one way trip. Yes, if you know the ins and outs you can copy Contacts back to your Mac and remove them from iCloud. You cannot, however, get your Group relationships back. That data has checked into Hotel iCloud.

With iCloud, OS X Mail and Address Book/Contacts are much more like Google Mail/Contacts -- which also has an offline mode. Apple's data lock is stronger than ever, and I have a very bad feeling about the next version of iTunes.

Saturday, June 16, 2012

Notes on converting a MacBook Core 2 Duo dual USB to Lion

Our vintage 2006 MacBook Core 2 Duo can, in theory, run Lion [1]. I just upgraded it from Snow Leopard, which it ran reasonably well albeit with lots of fan activity.

I didn't upgrade happily. There's a reason I've waited this long. I wanted to stay with Snow Leopard, but Apple's MobileMe migration was going to cause problem for Emily's Address Book/iPhone Contact sync. Yes, that was the primary reason. Sad.

I upgraded all the apps we use, said farewell to AppleWorks and many games the kids no longer use, removed all PreferencePanes and extensions, unplugged all cables, did a safeboot cleanup and two completely independent backups (one a clone). Then I upgraded.

It seemed to go well enough once Spotlight rebuilt its indices and I refreshed everyone's account. it does take a very long time to log out; Lion is saving a lot of state information and the MacBook doesn't like that.

It went well enough, that is, until my old account. Then things got slow. Even though I'd updated VMWare Fusion to the final "Lion Compatible" 3.x version, I suspected it was the problem. I tried running my VM -- that was a disaster. After power down and a safe boot I uninstalled VMWare 3 per directions.

That helped a great deal. In fact, the laptop is quieter than it used to be. I suspect Fusion has been causing problems for a while [3].

So will I try Fusion 4?

No, not on this old machine. I really don't use my VM's very often, and neither Parallels nor Fusion are sold through the App Store. To do their magic without Apple's help they must be hacking the underlying OS; and OS X is increasingly hard to safely hack.

For now the MacBook seems to have survived Lion - albeit at the cost of a little used VM I'm probably better off without and a dozen or so older games -- and AppleWorks.

So far, better than it could have been. I am, however, regretting obeying Lion's command to update my Airport firmware [2]. 

[1] That's almost six years ago! I'd forgotten how old it really was. Maybe I shouldn't be too upset it can't run Mountain Lion. [2] Lion really wanted me to undo my Time Capsule firmware regression, so I did. Now I'm seeing more problems with losing connectivity, i saw a -1 error again, and I'm again having to rebuild Spotlight indices of the backups. I have a strong feeling I'll be reverting again. I seem to be the only one with this problem though. [3] My best Fusion experience was version 2 with a Windows 2000 VM -- on that old MacBook probably with Leopard (10.5). It's never worked as well since. 

Update 6/17/2012: Logging out and user switching is much slower. It takes about 20-30 seconds to log out and 15-20 to switch. I think it's because of all the context saving Lion does; this old machine can't handle it. There are ways to disable saving of application states, but I'm going to wait a while on this one. Otherwise things aren't too bad. The machine is quieter than it has been for years, the fan no longer roars. I suspect that's due to uninstalling Fusion 3.x, but it could be a Lion improvement.

Update 6/17/2012: I went through each user account and turned off 'save and restore windows' in system preferences. Then I logged out and unchecked the restore windows on login option shown there. No logout and login is back to Snow Leopard times. Now I have to figure out what to do about #$@$ Google Software Update. it keeps popping up in managed accounts that don't have privileges to run it.

Thursday, May 17, 2012

iOS Restore problems: App Sync failure and missing Google Contacts

Fresh off discovering iOS Restore problems with In-App purchases and video podcasts I was called in to help a friend transition from two 3GS to two 4S iPhones.

He had a different set of problems, some familiar, some weird, some new to me:

  • When restoring Mail, Calendar and Contacts from Google Active Sync, the best option is to delete the Google account and restart. I've run into this several times, and reentering a password on a restored Google Active Sync (Exchange) data set works about half the time.
  • He ran into a real mess with AppleID vs. me.com ID. I think he'd started out with one Apple ID (Store Preferences) on his iPhone, then switched to another. As a result he had apps on his iPhone that no longer matched his iPhone Apple Store ID. They'd run, but they wouldn't sync to the desktop or the Cloud. When we did a restore they were gone. There's no way around this; he either has to switch to his old ID (but it's lost) or repurchase the apps that were owned by the lost ID. Yech.
  • He and his wife share one instance of iTunes, but their apps have different Apple Store IDs. This can be done, but it requires a mixture of Cloud and iTunes sync. Double Yech.
  • To enable email delete on Gmail with Mail.app delete, we had to rediscover the insanely obscure m.google.com/sync setting that makes Gmail behave like a rational piece of software.

Those are the familiar bugs. Yes, Apple Store IDs and App DRM are a mess. Apple isn't a genius every day of the week.

The weird bug was, I think, a Google bug. After we reentered his Google credentials and synchronized, we found several missing entries in his Phone shortcuts. That's because the Contacts were missing. After a few cycles of removing/restoring Contacts Sync I could see the missing entries varied.

Turns out that Google was giving us exactly 100 out of 216 contacts; but the 100 varied. It seems Google was throttling the phone update.

Maybe if we'd waited a while we'd have gotten the rest. Instead I changed Sync from Push to Manual. Tried a few manual updates to no effect, then switched back to Push and we had all 216. A real pain, and I found zero hits on this problem. (Now there is one.)

The iOS user experience could use some work. In particular, the Backup story is pretty feeble.

Thursday, May 03, 2012

MobileMe to iCloud - Is Apple getting nervous?

Apple's MobileMe service officially ends on June 30th, almost 8 weeks from now. When it ends we either need to go to iCloud or give up on easy address book synchronization across our iOS and OS X devices and user accounts. (I've already moved the kids over.)

I'm in no rush. iCloud/Mac requires Lion and Lion won't install on the older iMac the kids still use. Worse, I don't even like Lion on the machines it supports. Lion is a disappointment.

I'd prefer to stay with Snow Leopard until early 2013, then switch to Mountain Lion. Except, of course, Mountain Lion won't install on our MacBook.

Yech.

I suspect I'm not the only one who isn't in a rush. Apple seems worried. The last time I used the MobileMe web interface I ran into a fake-out splash screen that tried to convince me MobileMe was already gone (nice try Apple). Recently Apple sent me a free Snow Leopard DVD to reduce the cost of a Lion upgrade (the cost is irrelevant). Today Apple is telling me I can keep my email address even on machines that don't support Lion [1]. Meanwhile, Macintouch, an old-school Mac site, shares ideas  from MobileMe dead-enders and iCloud denialists.

The pushback is strong enough that hard core geeks are coming up with inventive ways to sync Snow Leopard Address Book and Calendar to iCloud [2].

One option, of course, is to go all in with Google. This was more appealing when Google was less evil than Facebook. Worse, it's not clear how well Snow Leopard Address Book did synchronizing with Google. Lion still supports this; I created an empty account and synchronized with Google. It ended up bringing over 598 cards from the group "My Contacts". (All Contact had 2091 members). I've no idea how reliable this is, but iOS synchronization with Google works quite well. So this might be an option for Lion machines, and non-Lion machines would use Gmail (which is evil now, but works well [3]).

Lastly there's Spanning Sync - an alternative Mac Address Book to Google Contacts option for those who want to abandon MobileMe for Google Apps. (It's an expensive option for a family however.)

Alas, these days I don't want to get closer to Google; I'm trying to move the other way.

So I'm stuck, waiting to see if anyone else comes up with something better. Maybe if I wait long enough Apple will make Mountain Lion run on my old MacBook.

[1] Sort of. Apple botched this half-measure, even by lowest of standards. There's no way to tell from Apple what this means.. What it really means is email and calendars currently in MobileMe will be accessible on iCloud via the web UI.Unofficially Snow Leopard Mail.app may also be able access this email via IMAP.

[2] I doubt this will work all that well; synchronization is hell even when it's supported.

[3] iCloud's web apps are better than MobileMe's -- but an earthworm could clear that bar. iCloud Contacts, for example, is even more awful than Lion Address Book.

Tuesday, November 29, 2011

Migrating MobileMe family accounts to iCloud

I've started migrating family accounts to iCloud. This explanation from Apple Discussions is helpful ...

With a Family Pack, the master account holder and the sub-account holders can each migrate to an individual iCloud account by going to http://me.com/move and entering their email address and password (not www.icloud.com). The order they do this in is immaterial; if the master account moves first it can no longer administrate the sub-accounts. If a sub-account moves first the master account cannot create a new sub-account to replace it. Once migrated each account becomes a full iCloud account entirely separate from the others. The master account holder will get the 20GB storage upgrade free until June 30th 2012; the sub account holders will not, and will have only the basic 5GB.

I began with #2, currently using a SIM-less iPhone 4. He's got almost no data to lose.

I want him to continue to use my Apple ID however ...

... iPhone, iPad, and iPod touch must have iOS 5 or later. Note: When you are asked to provide your Apple ID during iOS setup, use your MobileMe email address and password. To use a different Apple ID for iTunes and iCloud, just go to Settings > Store on your iOS device after you've finished the iOS setup assistant...

... Inactive and expired MobileMe accounts do not need to move to iCloud. Simply use your inactive or expired account to sign up for an iCloud account and follow the onscreen instructions. See this article for more information...

and

When you first set up your iOS 5 device, enter the Apple ID you want to use with iCloud. If you skipped the setup assistant, sign in to Settings > iCloud and enter the Apple ID you’d like to use with iCloud.In Settings > Store, sign in with the Apple ID you want to use for store purchases (including iTunes in the Cloud and iTunes Match). You may need to sign out first to change the Apple ID.

Following this path I learned that #2 will still have MobileMe Gallery, iDisk and iWeb Publishing through 6/30/2012 -- even after moving to iCloud. Better than I'd expected. No keychain sync though. I hope that comes back in some form.

I had to enter his iCloud (same as MobileMe) credentials to setup Facetime and iMessage on his SIM-less device. Although the phone doesn't need a SIM for this, it can't be in Airplane mode.

So far, it has gone better than expected.

Update: Thinking this over, I realize I need to update my machines to Lion before I move Emily and I. So this will take a while ...