Saturday, January 12, 2013

Removing 3500+ iCloud Contacts with no content (No Name) and Contacts.app use of 25GB of virtual memory

I suspect many people's iOS Contacts have been trashed by iCloud, but only a small percentage realize how bad things are. It may explain some of the mysterious data use and battery life problems some see.

I can't prove this of course; I'm not about to study a random sample of 100 devices. I can only go by my own disastrous experience and Emily's recent malfunction.

In my case the bug had to do with a change between MobileMe and iCloud in the handling of line endings resulting in large numbers of 'Groups' metastasizing between my devices. It was very hard to diagnose and fix that bug; since then things have seemed to go well with my use.

Emily's malfunctions seemed simple at first. I noticed she had both addresses on her device and in iCloud. I didn't expect that, but I figured it would be easy to clean up. [1]. Then I discovered about 3,500 entries in Contacts that showed up only as 'No Name'. This isn't a new problem and I don't know how long Emily had it. She never noticed, I suspect most non-geeks ignore that sort of thing. I wonder if it explains some battery life issues she's had, and even Spotlight and disk access issues on some of our older machines as these malformed entries propagated. (As noted below, during the removal process the OS X Contacts app used 25 GB of virtual memory and locked up my 27" iMac.)

Removing these malformed null-value entries was much harder than expected. Something about processing the deletes using Mountain Lion Contacts.app consumed vast amounts of virtual memory -- almost completely paralyzing my 2 yo iMac with the usual Spinning Beach Ball of Death (SPOD, SBOD, spinning wait cursor). (See: Getting control of a Mac when rogue software consumes virtual memory.)

In retrospect I think I wonder if I should have disabled my network connection, made the changes, and then reconnected and let iCloud process the cached change list. In any event, this is more or less what I did to delete them; in our home we have 1 iOS device (Emily iPhone) and two OS X devices (Mountain Lion Contacts.app and older Lion-only machine running Address Book.app) that synchronize to her iCloud account.

  1. Work from our primary Mountain Lion machine, and create a local backup using the Contacts Archive function.
  2. Turn off iCloud synchronization from the other iOS and OS X devices and delete all local contacts. On iOS the OS will offer the delete option, on OS X you have to delete them manually. (With iCloud sync off delete is easy.)
  3. Open Safari to view changes there.
  4. Because of the problems I ran into I suggest turning off Time Machine and, if you know how to do it, disable Spotlight disk indexing. At least uncheck Contacts from the Spotlight Search Results in the Pref Pane. You'll add them back in after you are done.
  5. Close all other apps and log out all other users.
  6. Do this incrementally - delete a few hundred, then quit and restart Contacts. When I skipped some of the above steps and tried to delete all 3,500 at once my machine locked me out. After some geek gymnastics I found Contacts.app was using 25GB of Virtual Memory -- enough to store about 1,000,000 contacts. Very little CPU was being used, but that much VM use is fatal.
  7. When you're done run Contacts Cleaner if you own it.
  8. Restart OS X, reenable Time Machine, Spotlight, etc.
  9. On iOS (iPhone for example) reenable iCloud account. (In Emily's case this brought up the merge with local dialog -- the original small problem. See [1] for the easy fix for that, it does mean letting changes sync back to desktop as well).
  10. Once that's settled down, reenable iCloud accounts on other devices. Let Spotlight catch up indexing them. You can disable then reenable Contacts search if you like
It will be interesting to see if her battery lifespan and Spotlight behaviors improve.

[1] If that's your only problem here's the way to clean it up:

  1. Remove iCloud account from phone (you'll put it back).
  2. Make sure only local device Contact remain.
  3. Add iCloud back -- you'll get offer to merge them. Merge.
  4. Clean up duplicates, easiest to do that on a desktop account using Contacts Cleaner (with my latest version I didn't disable network connectivity when doing this) or the dangerous fully automatic built-in Mac desktop duplicate remove.

See also:

More on the No Name bug:

at 2 comments:
Labels: , , , , , ,

Getting control of a Mac when rogue software consumes virtual memory.

Adam Fields claimed something is deeply broken in OS X memory management. He says it's been partly fixed in Mountain Lion, but .. "the VM subsystem still falls over if you push it too hard".

That fits my experience when a bug with iCloud and Mountain Lion Contacts used Contacts.app to consume 25GB of virtual memory on my 8GB iMac. My system ground to a halt, even though CPU use was light. My system was locked up by disk bound memory paging.

Actually, it wasn't quite locked up. It was just excruciating slow to respond; a mouse click took minutes to process while I watched the OS X Spinning pinwheel.

I wanted to figure out what was happening, so I didn't simply power down. I was able to get control, which is how I identified the iCloud/Contacts bug and was able to gracefully quit Contacts, shut down normally, and then deal with the Contacts bug [1]

To get control I needed to get Terminal.app running. Terminal will work where most other things are frozen, and from Terminal I could control some of the usual bad actors. Note that I run with a non-admin account and I don't have root enabled.

At first I tried to ssh from a remote machine. To do that I needed to know the current IP address of my iMac; it didn't help to know the afp:// network path [2]. The only way I know to do that is to via this Mac OS X Hint. Note the hint has an error in the string, the comment versions are correct. From my remote machine I connected to iMac from the Finder and then used one of these Terminal commands (first is what I did and it works fine):

  • netstat -na | grep 548
  • netstat -f inet -n | awk '$5 ~ /\.548$/ {sub(/\.[^.]+$/, ""); print $5}'

Alas, that didn't help. I hadn't enabled Sharing:Remote Login for my iMac (I've since fixed that). So I had to return to my iMac and, carefully and slowly, get Launchpad [3] up and then Terminal. [5]

Since the account was non-admin I had to get Admin privileges so I could run sudo (root, superuser) commands:

  •  su Admin
Spotlight is always a problem with system resources are low, so my first action was to get it out of the way. This is hard to do; I've read that the process restarts when killed but I tried anyway so I'd get a clean version restarting:
  • sudo killall mds
Then I used renice to set spotlight priority to a lower value [4], first finding the process ID (it was 43702 at the time) then renicing it:
  • ps -acx | grep mds
  • sudo renice -n 19 -p 43702
That bought me more responsiveness. I was then able to slowly quit all running apps and gradually free up system resources until I could run Activity Monitor and figure out what was wrong with Contacts.app

[1] So there are at least two Mac bugs involved in this. One is a longstanding problem with OS X memory management that, I suspect, goes back to OS X 10.0. The other is one of several newer bugs with iCloud and Contacts. [2] That can be found by using Finder "Connect to Server" dialog from a remote machine. [3] LaunchBar and Finder weren't responding but Doc and Launchpad were slow but did respond. [4] It's annoying that renice isn't accessible from Activity Monitor -- in Windows there's a GUI for this control.  [5] I now put Terminal in the dock of my primary non-admin account for faster access next time and I enabled remote login in Sharing options: To connect use ssh username@ip_address, example "ssh molly@10.0.1.4".  If you have enabled root I think you can use "ssh root@10.0.1.4", but in my testing with ML where I have not enabled root that doesn't work.

See also:

at No comments:
Labels: ,
Subscribe to: Posts (Atom)