Getting control of a Mac when rogue software consumes virtual memory.

Adam Fields claimed something is deeply broken in OS X memory management. He says it's been partly fixed in Mountain Lion, but .. "the VM subsystem still falls over if you push it too hard".

That fits my experience when a bug with iCloud and Mountain Lion Contacts used Contacts.app to consume 25GB of virtual memory on my 8GB iMac. My system ground to a halt, even though CPU use was light. My system was locked up by disk bound memory paging.

Actually, it wasn't quite locked up. It was just excruciating slow to respond; a mouse click took minutes to process while I watched the OS X Spinning pinwheel.

I wanted to figure out what was happening, so I didn't simply power down. I was able to get control, which is how I identified the iCloud/Contacts bug and was able to gracefully quit Contacts, shut down normally, and then deal with the Contacts bug [1]

To get control I needed to get Terminal.app running. Terminal will work where most other things are frozen, and from Terminal I could control some of the usual bad actors. Note that I run with a non-admin account and I don't have root enabled.

At first I tried to ssh from a remote machine. To do that I needed to know the current IP address of my iMac; it didn't help to know the afp:// network path [2]. The only way I know to do that is to via this Mac OS X Hint. Note the hint has an error in the string, the comment versions are correct. From my remote machine I connected to iMac from the Finder and then used one of these Terminal commands (first is what I did and it works fine):

• netstat -na | grep 548
• netstat -f inet -n | awk '$5 ~ /\.548$/ {sub(/\.[^.]+$/, ""); print $5}'

Alas, that didn't help. I hadn't enabled Sharing:Remote Login for my iMac (I've since fixed that). So I had to return to my iMac and, carefully and slowly, get Launchpad [3] up and then Terminal. [5]

Since the account was non-admin I had to get Admin privileges so I could run sudo (root, superuser) commands:

•  su Admin

Spotlight is always a problem with system resources are low, so my first action was to get it out of the way. This is hard to do; I've read that the process restarts when killed but I tried anyway so I'd get a clean version restarting:

• sudo killall mds

Then I used renice to set spotlight priority to a lower value [4], first finding the process ID (it was 43702 at the time) then renicing it:

• ps -acx | grep mds
• sudo renice -n 19 -p 43702

That bought me more responsiveness. I was then able to slowly quit all running apps and gradually free up system resources until I could run Activity Monitor and figure out what was wrong with Contacts.app

[1] So there are at least two Mac bugs involved in this. One is a longstanding problem with OS X memory management that, I suspect, goes back to OS X 10.0. The other is one of several newer bugs with iCloud and Contacts. [2] That can be found by using Finder "Connect to Server" dialog from a remote machine. [3] LaunchBar and Finder weren't responding but Doc and Launchpad were slow but did respond. [4] It's annoying that renice isn't accessible from Activity Monitor -- in Windows there's a GUI for this control.  [5] I now put Terminal in the dock of my primary non-admin account for faster access next time and I enabled remote login in Sharing options: To connect use ssh username@ip_address, example "ssh molly@10.0.1.4".  If you have enabled root I think you can use "ssh root@10.0.1.4", but in my testing with ML where I have not enabled root that doesn't work.

See also:

• View connected file sharing clients' IP numbers - Mac OS X Hints
• Tutorial: Troubleshooting Spotlight | MacFixIt - CNET Reviews: 2006! Old, but sadly still relevant.
• Gordon's Tech: Spotlight indexer (mdworker) priority is too high in Lion/Mountain Lion 9/2012 - permanently setting Spotlight to lower priority. Excluding Aperture did help me.