Sync disaster #257 - New Missing Sync 5.0 wipes out address book on my i500 cell phone

Computers have caused me vast pain over the years, but nothing has hurt so much as synchronization. I had disaster #257 when an attempt to sync my Samsung i500 PalmOS 4.1 address book with my OS X Address Book using Missing Sync 5.0 locked up and then wiped out my phone book.

Why does sync cause so much pain?

I think it's one of those marvelous problems that is:

1. Actually quite trickly and complex, since it involves syntax, semantics, data model reconciliation, mapping and more. To anyone who's worked in the knowledge representation industry these are words to inspire deep fear.

2. Thought by management to be quite simple.

That's an equation for disaster. The only thing worse than a fundamentally hard problem is a hard problem that's grossly underestimated. Mapping disasters killed Palm (Outlook data model reconciliation issues). It won't kill OS X, but it's interesting to see how Apple bodged this one.

They made a big deal about iSync, which was really pathetic and confusing, but now they have 'Sync Services' in Tiger as well as iSync (probably because they want iTunes on Windows to sync Outlook data to iPods, so they had to move sync into a layer that could be shipped on Windows). Add in something like Missing Sync 5.0 and disaster is well nigh unavoidable (MS 5.0, for example insists on including an iPod in the sync melange, as though two device sync wasn't hard enough!

So, now I'm trying to recover my address book. addressbook.data tiger - Google Search helped. The file is AddressBook.data in Tiger and it's easy to find using Spotlight. There are previous versions too, so I'll try swapping one of those, or just restore from my backups. Don't be fooled by all the cache files that contain addresses; those are generated to help Spotlight work.

Update 11/7/05: I'm not sure this wasn't due simply to the fact that the i500 appears to be USB 1.0, I have about 3000 contacts, and there seems to be a lot of work going on with each contact synched. I let the thing run overnight and it might have completed.

iMac heating problems and the curse of the Canon printer driver

Apple's forum has a not-yet-deleted thread on heat concerns with the iMac (ambient light sensor Rev B) model.

Some excerpts:

Apple - Discussions - OVERHEATING (of new iMac & version-B)

... fans started revving higher over the last couple of weeks. Didn't change after I installed 10.4.3. I reset the smu, no change. Then i remembered some one mentioning a dust factor in the bottom vent. So I took off the back and looked. No blistering (actually I've only heard of one case of als blistering yet many seem to have this belief that the rev b hardrives blister from excessive heat.), no capacitors messed up and clean as a whistle. However when I took some air to the vent there was quite a bit of dust in the vent. I made sure all of it was out and the inside were clean, closed it up, reset the smu again and for the last week its been even quieter than when new. temps are normal, except the harddrive is cooler and the fans have been great. If they do start to get louder do to a load, they come back down very quickly after. In short, its behaving better than when I got it.

... I've had more fan noise in the past few weeks. (machine specs as in footer, 3-4 month old ALS G5 2GHz 400MB 1.5GB RAM 20")

I can't tell why, but I think the system runs hotter. I had upgraded from 512 to 1.5GB and I wonder if that's tipped something to a higher temp. I switched to 10.4.3 and did some system maintenance (reset the SMU

Using ThermographX I find:

- low speed mode: CPU 145 F, drive 114 F
- high speed mode: CPU 192 F, drive 126 F with a noticeable persistent fan hum
- automatic: behaves like high speed mode, for some reason I don't seem to stay at the low speed temp once I go to automatic

By your note my system is running consistently ABOVE the "critical point" for the CPU (187F) you referred to.

... AppleCare is not a panacea however. My impression from Macintouch reading is that Apple outsources AppleCare and that the 'repairs' can be questionable. If time is an issue one may be better off going with the extended warrantees that are available with some credit cards and using them to pay for repairs done by a non-Apple authorized service center -- if you can find one! We do have one such in the twin cities.

In the meantime I think persons who bought a Rev B thinking Apple had fixed the design problems with the Rev A need to assume that the Rev B is similarly flawed. Clean the air filters, monitor temperature (themographX is $7), set the machine to sleep when not in use, don't use it as a media server, set the CPU to lower speed.

Update 11/7: This was solved.

When I set Activity Monitor to show ALL processes, not just my processes, I discovered that PrintJobMgr was occupying 90% of my CPU. A google search revealed this is a known problem; a stuck Print Job can drive CPU utilization up, shift the iMac to highest CPU setting, and drive system heat up. The print job was stuck because my Canon IP 4000 ink jet has been flaky ever since I attached it to my Airport Extreme USB print server. I don't like print servers! In this case my wife initiated the print job and I had no idea it never completed. I've also read a comment from an expert source that Canon printer drivers in OS X seem to be very CPU intensive and can cause heat spikes.

The CPU is now running abou 100-110 F and the hard drive about 100-110 F.

In Print Center I found the stuck print job and removed it. The machine started to cool down almost instantly. In automatic mode again the temperature continued to drop, as tracked by 'Hardware Monitor' (which I will buy).

When I figured this out I felt a bit guilty about lambasting Apple for their iMac Rev B persistent heat problems. After further consideration, however, I think a bit of basting is in order (I'll take my lumps too.) There's something wrong when an errant print job can cook a $2000 computer. Four things in particular:

• Why did this job hang in the first place? Why is printing to Apple's Airport Extreme print server unreliable with the Canon IP4000? How much of this is Canon's fault vs. Apple's fault?
  
• Why does PrintMgr hog so much of the CPU?
• Why didn't some process notify the user of a rogue job?
• Lastly, of course, this does emphasize that the iMac G5 remains vulnerable to overheating with CPU intensive processes.

So Apple does have something to answer for.

BTW, this issue of heat and computation, and monitoring systems for thermal burden, isn't going away. Heat production and power consumption are the key drivers now of system and CPU design -- and this will be so until we cook the planet. (The physical limits of computation are, I dimly recall, somehow related to the ability to dump heat ...)

Update 11/8: A few odd thoughts and tips related to managing heat output on all iMacs

• via Macintouch: turn down LCD brightness
  "The LCD brightness setting has a significant effect on power consumption: minimum 82 W; midpoint 88 W; maximum 106 W. With this in mind, I've set the brightness to minimum, which is perfectly adequate (the factory setting is maximum). This should reduce component thermal stress issues and extend display life, because much of that power is being dissipated as heat, rather than going into actual light output."
• have machine sleep often and rest hard drive aggressively
• this is not a great machine to use as a video server!
• run at temperature monitor and activity monitor routinely -- look for spikes and rogue processes (caveat: I'm seeing some odd system instability doing this, I need to investigate further.)

And a thought on fan noise:

• If the fans are loud -- ask is the system hot (appropriate fan activity) or not (OS bug). Just put your hand in the exhaust stream and feel if it's hot.
• If the system is cool, reset the system management unit (SMU) or even zap pram or reset the open firmware, repair permissions, do a safe start, sacrifice a chicken beneath the full moon, etc.
• If the air exhaust is hot, find out why. Check active processes. Clean the intake vents. Consider the preceding voodoo -- but it likely won't help. Don't let the system run hot -- this is bad.
  

Getting started with automator: MacWorld feature

Macworld: Feature: Take Automator for a trial run

Socialtext: a commercial and open source enterprise wiki

An interesting product for distributed collaboration: Socialtext -- Enterprise Social Software. They notify via email, I can't tell if they notify via RSS as well.

Microsoft Office SP2 disables ability to edit an Excel spreadsheet via Access

This morning a routine tool I use stopped working. My access query that appends to an Excel spreadsheet gave me this unhelpful error message: Operation Must Use an
Updateable Query.

I spent an hour fruitlessly trying to figure out what had happened, then I gave up and rebuilt it using another approach. I asked around on usenet and got a quick and accurate response

The response pointed to this Microsoft kb article:

You cannot change, add, or delete data in tables that are linked to an Excel workbook in Office Access 2003 or in Access 2002

...This expected behavior occurs when either of the following conditions is true:

• You have installed Microsoft Office 2003 Service Pack 2 (SP2).
• You have installed the update for Access 2002 (KB904018) that is dated October 18, 2005.

...
MORE INFORMATION

Because of legal issues, Microsoft has disabled the functionality in Access 2003 and in Access 2002 that let users change the data in linked tables that point to a range in an Excel workbook. However, when you make changes directly in the Excel workbook, the changes appear in the linked table in Access.

The kb article includes a set of workarounds that are pretty humorous, along the lines of 'open Excel and type in the data'. Evidently Microsoft has fallen afoul of a patent (a stupid patent I'd wager).

Wow. I wonder how great a productivity hit this is around the world. I would really like to know why Microsoft didn't put a prominent warning in the installer for SP2 or on their SP2 web site. Did the courts forbid notification? Did Microsoft think no-one would notice?

This combination of non-notification and loss of functionality from a "service pack" is unprecedented. Microsoft seems to have become a very troubled company very quickly.

Working around iTunes lack of multi-user support

Update 2/28/06: Geez, this is complex. I ended up giving up on the sync approach when I found I could do just as well by dragging and dropping a music folder from my Public folder onto my wife's copy of iTunes. (I only need an update sync, not a true sync.) However, some things didn't work. My wife couldn't select a bunch of tunes and edit ratings and preferences across the group, she could however set preferences on one file at a time. I had a hunch permissions were an issue. She could read from but not write to the Public folder.

So, I next used Sharepoints to create a new Group called Parents (grrrr. OS X should have an easy bundled GUI app to do this -- in this regard OS X is as bad as XP Home.) and added Emily and I to that Group. Then I set permissions on the Music folder within my Public folder so Parents could read and write to the Music folder. This fixed the problem.

Why is write access required to set ratings using one method, but not with another method? I don't know, but I'd bet I've uncovered some hacked designs in iTunes.

Update 11/8/05: MacOSX Hints has a version of this that's at least comparable if not better. I might integrate the two some day or leave it up to the reader! The MacOSXHints submission includes an AppleScript that looks like it may update the Client's Library database when tunes are added to the Master Library. That sound great and I'll try it.

Update 11/10/05: The synchronization script referenced in Mac OS X Hints didn't work when two users were on the same machine and used Fast User Switching. I figured out how to revise it to support FUS (my first ever AppleScript project). I posted the sample code to the comments section on the Mac OS X Hints article. I may create a web page for this sometime; the techniques are apparently undocumented. (Parenthetically, I think AppleScript's days are numbered -- the lack of documentation is impressive.)

Update 12/11/05: It turns out there's a much easier approach that just works with iTunes. After adding new tunes to the 'master user', switch to the 'client user'. Then click on the folder containing all the music. Drag and drop to the left side of iTunes. iTunes compares and only adds references to the new music. View by date added and add to playlists as desired.

Here's my original article with its updates:

---

There's a "design flaw" in iTune when it comes to multi-user support.

Here's a brief version of the hack to work around it. I'll write more when I'm less tired.

1. Move iTunes Library to Public folder of iTunes owner. (If you don't know how to do this you shouldn't be trying a hack like this!).
2. User 2 opens Public Folder. Copy 'iTunes Library' file to User 2 music folder.
3. Open iTunes. Change advanced prefs on Library to 'don't copy when add'.

User 2 now has their own database pointing to the music repository. They can rate songs, create playlists, etc -- all independently. They can sync with their own contacts and calendar and their own version of iPhoto. (Ahh, but you want to use the shared iPhoto library? Tough luck! Similar set of problems as with iTunes. I'm still working this one. For now I export the images that will go on my wife's Nano.)

But what about songs I add to my collection in future? I have experience with this in another setting. You keep track of what you added and you drag and drop the folder into iTunes for User 2. iTunes will add the songs by reference (since pref is set to 'no copy'.) A bit tedious, but not that hard really. We only add songs occasionally. When you do this drag and drop by the way, iTunes does not create duplicates, it's very good at resolving these conflicts.

Can user 2 do damage? Not to the music library. They have read but not write privileges on the Public folder. [see 2/28 update for why this doesn't quite work]

Update 11/3: A longer version of the above that I submitted to MacOSXHints:

iTunes is designed to support multiple iPods, but not multiple users. In particular attributes like "last played" and "ratings" are meaningful only for a single user. Contacts and Calendars are also user specific.

That's fine, unless your spouse (SO, daughter, son, etc) has their own iPod. Each time they sync they'll mess up smart playlists that depend on "last played", and "ratings wars" may occur.

Of course they could have their own music collection and sync in their own user account. That's presumably what music owners would like. Communal property is not their favorite topic. Apple could readily fix this (see below) but their music partners would likely object.

There's a workaround. I used something similar to this in the past and it works reasonably well if you have a mature music collection and are not adding a ton of new music.

User 1 moves iTunes Library Music to User 1 Public Folder (iTunes will do this for you if you change Library location then tell it to consolidate). Copy User 1 iTunes database there to for the moment using the Finder (ignore the xml file, iTunes recreates that).

User 2 copies the iTunes database file to their default location, deleting their old database. Start iTunes. Change advanced options to uncheck import into iTunes when adding music.

User 2 now switches to their account when syncing. They set ratings, etc. They have their own Playlists.

What about new music?

User 1, Administrator, handles all new music import.

After new music is added Administrator switches to user 2 account (need to know pw of course, otherwise user 2 has to do this) and uses Finder to navigate to newly added music.

Administrator starts user 2 iTunes and drags and drops new music folder into user 2 iTunes.

iTunes is very good at adding references to the new music, but it does not move the music into iTunes because we unchecked the 'move into library' option.

Note that User 2 cannot delete music (read-only privileges on Public folder). User 1 can delete, but User 2 will have dangling references. So, periodically Administrator must delete dangling references in shared libraries, though in fact they don't really do anything -- they just look unsightly. (I think there's an AppleScript that might do this.)

OS X users and groups: not much progress since 2001!

I have a scheme in mind to get around the DRM-constrained design "flaws" in iTunes for a multi-user family.

The first step is to create a shared folder that's read-only for most of the family. That's where the iTunes music files will live. Problem is, that's not easy to do in OS X. As near as I can tell this 2001 article is still true today: The Mac Observer: Hot Cocoa - Unlocking The Power Of Groups In Mac OS X. I think there are some more GUI utilities one can buy (ex: Sharepoints), but Apple has not addressed this -- probably because this is the province of OS X server. Mac Classic was far more useful as a home file server.

Microsoft did the same yucky thing; in many ways Windows 95 was a better fileshare than Windows XP Pro. In this case, though, Apple is even worse than Microsoft.

Anyway, getting the file share working is the least of my concerns! I can do the command line stuff or use sharepoints. This will be interesting ...

Update 2/27/08: I put my iTunes Library in my Public folder, which grants read access to everyone, but no write access. The Shared folder grants read and write access to everyone. That seemed to work, but I discovered that iTunes actually uses a mixed model to store metadata. Some is stored in the internal database, some stored in the local file system. It's complex, I think there are some significant deep design problems with this in iTunes. I need to experiment with granting write access to one user but not the children. Looks like I'll be donating money to Sharepoints if it works!