Friday, November 28, 2008

Entourage perspective on OS X Sync Services

Ths article by a Program Manager in Microsoft's Mac Business Unit gave me some new insights into OS X sync services. It illustrates how very hard it is to do synchronization of any sort between disparate products. Perfection is impossible, the best vendors can do is reduce the outrage rate.

Synchronization is Hell.

Andy Ruff includes a discrete link to a section of Apple's developer documentation on the Sync Services Truth Database. To the bloodied sync veteran every word in these paragraphs could be written in blood (Entourage, according to Andy Ruff, defined a schema for Notes since Apple didn't have one. I find Apple's approach to schema definitions interesting.) ...

... The truth database contains an aggregate of all the client’s records. Consequently, the truth database uses a canonical schema that is an aggregate of all the schemas used by all the clients.

A sync schema is based on an entity-relationship model similar to that used by other Cocoa technologies. Read Cocoa Design Patterns in Cocoa Fundamentals Guide to learn more about entity-relationship models and terms such as entity, property, attribute, relationship, to-one, and to-many.

You can use one of the existing sync schemas—for example, for contacts, calendars, and bookmarks— extend one of these schemas or create your own. If you extend a schema or create your own, then you need to create an entity model for your custom objects and save it in a schema format that Sync Services understands.

This format, called sync schema, is a property list that specifies details about the entities in your model. For entities, you might specify the name of the entity and names of its attributes and relationships (collectively referred to as its properties). For attributes, you might specify the name of the attribute and its data type. For relationships, you might specify the name, destination entity name, cardinality, and delete rule. See “Creating a Sync Schema” for a complete description of the sync schema property list.

A sync schema defines a template for records stored in a database whose records are of a particular type (records belong to an entity) and may have relationships to other records. Records stored in the truth database are dictionary objects with key-value pairs, one for each property defined in the entity. Each record dictionary also has an entity name property and an associated unique record identifier. The record identifier is not stored with the record but is instead used by the client and sync engine when referring to a record. The truth database can also store custom fields in a record that are not defined in the schema. For example, these fields can be used to store client information added by a device.

The truth database doesn’t store arbitrary key-value coding-compliant objects—it stores record dictionaries. Therefore, unless all your entities are dictionaries, you typically transform records back and forth between the sync engine’s record representation and your client’s object representation. However, when fast syncing, you can apply changes only to properties—you don’t need to push and pull entire records when only a few property values changed.

Because the truth database is an aggregate of all the client schemas, it can contain a lot of information that your application doesn’t care about. Your client can filter the records that it pushes and pulls in several different ways...

The more I suffer with Apple's approach to Calendars, Tasks (phhhtt!) and Notes/Memos the more I'm interested in Entourage. The problem is there's no way for a 3rd party to sync directly with the iPhone Calendar. Microsoft would have to create a complete Entourage-friendly iPhone PIM suite, and I rather doubt they're going to do that.

Update: Macintouch has a great, meaning despair-filled, thread on Entourage to iPhone synchronization. Synchronizing disparate data models is not a soluble problem. Even a deity-equivalent AI can't perfectly reconcile disparate data models.

Chapura KeyTasks for the iPhone

Chapura is offering an iPhone Task management application tied to an online "cloud" service.

If you're an Outlook user you can sync your tasks from Outlook to the "MyChapura" cloud service, and from the cloud service to your iPhone/iTouch -- even if you sync Outlook with Exchange Server. (Unlike MobileMe, which has been changed to no longer sync with Outlook when exchange server is involved.)

As far as I can tell the MyChapura cloud doesn't include any user interface for manipulating the Task data, it's strictly a store that gets around all the issues with syncing an iPhone to multiple machines and the inability to sync over the Apple calbe.

Grrr. Maybe I should have left all my PIM data in Outlook -- rather than moving it to the pathetically inferior OS X environment. (Only Address Book is in any way competitive to the Outlook PIM equivalent.)

So is Chapura going to go all the way, and add Calender, Contact and Memo/Notes support to MyChapura, with complementary apps on the iPhone? That would be a large project, but it's very much like their Palm KeySuite application.

I'm definitely interested; it's a shame Chapura doesn't have a blog I could track. Tasks synchronization alone isn't worth the risk of installing software into Outlook/XP (very unstable environments these days), but if Chapura goes further ...

Modern software development on the web

This is a well done essay on model-view-controller development that happens to also talk about the web: Advogato: Blurring of MVC lines: Programming the Web Browser.

If you want to catch up a bit on current programming styles (I'm a bit behind myself) the Wikipedia article on RESTful development is a good complement. This bit of the article is particularly helpful, especially if you know that "RPC" is often used to refer to SOAP services, that Dave Winer was a champion of SOAP and RSS alike, and that Atom was championed by Google over RSS (phew!) ...

... It is possible to claim an enormous number of RESTful applications on the Web (just about everything accessible through an HTTP GET request or updateable through HTTP POST). Taken more narrowly, in its sense as an alternative to both Web Services generally and the RPC style specifically, REST can be found in a number of places on the public Web:

Note that WSDL version 2.0 now offers support for binding to all the HTTP request methods (not only GET and POST as in version 1.1).[6]

I think of Jon Udell as a godfather of the RESTful world, but I had trouble finding a review essay of his on the topic; his early discussions are fairly brief. I've got a very good reference somewhere, I'll add it here when I find it....

Missing JPEG preview icons restored by Onyx rebuild LaunchServices

This is probably not a new 10.5 bug, now that I know the fix I can find mentions of similar problem from five years ago.

OS X 10.5 Leopard will, like XP's thumbnail view, show a preview of a JPG, PDF or other image file in place of the large icon.

In theory.

In practice it's prone to losing the preview and just showing the generic icon. As per this discusison thread and an older thread that has the real advice the OS X launchServices database is fragile and often broken. (For example.)

One fix is to run a fairly complex terminal command that rebuilds that too fragile data source. Another is to download Onyx and run the rebuild launchServices and cleanup scripts (remove caches) then restart.

That restored my icon previews.

Another few hours lost to complexity collapse.

I'd really like to know why Apple hasn't either fixed this ancient bug or included a launchServices rebuild in the monthly maintenance script OS X runs.

I sent Onyx a $10US donation. I've used this app long enough to contribute something.

Monday, November 24, 2008

Using my business card as iPhone wallpaper

I have the world's dullest iPhone wallpaper -- my business card with my phone number and personal email written in big ink letters.

I used to have a photo of the dog (that way the kids don't accuse me of favoritism), but I realized that if the phone was lost there was nothing to point to me. I set the phone to lock after a few minutes without use, so anyone finding it can't get through to my phone list (for example).

Palm devices, and probably most devices, have a security option to show a user-defined text screen when locked. Not so the iPhone! [1]

So I used my Griffin Clarifi macro lens and took an iPhone picture of my business card, with annotations. Then I set the business card as wallpaper. I framed it so the key information would be easy to see.

So if my phone is lost, it's now easy for someone to return it to me.

I'll try making a better picture with my dSLR, but really the card picture is quite readable even in the wallpaper version.

[1] Yes, I too would like to lock the iPhone team in a room with a Palm Vx for a month until the good bits of that device/software package were permanently burned into their brains.

Sunday, November 23, 2008

Has iPhone 2.2 created a new class of application update problems? Remote is broken now.

At first we thought 2.2 had fixed an old problem ...
Gordon's Tech: iPhone 2.1 - unknown application 0xE800002E - A FIX!

...Update 11/22/08: The Nov iTunes and iPhone releases may have squashed this one. I can't speak from personal experience since I long ago fixed my problem, but I'm seeing positive reports....
Now I've had to manually remove and reinstall 2 applications, MPR and Remote, to get them to work - partly.

Just like the old days, except now there's no application error.

Remote is no longer showing me controls for my remote airport express speakers. That's a slight improvement, before the forced restore it was just giving me an "unknown error" message when I tried to use it. I think that's tied to the airport express problem.

This is not looking so good.

If it is a return of the old horrible bug with application updates, I suspect the return may have root in Apple's broken identity management system. It broke with the .mac to .me transition for those who had been original .mac users but were not users at the time of the .me transition ...

Update: I remain suspicious that there's a new bug, or a twist on an old bug, with propagating app updates to the iPhone. On the other hand, the loss of remote speakers was due to my Airport Express going offline. When I power cycled it I again saw the remote speaker option.

Why did the AE go offline for the first time in a year or so? That's another question, maybe related to an update to my Airport Extreme, maybe coincidental.

Update: Yep, I can't sync my iPod now...

Update: Deauthorizing and reauthorizing my Mac had no effect. I found a slightly relevant discussion group thread. I rebooted the iPod to no effect, but then restarted my Mac and was able to sync all my devices.

Update 11/24/08: Rebooting the Mac solved the problem. Alas, that was too easy a solution for this to be entirely an Apple error. I have a bad feeling that the error was intracranial, that is, my user error.

For reasons too complex to bother explaining it's just barely possible that I was running a user account that points to a shared multi-user Tunes library.

I've also verified that at least one iPhone App Store update worked as expected and functioned without my having to remove and restore the app.

Saturday, November 22, 2008

Resolving the Windows Live Installer Catastrophic Failure bug

In the process of tracking down the root of my Outlook 2007 "missing manifest" bug I discovered that Windows Live Installer was failing with a "catastrophic error". (Love that! It's not as good as "Abort, Retry, Fail" but, for the anemic Microsoft of modern times it's not bad.).

I went down this root because the nature of the missing manifests suggested the bad behavior was related to either Office Communicator or Live Messenger. Installing Communicator didn't fix things, so Live Messenger was up next. Trying to install Live Messenger led to my "catastrophic error".

Others have run into this problem with Live Suite installs. That page pointed to the beta version of Windows Live apps as the problem. So it was with me. I used the Add/Remove control panel Windows Live Beta installer to remove all my Windows Live products. (Some had been updated to non-beta status by Windows Update, but it appears that wasn't enough.)

After that I was able to run the Windows Live Installer normally and to add them all back in.

So now on the Manifest bug ...