FileMaker 10: if only they'd provided support for iCal and Address Book

Bento allows direct access to the SQLite tables that support iCal and Address Book.

FileMaker Pro 10 doesn't.

So sad. Bento is worthless, but earlier versions of FM weren't bad. If FM 10 had direct support for SQLite work on iCal and Address Book I'd have bought it in a heartbeat.

It does make me wonder what the heck they were thinking.

I try NuevaSync on my secondary iPhone

I've been holding off on NuevaSync. I liked the idea, but I wanted to wait until they were out of beta. It didn't help my confidence that NuevaSync, maybe deliberately, has been keeping a minimalist (amateurish really) web presence. (BTW, "neuvasync" is a typo/phishing site, so type carefully.)

So I've been waiting for about five months, even as MobileMe festered and I've grown disenchanted with Apple's iPhone and Apple's iCal alike.

Now I'm relatively fond of Google, I'm about ready to try anything that's not Apple, and I've been receiving kind comments encouraging me to try the Google/NuevaSync alternative.

So I decided I'd try NuevaSync out on our pseudo-iTouch. If it works there, I'll look at wiping what's on my iPhone. (When you switch to a MobileMe or Exchange account on an iPhone existing data is purged.)

NuevaSync puts an Exchange Server front end on Google Contacts and Google Calendar (gCal).

The spartan details can be found off of NuevaSync - Over the Air Synchronization once you've setup a free beta account (this article has much more detail, though it's dated). When NuevaSync asks for your "Google ID" they mean your Gmail email address. You don't give NuevaSync your Google password, instead you're taken to a Google page where you can give NuevaSync special read/write privileges.

When setting up your phone pay attention to the settings, esp time zone (off) and SSL (on). During initial setup the domain is left blank, so you have to wait for the phone to time out and ask for a server name.

My initial configuration worked well for the Calendar; I'm seeing all of the Google Calendars that belong to the account I used. NuevaSync is recognized by the phone as Push provider, be sure you have data updates turned on. (Mine was off to save battery life).

Contacts didn't work as well. Nothing came through.

I checked the NuevaSync Status page and Sync status page and found this message:

Your Google account contains bad data, probably added by a third-party sync application. We recommend exporting and re-importing your Google contacts to clean up the data.

Right. Have I mentioned that Synchronization is Hell? Glad I am that I'm bloodied enough to try this first on the "iTouch".

Yeah, there could be "bad data", or there could simply be legal data attributes that NuevaSync isn't expecting, or character sets it doesn't like. "Exporting and re-importing" would be quite a bit easier for a data geek like me if, for example, there were a data exchange format that preserved carriage return/line spaces in text fields.

To be continued ...

Update: The NuevaSync team doesn't rest. When I emailed them about the Contacts problem they replied Saturday night:

... The data in this case is 'bad to the bone' in that it is blowing up inside Google's own client library...

In other words they can't work around the problem since Google's code is in control -- unless they can bypass use of the Google client code. They did suggest looking for odd characters in email addresses (semi-colons, etc). So I might try exporting the data and then looking for the badness that way.

Update 2/9/09: A bad day for Nuevasync. Damn, I feel for them.

Look for the Sync topic on Google's iPhone Devices page.

Google's approach to Contact Sync is to first sync via iTunes to Google, then turn on the Exchange features and wipe out the phone. You'd backup first of course. If you want Address Book on OS X to then reflect the phone/Google you'd need to sync it to Google (Spanning Sync, for example).

I'll try this on my iTouch/iPhone device.

Update 2/9/09: Google's exchange ActiveSync service worked, including the Address Book. I am genuinely sad for the NuevaSync team.

Synchronization case study: Address Book, the iPhone, and Google Contacts

The following is excerpted from the documentation for Spanning Sync, an application that synchronizes my OS X and iPhone Address Book contacts with Google's contacts. Read it and despair, but remember the fault is not in Spanning Sync ...

Spanning Sync - Sync iCal and Google Calendar

... The first option, “Preserve Address Book data” is recommended because it guarantees that names and street addresses in Address Book maintain their formatting. For example, a first name "Vincent" and last name "Van Gogh" doesn't become first = "Vincent", middle = "Van", last = "Gogh". The downside is that the extra formatting may keep autocompletion from working as expected in Gmail ... When editing contact names and postal addresses, do so in Address Book and not in Gmail if you're using the Address Bookcompatibility mode (recommended). Deleting these fields in Gmail is fine, as is editing other fields such as email addresses and phone numbers... Google does not allow multiple contacts to have the same email address. If you have contacts in Address Book that share the same email address, Google will reject all but one of them and Spanning Sync will notify you of the conflict.

Yes, Synchronization is Hell. Google and OS X Address books have different data models for a Contact. Full reliable synchronization is not possible.

Of course on can more or less sync Address Book Contacts with MobileMe, but MobileMe calendaring is unacceptably feeble so that's not an acceptable option. Attempting to sync Outlook contacts will produce lesser but similar conflicts -- at least until 10.6 OS X adopts Exchange Server's definition of a Contact.

I wonder if anyone at Apple understands this. I imagine someone did, but when he tried to explain the problem to Jobs he was banished to the Dark Tower ...

See also: Why you can't get your work calendar on your iPhone

Update: Read the comment from Charles Wood, one of the founders and developers of Spanning Sync. There's a reason I'm a Spanning Sync customer -- I appreciate that they tackle this terribly hard problem. Dang, but we're at a frustrating point in the general problem of 'synchronization' and data exchange. I can only hope understanding of the fundamental problem is percolating from engineering up to the executive suite ...

Office 2007 - the bizarre bits

I've been using Office 2007 on XP for a few months.

My overall impressions are:

1. Excel: few real changes, maybe slightly improved
2. PowerPoint: Using themes in PPT 2003 mode is ... a recipe for madness.
   
3. Word: mostly better, but some odd new gaps
4. Access: buggy and mostly worse for what I do (data analysis, manipulation)
5. Outlook: almost every significant new feature is broken, some hideously broken. For heaven's sake, don't try the feed reader sync with IE 7 on a non-trivial feed list -- unless you like to see your machine meltdown. Some ancient bugs were fixed.
   

If you're on Office 2003 you should wait about 2 months post Office 2007 SP2.

The main benefits of Office 2007 seem to be that it does work, awkwardly, with Sharepoint 2007 document management, whereas Office 2003 is a total fail. (Non-Office applications are even worse, which suggests that the European anti-trust effort has finally collapsed.)

Which brings me to the bizarre stuff I keep running across then forget about. I'll put one in one or two then add more. These aren't bugs -- those are too numerous (esp in Outlook 2007) to list. These aren't rants on the half-baked user-customizeable quick reference bar or the new UI (which is dumb but I'm used to it now), these are just bizarre design choice.

1. Word tables: There's a whizzy new table creator, but we've lost control over row and cell attributes. Try to copy table cell shading from one cell to another cell. The table overall didn't work; the Office team should figure out what went wrong and fix it.
2. Word auto-format: Even more intrusive, though less often wrong.
3. In Outlook 2007, unlike 2003, you can't
   
4. PowerPoint "Theme" revision - at least when used in PPT 2003 compatibility mode. Oh, my, Lord ...

Update 11/4/09: The more I use it, the more I wish Microsoft would go away (they won't)

1. In Outlook there's no visual indicator to differentiate between blocked times (appointments) and meetings (has attendees). When I look at my calendar I'm not sure whether I'm late for some work I set for myself or if I've got people waiting for me.
2. I hate, hate, hate that I can't get PowerPoint to open windows on more than a single monitor. (To be fair, I think Microsoft screwed this up ages ago).

See also:

• Lessons from Outlook 2007 Notes colors
• Reverting from Office 2007 to Office 2003: MAPI32.DLL vs. MSMAPI32.dll
• Office 2007 Diagnostics includes memory and disk check tools

Gmail tasks grow up - slightly

I've been rather unimpressed with Google/Gmail Tasks. They remind me of Apple's disastrous Mail.app tasks, except they don't even have categories.

They're not dead yet though ...

Official Gmail Blog: Tasks: Paper vs. iPhone

We set out to fix this by making Tasks available from your phone with a version optimized for the small screen. And starting today, you can manage your task list from your iPhone or Android device...

... Just go to gmail.com/tasks from your phone's browser and log in. If you already use the version of Tasks in Gmail Labs, you'll see the same task list that's always in sync...
... so try it out and let us know.

P.S. There's a new gadget version of Tasks too -- so if you want to add your same task list to iGoogle, now you can.

I put my request list into the let us know bucket:

• an API
• Calendar integration (esp. something like this)
• The "palm to do" minimal attribute set

If they get the API and calendar integration I'd love to see Appigo's iphone ToDo.app sync to Google Gmail tasks instead of to Toodledo.

Using Bloggers undocumented label (category) feeds and Yahoo Pipes to create a tech opinion feed out of Gordon's Notes

[3/14/09: See Update below. I loved how this worked, but I had to give up on it. If blogs survive, maybe Google will add Pipes-like features properly.]

Dang. What's a geek got to do to be fully informed these days?

I've only now learned that my 2nd biggest Blogger feature request, feeds on Blogger "labels" (aka categories, sometimes called tags), has been unofficially available for at least a year ...

Blogspot ... Feeds by Category / search labels

Here's the default URL for any search label in Blogger:

xxx.blogspot.com/search/label/labelname

To convert that search label into a Blogger ... feed, replace the string /search/label/ with /feeds/posts/default/-/ so the new feed URL for that search label has the following syntax:

xxx.blogspot.com/feeds/posts/default/-/labelname

For this to work, you should have enabled Site Feeds for that blogs. Blogger Label Feeds will inherit the full or partial text settings from the main feed itself.

A year! I could have easily solved the problem I flailed at only 5 months ago.

The problem is I my opinion blog, Gordon's Notes mixes both technical and liberal political opinions. The technical opinions might fit in with tech.kateva.org but the political opinions won't. In any case, I like to keep this site to the apolitical "facts", so even technology opinions are a bit out.

Now I can organize feeds however I like, using this undocumented feature and Yahoo Pipes.

To start with, here's the label for "technology"

http://notes.kateva.org/search/label/technology

and here's the feed (Atom) following the above pattern

http://notes.kateva.org/feeds/posts/default/-/technology

It's a well formed feed, except that it doesn't get a useful name like "Gordon's Notes - technology"; it just gets called "Gordon's Notes".

That's good, now we add Yahoo Pipes (best thing they've ever done).

At Yahoo Pipes I can merge multiple GN opinion feeds (Case matters, spaces are sort of ok but, you know, spaces are always a pain)

http://notes.kateva.org/feeds/posts/default/-/Apple
http://notes.kateva.org/feeds/posts/default/-/data lock
http://notes.kateva.org/feeds/posts/default/-/drm
http://notes.kateva.org/feeds/posts/default/-/Google
http://notes.kateva.org/feeds/posts/default/-/Google Apps
http://notes.kateva.org/feeds/posts/default/-/netbook
http://notes.kateva.org/feeds/posts/default/-/os x
http://notes.kateva.org/feeds/posts/default/-/palm
http://notes.kateva.org/feeds/posts/default/-/spam
http://notes.kateva.org/feeds/posts/default/-/technology
http://notes.kateva.org/feeds/posts/default/-/xanadu

The feed summation is pretty trivial ...

After I save and rename the YP feed I end up with:

http://pipes.yahoo.com/jfaughnan/gn_tech

Google Reader treats this as a proper feed. So, after years of blogging, I'm now able to do my own syndication.

This will be fun.

Update 2/7/09: Since I had to treat each label as a separate feed, and since a post can have multiple labels, the above design produced a lot of duplicates. It also had no sort order. I added the "Unique" (by ID) and "Sort" (pub date) operations to fix those problems.

I've also created a new feed that takes the the pipe for http://pipes.yahoo.com/jfaughnan/gn_tech and then adds (Union) the feed from Gordon's Tech (this blog) and sorts by publication date. I didn't need another unique operator because I wasn't creating more duplicates.

This new feed combines all my posts from Gordon's Tech with my technology opinions posted in Gordon's Notes. It's

http://pipes.yahoo.com/jfaughnan/tech_plus

and the new feed is

http://pipes.yahoo.com/pipes/pipe.run?_id=CGBvCFr13RG4ekbA1JzWFw&_render=rss

I might add this to my post footer but I've noticed two limitations of this type of feed.

• The posts sort correctly when I view the generated Pipe web view, but the feed from that view gives all posts close to the same pub date.
• The feed does not inherit the title set in properties, it gets the URL string.
  

Update 3/8/09: hmm. There might be a serious problem here. Lately Google isn't indexing my stuff as well as they used to. I've seen that before when Google gets confused by splogs. It indexes splogs that have harvested my posts, then it starts treating my blog as though it were also a splog. Usually it self-corrects after a few days but this time it seems to persist.

Alas, by creating Pipes from my feeds I wonder if I've created my own de facto splogs. I tried to submit a question to Yahoo about making these non-indexed but the submit forum crashed. I think I'll have to take these offline. Unfortunately it's very hard to learn Google's policy on these things.

Update 3/15/09: I never did get an answer on my Yahoo or Google Groups questions. I did find that I was getting odd results from subscribing to the Pipes I created. Google Reader persistently showed me old posts as new, probably because when a Yahoo Pipe is generated all the source posts get the same date.

In addition, even though I "unpublished" the Pipes I linked to them from various posts, and I think Google was finding and indexing them. I don't see how Google could differentiate them from spam blogs, and we know that the GoogleMind associates splog taint with the parasitized victim. (The GoogleMind is a harsh and unforgiving Yahwhite god.)

Yahoo Pipes needs a no-index tag and it needs to handle post dates differently.

I didn't want to delete my Pipes in case I wanted to work with them in the future. So I simply changed the URLs to something new and thus invisible.

Prediction: We won’t see a fixed MobileMe until 10.6 is out

I usually keep my editorial opinions off my tech blog, and post them in Gordon’s Notes (tech) instead. This one’s a bit of an exception.

I’ve been spinning cycles thinking about why Apple’s MobileMe and synchronization solutions are so miserable – and why they’re not improving.

Incompetence alone is not an explanation. Yes, Apple uses early adopters as beta testers, and yes, the MobileMe launch demonstrated Apple has serious internal issues, but they’ve still got good people.

Marketing is not an explanation. I can’t imagine any reason why Apple would not want MobileMe calendaring and synchronization to be a wonderful experience that would tie their desktop and phone customers ever more closely to Apple.

That leaves one explanation. Synchronization Hell.

The fundamental problem with “synchronization”* is that computers are not yet super-sentient. If a transaction hub had full knowledge of the world, and an IQ of 150, and lots of time to think things over, it could figure out how to translate an “all day event” represented as an attribute on a day to one represented as a “24 hour event” with a buggy representation of time zones and daylight savings time.

We’re not there yet.

It’s impossible, in the sense of mathematically not possible, to perfectly migrate data between systems that represent the semantics (meanings) of the data differently. Small discrepancies are amplified with each synchronization. Things get left out. Not only are the results inexplicable to users, they’re inexplicable to engineers.

In Apple’s case they’re deeply stuck. They have desktop systems running 10.4 and 10.5 that they’re trying to support, each with somewhat different models for calendar items, contacts, and tasks. (Let’s not even mention the horrors of the Mail.app vs. iCal.app task fiasco). The iPhone has yet another data model for Calendar and Contacts, not to mention Notes (but no Tasks). The iPhone has to “sync” with Outlook (when Exchange is not present) and iCal/Address Book (OS X) and MobileMe and Exchange Server.

The solution is to surrender to Microsoft and accept the Exchange Server model of the world. Yes, Microsoft’s monopoly is not dead.

So in OS X 10.6 everything adopts the Microsoft Exchange Server data and semantic model of Contacts, Tasks, Notes, Calendar events, etc. At that point MobileMe, the iPhone, and the OS X desktop will align. That’s probably good news, though users of PPC machines may be out of luck**.

“Snow Leopard” is probably coming out later this year. I don’t think MobileMe and the iPhone will give us the basic PDA functions we need until after 10.6 is widely available.

--

* In the world of HealthCare IT, where I live during daylight hours, the equivalent words are “mapping”, “messaging” and “interoperability”.

** It’s generally assumed that 10.6 will be Intel only. I would not be surprised if Apple has not yet made a final decision.