Google saves my iPhone

Ok, so I don't know if it works, and I do feel sorry for the Nuevasync team.

Still, I'm grateful for Google's Exchange ActiveSync service service. With MobileMe hopeless and near worthless, and my iPhone love broken-hearted, things were looking pretty damned bleak -- until today.

Perhaps in honor of this launch, Google has an iPhone device page with lots of Google related topics in one place. The Sync set is towards the bottom of the page ...

iPhone Devices - Mobile Help

... Sync

• Overview
• Set Up Your iPhone or iPod Touch
• Choose Which Calendars to Sync
• Can I Still Sync with iTunes?
• Contacts Upload via iTunes (PC)
• Contacts Upload via iTunes (Mac)
• Synchronizing Contacts or Calendar
• Contact Search Feature
• Data Disappears During Re-Sync
• Known Limitations

No doubt about it. Google loves me. I ain't crawling back to Apple no more.

Update: Google licensed ActiveSync from Microsoft for this. I assumed they'd cloned it. I wonder what a ten million user license of ActiveSync costs? I don't imagine Microsoft gave Google much of a discount. It's an amazing testimony to the power of Microsoft's Exchange monopoly, and a marker for how serious Google is about making this work.

Update: It worked on our iPhone touch -- Calendar and contacts alike. I then wiped everything from the iTouch and proved I could sync just the calendar, and leave the contacts alone. That worked. Then I chose my sync calendars (config site is http://m.google.com/sync, you must visit it from an iPhone). I actually ran up against the 5 calendar limit (my work, emily calendar, my personal, MN Special Hockey and US Holidays), but that's good for now. The 5 calendar limit appers to be related to an iPhone bug.

So next I will sync and backup my iPhone and create special backups of my OS X Address Book and Calendar. Then, for now, I will turn off Spanning Sync while I do my testing -- so OS X iCal will no longer be connected to my true calendar (no loss).

Update: After backing up as above I'm on to my iPhone. It was already setup to do IMAP sync with my Gmail account; I added the Exchange server connection. I got the "invalid certificate" warning during Exchange setup -- that's a known bug. I turned Exchange Mail OFF, Contacts OFF (for now) and Calendars ON. I received the warning that existing calendars would be removed from my iphone.

After setup my Mail, Contacts, Calendar settings showed two accounts: one for Mail and one for Calendars. Although NuevaSync recommends turning Time Zone support OFF for Calendars Google didn't make any recommendations, so I left it ON.

I checked Calendar, and saw one calendar was synchronized. That was curious, since I'd configured the iTouch I'd specified 5 calendars.

Here's where things get interesting. When I visited the setup page I found configurations for two devices.

So you can sync multiple devices to a single set of Calendars, and you can configure separately which calendars each one syncs to. [or maybe not - see update]

Interesting ...

Anyway, so far it works.

Update: This is so cool. I play around with an item time on my iPhone, and moments later it's switched on my wife's BB Pearl (yech) and her Google Apps Calendar (yay). I'm holding off on Contacts for now; the Google contact model is pretty sparse. I want to give that migration a bit of thought. With Google for Calendaring and soon for Contacts I can live with Todo.App, Notebook.app and Evernote for a while longer.

Google has Gmail Tasks now. How long before they, or someone else, provides an iPhone Task app that will sync with Google Tasks? If they build out the data model I'd love to see Appigo selected for the iPhone app.

Update 2/10/09: Ok, maybe it wasn't a good idea to sync two iPhones to one account. When I go to http://m.google.com/sync I can't change which five calendars I sync to.

Update 2/14/09: It may be coincidental, but a day or two after I discontinued sync with the 2nd iPhone I was again able to edit my subscription limit (still only 5 calendars pending bug fixes). Incidentally, now that I'm getting Push Calendar updates I see why people complain about the iPhone's battery life. OS X wasn't built to be a power miser.

Update 5/22/09: Every few months the calendar seems to stop updating. I turn off "calendars" in the iPhone Exchange ActiveSync screen; that removes all calendars from the phone. Then I turn it on again. They then update normally.

FileMaker 10: if only they'd provided support for iCal and Address Book

Bento allows direct access to the SQLite tables that support iCal and Address Book.

FileMaker Pro 10 doesn't.

So sad. Bento is worthless, but earlier versions of FM weren't bad. If FM 10 had direct support for SQLite work on iCal and Address Book I'd have bought it in a heartbeat.

It does make me wonder what the heck they were thinking.

I try NuevaSync on my secondary iPhone

I've been holding off on NuevaSync. I liked the idea, but I wanted to wait until they were out of beta. It didn't help my confidence that NuevaSync, maybe deliberately, has been keeping a minimalist (amateurish really) web presence. (BTW, "neuvasync" is a typo/phishing site, so type carefully.)

So I've been waiting for about five months, even as MobileMe festered and I've grown disenchanted with Apple's iPhone and Apple's iCal alike.

Now I'm relatively fond of Google, I'm about ready to try anything that's not Apple, and I've been receiving kind comments encouraging me to try the Google/NuevaSync alternative.

So I decided I'd try NuevaSync out on our pseudo-iTouch. If it works there, I'll look at wiping what's on my iPhone. (When you switch to a MobileMe or Exchange account on an iPhone existing data is purged.)

NuevaSync puts an Exchange Server front end on Google Contacts and Google Calendar (gCal).

The spartan details can be found off of NuevaSync - Over the Air Synchronization once you've setup a free beta account (this article has much more detail, though it's dated). When NuevaSync asks for your "Google ID" they mean your Gmail email address. You don't give NuevaSync your Google password, instead you're taken to a Google page where you can give NuevaSync special read/write privileges.

When setting up your phone pay attention to the settings, esp time zone (off) and SSL (on). During initial setup the domain is left blank, so you have to wait for the phone to time out and ask for a server name.

My initial configuration worked well for the Calendar; I'm seeing all of the Google Calendars that belong to the account I used. NuevaSync is recognized by the phone as Push provider, be sure you have data updates turned on. (Mine was off to save battery life).

Contacts didn't work as well. Nothing came through.

I checked the NuevaSync Status page and Sync status page and found this message:

Your Google account contains bad data, probably added by a third-party sync application. We recommend exporting and re-importing your Google contacts to clean up the data.

Right. Have I mentioned that Synchronization is Hell? Glad I am that I'm bloodied enough to try this first on the "iTouch".

Yeah, there could be "bad data", or there could simply be legal data attributes that NuevaSync isn't expecting, or character sets it doesn't like. "Exporting and re-importing" would be quite a bit easier for a data geek like me if, for example, there were a data exchange format that preserved carriage return/line spaces in text fields.

To be continued ...

Update: The NuevaSync team doesn't rest. When I emailed them about the Contacts problem they replied Saturday night:

... The data in this case is 'bad to the bone' in that it is blowing up inside Google's own client library...

In other words they can't work around the problem since Google's code is in control -- unless they can bypass use of the Google client code. They did suggest looking for odd characters in email addresses (semi-colons, etc). So I might try exporting the data and then looking for the badness that way.

Update 2/9/09: A bad day for Nuevasync. Damn, I feel for them.

Look for the Sync topic on Google's iPhone Devices page.

Google's approach to Contact Sync is to first sync via iTunes to Google, then turn on the Exchange features and wipe out the phone. You'd backup first of course. If you want Address Book on OS X to then reflect the phone/Google you'd need to sync it to Google (Spanning Sync, for example).

I'll try this on my iTouch/iPhone device.

Update 2/9/09: Google's exchange ActiveSync service worked, including the Address Book. I am genuinely sad for the NuevaSync team.

Synchronization case study: Address Book, the iPhone, and Google Contacts

The following is excerpted from the documentation for Spanning Sync, an application that synchronizes my OS X and iPhone Address Book contacts with Google's contacts. Read it and despair, but remember the fault is not in Spanning Sync ...

Spanning Sync - Sync iCal and Google Calendar

... The first option, “Preserve Address Book data” is recommended because it guarantees that names and street addresses in Address Book maintain their formatting. For example, a first name "Vincent" and last name "Van Gogh" doesn't become first = "Vincent", middle = "Van", last = "Gogh". The downside is that the extra formatting may keep autocompletion from working as expected in Gmail ... When editing contact names and postal addresses, do so in Address Book and not in Gmail if you're using the Address Bookcompatibility mode (recommended). Deleting these fields in Gmail is fine, as is editing other fields such as email addresses and phone numbers... Google does not allow multiple contacts to have the same email address. If you have contacts in Address Book that share the same email address, Google will reject all but one of them and Spanning Sync will notify you of the conflict.

Yes, Synchronization is Hell. Google and OS X Address books have different data models for a Contact. Full reliable synchronization is not possible.

Of course on can more or less sync Address Book Contacts with MobileMe, but MobileMe calendaring is unacceptably feeble so that's not an acceptable option. Attempting to sync Outlook contacts will produce lesser but similar conflicts -- at least until 10.6 OS X adopts Exchange Server's definition of a Contact.

I wonder if anyone at Apple understands this. I imagine someone did, but when he tried to explain the problem to Jobs he was banished to the Dark Tower ...

See also: Why you can't get your work calendar on your iPhone

Update: Read the comment from Charles Wood, one of the founders and developers of Spanning Sync. There's a reason I'm a Spanning Sync customer -- I appreciate that they tackle this terribly hard problem. Dang, but we're at a frustrating point in the general problem of 'synchronization' and data exchange. I can only hope understanding of the fundamental problem is percolating from engineering up to the executive suite ...

Office 2007 - the bizarre bits

I've been using Office 2007 on XP for a few months.

My overall impressions are:

1. Excel: few real changes, maybe slightly improved
2. PowerPoint: Using themes in PPT 2003 mode is ... a recipe for madness.
   
3. Word: mostly better, but some odd new gaps
4. Access: buggy and mostly worse for what I do (data analysis, manipulation)
5. Outlook: almost every significant new feature is broken, some hideously broken. For heaven's sake, don't try the feed reader sync with IE 7 on a non-trivial feed list -- unless you like to see your machine meltdown. Some ancient bugs were fixed.
   

If you're on Office 2003 you should wait about 2 months post Office 2007 SP2.

The main benefits of Office 2007 seem to be that it does work, awkwardly, with Sharepoint 2007 document management, whereas Office 2003 is a total fail. (Non-Office applications are even worse, which suggests that the European anti-trust effort has finally collapsed.)

Which brings me to the bizarre stuff I keep running across then forget about. I'll put one in one or two then add more. These aren't bugs -- those are too numerous (esp in Outlook 2007) to list. These aren't rants on the half-baked user-customizeable quick reference bar or the new UI (which is dumb but I'm used to it now), these are just bizarre design choice.

1. Word tables: There's a whizzy new table creator, but we've lost control over row and cell attributes. Try to copy table cell shading from one cell to another cell. The table overall didn't work; the Office team should figure out what went wrong and fix it.
2. Word auto-format: Even more intrusive, though less often wrong.
3. In Outlook 2007, unlike 2003, you can't
   
4. PowerPoint "Theme" revision - at least when used in PPT 2003 compatibility mode. Oh, my, Lord ...

Update 11/4/09: The more I use it, the more I wish Microsoft would go away (they won't)

1. In Outlook there's no visual indicator to differentiate between blocked times (appointments) and meetings (has attendees). When I look at my calendar I'm not sure whether I'm late for some work I set for myself or if I've got people waiting for me.
2. I hate, hate, hate that I can't get PowerPoint to open windows on more than a single monitor. (To be fair, I think Microsoft screwed this up ages ago).

See also:

• Lessons from Outlook 2007 Notes colors
• Reverting from Office 2007 to Office 2003: MAPI32.DLL vs. MSMAPI32.dll
• Office 2007 Diagnostics includes memory and disk check tools

Gmail tasks grow up - slightly

I've been rather unimpressed with Google/Gmail Tasks. They remind me of Apple's disastrous Mail.app tasks, except they don't even have categories.

They're not dead yet though ...

Official Gmail Blog: Tasks: Paper vs. iPhone

We set out to fix this by making Tasks available from your phone with a version optimized for the small screen. And starting today, you can manage your task list from your iPhone or Android device...

... Just go to gmail.com/tasks from your phone's browser and log in. If you already use the version of Tasks in Gmail Labs, you'll see the same task list that's always in sync...
... so try it out and let us know.

P.S. There's a new gadget version of Tasks too -- so if you want to add your same task list to iGoogle, now you can.

I put my request list into the let us know bucket:

• an API
• Calendar integration (esp. something like this)
• The "palm to do" minimal attribute set

If they get the API and calendar integration I'd love to see Appigo's iphone ToDo.app sync to Google Gmail tasks instead of to Toodledo.

Using Bloggers undocumented label (category) feeds and Yahoo Pipes to create a tech opinion feed out of Gordon's Notes

[3/14/09: See Update below. I loved how this worked, but I had to give up on it. If blogs survive, maybe Google will add Pipes-like features properly.]

Dang. What's a geek got to do to be fully informed these days?

I've only now learned that my 2nd biggest Blogger feature request, feeds on Blogger "labels" (aka categories, sometimes called tags), has been unofficially available for at least a year ...

Blogspot ... Feeds by Category / search labels

Here's the default URL for any search label in Blogger:

xxx.blogspot.com/search/label/labelname

To convert that search label into a Blogger ... feed, replace the string /search/label/ with /feeds/posts/default/-/ so the new feed URL for that search label has the following syntax:

xxx.blogspot.com/feeds/posts/default/-/labelname

For this to work, you should have enabled Site Feeds for that blogs. Blogger Label Feeds will inherit the full or partial text settings from the main feed itself.

A year! I could have easily solved the problem I flailed at only 5 months ago.

The problem is I my opinion blog, Gordon's Notes mixes both technical and liberal political opinions. The technical opinions might fit in with tech.kateva.org but the political opinions won't. In any case, I like to keep this site to the apolitical "facts", so even technology opinions are a bit out.

Now I can organize feeds however I like, using this undocumented feature and Yahoo Pipes.

To start with, here's the label for "technology"

http://notes.kateva.org/search/label/technology

and here's the feed (Atom) following the above pattern

http://notes.kateva.org/feeds/posts/default/-/technology

It's a well formed feed, except that it doesn't get a useful name like "Gordon's Notes - technology"; it just gets called "Gordon's Notes".

That's good, now we add Yahoo Pipes (best thing they've ever done).

At Yahoo Pipes I can merge multiple GN opinion feeds (Case matters, spaces are sort of ok but, you know, spaces are always a pain)

http://notes.kateva.org/feeds/posts/default/-/Apple
http://notes.kateva.org/feeds/posts/default/-/data lock
http://notes.kateva.org/feeds/posts/default/-/drm
http://notes.kateva.org/feeds/posts/default/-/Google
http://notes.kateva.org/feeds/posts/default/-/Google Apps
http://notes.kateva.org/feeds/posts/default/-/netbook
http://notes.kateva.org/feeds/posts/default/-/os x
http://notes.kateva.org/feeds/posts/default/-/palm
http://notes.kateva.org/feeds/posts/default/-/spam
http://notes.kateva.org/feeds/posts/default/-/technology
http://notes.kateva.org/feeds/posts/default/-/xanadu

The feed summation is pretty trivial ...

After I save and rename the YP feed I end up with:

http://pipes.yahoo.com/jfaughnan/gn_tech

Google Reader treats this as a proper feed. So, after years of blogging, I'm now able to do my own syndication.

This will be fun.

Update 2/7/09: Since I had to treat each label as a separate feed, and since a post can have multiple labels, the above design produced a lot of duplicates. It also had no sort order. I added the "Unique" (by ID) and "Sort" (pub date) operations to fix those problems.

I've also created a new feed that takes the the pipe for http://pipes.yahoo.com/jfaughnan/gn_tech and then adds (Union) the feed from Gordon's Tech (this blog) and sorts by publication date. I didn't need another unique operator because I wasn't creating more duplicates.

This new feed combines all my posts from Gordon's Tech with my technology opinions posted in Gordon's Notes. It's

http://pipes.yahoo.com/jfaughnan/tech_plus

and the new feed is

http://pipes.yahoo.com/pipes/pipe.run?_id=CGBvCFr13RG4ekbA1JzWFw&_render=rss

I might add this to my post footer but I've noticed two limitations of this type of feed.

• The posts sort correctly when I view the generated Pipe web view, but the feed from that view gives all posts close to the same pub date.
• The feed does not inherit the title set in properties, it gets the URL string.
  

Update 3/8/09: hmm. There might be a serious problem here. Lately Google isn't indexing my stuff as well as they used to. I've seen that before when Google gets confused by splogs. It indexes splogs that have harvested my posts, then it starts treating my blog as though it were also a splog. Usually it self-corrects after a few days but this time it seems to persist.

Alas, by creating Pipes from my feeds I wonder if I've created my own de facto splogs. I tried to submit a question to Yahoo about making these non-indexed but the submit forum crashed. I think I'll have to take these offline. Unfortunately it's very hard to learn Google's policy on these things.

Update 3/15/09: I never did get an answer on my Yahoo or Google Groups questions. I did find that I was getting odd results from subscribing to the Pipes I created. Google Reader persistently showed me old posts as new, probably because when a Yahoo Pipe is generated all the source posts get the same date.

In addition, even though I "unpublished" the Pipes I linked to them from various posts, and I think Google was finding and indexing them. I don't see how Google could differentiate them from spam blogs, and we know that the GoogleMind associates splog taint with the parasitized victim. (The GoogleMind is a harsh and unforgiving Yahwhite god.)

Yahoo Pipes needs a no-index tag and it needs to handle post dates differently.

I didn't want to delete my Pipes in case I wanted to work with them in the future. So I simply changed the URLs to something new and thus invisible.