Thursday, December 23, 2010

Google IMAP and the intractable duplicate IMAP email problem

I've only recently returned to using; for a couple of years I was almost pure Google. Since I integrated my personal Contacts across OS X Address Book/MobileMe/iPhone/Google Contacts however I've returned to enjoying the power of a dedicated email client.

Alas, there's a catch. In my time away I'd forgotten the Gmail IMAP to sync duplicate email problem. It's not really a specific problem, every IMAP client has the same problem.

This is an intractable problem. The standard model for email is that that every message belongs to exactly one folder. Folders can contain folders. It's very much like traditional directories since the days of DOS and well before. (Tree data structure)

Google has a very different model. Google's mail "folders" are an illusion, in reality all Gmail messages are in a single repository. Each message can have many tags, and a single tag can be used for many messages. Each tag is treated in the UI like a "folder", but a message belongs equally to each "folder' (they're just tags).

This is more than a mere "physics" problem -- it's a math problem. 7 != 3, and a tree-type folder hierarchy cannot represent a tag collection.

There are two approaches to "synchronizing" a traditional email application with Gmail. You could make the first tag of an message the "folder" tag, and ignore the other tags. The other approach, which Google took, is to replicate emails. So a single message in Gmail with 4 tags becomes 4 messages in in 4 folders.

Of course this wastes space, but space is cheap. Much worse, however, is that it clogs up searches.

I don't see this problem going away - unless Google admits defeat and regresses to using standard folders. For now, however, I'd love to see a program that would go through my database and remove all the duplicate emails - even if it randomly assigned them to a single folder.

See also:


Anonymous said...

... I'm using Thunderbird as an IMAP client to GMail and I don't have any problems. Folders work as you'd expect and I don't have any duplicate messages. I'm afraid I don't have much experience with the OSX mail app though (I've pretty much always used TB -- cross platform). Same thing with IMAP and my touch -- no problems. Kick mail to the curb and go with a different client?

JGF said...

Wow. I'm amazed.

Can you try this experiment?

Assign an existing Gmail message of yours four simultaneous tags (they can be anything).

Now sync and view in Thunderbird.

What folder is it assigned to?

Anonymous said...

I see what you're saying now. I just use the traditional folders in TB ... not tags in GMail. When you folder them traditionally in the client, GMail behaves as you would expect it to. Personally, I like the 'tags' feature (and will probably start using them now) ... one thing I do in my client is only subscribe to certain folders and leave others available only through GMail itself (like my archive2010 folder for instance). Good luck!

Anonymous said...

Just saw this and made me think of your post from January ... not sure if it's helpful: