There's no way to make Gmail and OS X Mail.app  into a happily married couple.
The Google Label (very shallow acyclic graph) vs. IMAP Folders (tree, single inheritance) models cannot be reconciled. If you use Gmail as an OS X Mail.app IMAP server, messages with multiple labels (Inbox, All Mail, Baseball, Work, etc) will be replicated in OS X. A single Gmail message may produce several OS X messages; Label-crazed geeks may find dozens of duplicates. Searches are a mess.
I'm happy to say, however, that I have found a way to make this dysfunctional couple settle down. I used How to make Gmail work well with Mail | Macworld as a guide. It claims too much, but the advice is still useful.
If you follow the articles advice, however, you will lose all your email archives from Mail.app; they will be only on Gmail. I inverted the advice. Using the advanced IMAP controls I turned off IMAP sync for all labels except for these four:
- All Mail
This means that my tagging/organization pseudo-folders exist only on Google. None of that is exposed to OS X, the Google "labels" are gone . On the other hand, I still have Spotlight Search and Smart Mailboxes, and my only duplicates are with All Mail and Inbox/Priority Inbox:
These duplicates go away when I move mail from Inbox to All Mail (archive).
Note I don't see the Sent folder (that would create All Mail/Sent folder dupes), but I have no use for the separate Sent folder (I can recreate an equivalent with Smart Search). I prefer the All Mail folder. I don't bother exposing my Gmail Spam label/folder -- it's immense.
I followed most of the rest of the articles advice. Turns out I wasn't saving OS X Mail.app Sent messages on the server -- I thought I was.
With this configuration Mail.app looks like this:
It ain't a match made in heaven, but it's no longer Hell. Purgatory maybe. Maybe Apple will fix this in Lion? 
 The fix is to stop treating Gmail tags as folders, and treat them as, you know, tags. Then support tags in Mail.app and use smart search to create additional derived folders. Who files email any more anyway? I stopped doing that years ago, and it's added years to my life. (By the way, I give a great lecture on email management.)
Update 3/17/2011: Alas, I can’t seem to eliminate the Sent folder. Even though I have the Label turned off on Gmail’s advanced IMAP control, it still produces dupes with the All mail folder. This is a more annoying problem than the small number of dupes in the All Mail folder. Maybe I need to move all my email from Gmail’s “Sent” folder to a new label.