Gmail spam filtering: A crisis with Gmail threatens all Google services

I posted this on Google Groups: Problem-solving (Update: 8 hours later, it's yet to appear on the the forum. It's good thing I'm not prone to paranoia ...).

My Google Gmail account is dying from dysfunctional spam filtering. That's bad enough, but the role of Gmail as the centerpiece of Google's identity management strategy means there are surprisingly widespread implications ...

... On a weekend 80% of my inbox is spam. During the work week 50-70% of my inbox is spam. My spam box has about 2000+ spam in it each month.

On the other hand, messages I send to myself using my visi.com authsmtp account using my faughnan.com return address are ALWAYS treated as spam. Based on my experiments I believe Google has blacklisted my personal domain - faughnan.com (see www.faughnan.com). It's a personal domain that, like many others, has been faked in signatures by spammers for years. Google appears to be using a kind of blacklisting that is lowest form of spam control and hasn't worked for years. No serious ISP uses such a crude approach any longer.

In contrast my non-Gmail accounts get only a few spam every day and almost never treat non-spam messages as spam. Yahoo does remarkably well, but the open source solutions my ISP uses also work well. I have no experience with AOL or MSN. I'm told the .Mac spam filtering is awful too, maybe they use the same approach as Google.

So I have a few questions:

1. How can one appeal what appears to be Google domain-specific blacklisting?

2. Is there a signed email or domain authentication approach that Google honors and that would improve the accuracy of their spam filtering? I may switch faughan.com to an ISP that supports domain signing if that would help.

3. How the heck do we get Google to admit it has a serious problem with its spam filtering methodologies?

I was the first Google users at a our .com startup in the 90s, back when Alta Vista was king and well before anyone had hard of Google. I was one of the very early cohort of Gmail adopters. I "bleed Google". So it's noteworthy that Google's use of Gmail as the centerpiece of their identity management strategy means a problem with Gmail threatens the entire Google relationships,

My Gmail account is my Google digital identity. If I abandon it I walk away from Google checkout and a heck of a lot of other Google services. Soon my blogger account will be tied to my Gmail identity, and I have thousands of postings in my active blogs [jf: also my $30/year Picasa Web Albums]. A crisis with Gmail is a crisis with a huge range of Google services. Gmail's spam issues have now reached that crisis point.

If Google can't fix this I'll have to walk away from my Google identity. They need to give their spam problems FAR more attention that they have to date. They have the resources, how can we get their attention?

Google has had a similar problem with their splog detection methodologies. I'm seeing much more of a downside in a close relationship to Google. We have much to learn about the consequences of a corporation owning one's digital identity. It won't be pretty.

Update: See also.

Update 9/18/06: My post to Google's help group still has not appeared, but really, not much is showing up there. I wonder if they've got a technical problem. I did solve the john@faughnan.com problem for myself at least. If you add a name to your contacts list it won't be filtered out.

Update 9/19/06: Well, it seems Google's Gmail help group has consigned my postings to the bitbucket. I've reworked all my mail streams and I'm back to the old days of POP/IMAP. I tried Yahoo's beta webmail service but really, it's exceedingly ugly. I will definitely miss Gmail, but overall this may work better for me. My guess is that my mail redirection was the problem. All the spam sent to my personal domain account streamed unfiltered at Google with two results -- Google decided my domain was bad news and it overwhelmed Gmail's feeble spam defenses. Now that stream will hit only visi.com (postini filtering) but they've handled it for years. Gmail is forwarding to my visi account as well.

Update 9/21/06: Light at the end of the tunnel?

Film scanning - via Aufflick

Marginal Software has some very good discussions on film scanning.

Google Earth ads multimedia location content

Official Google Blog: The best stories on Earth

Documents, audio, video -- in a new "featured content" layer for Google Earth. I need to fire that thing up again.

Mail to iPod - now this is useful

Mail to iPod AppleScript - The Unofficial Apple Weblog (TUAW)

My wife might use this with her Nano. Drag an email to a mailbox, it goes to iPod next sync. Actually useful - if it works. Update after testing.

Weird lacunae: you can so view iTunes movies on a TV

Sometimes I think I've shuffled into this world from a parallel universe.

In my previous universe, you could buy a video cable (looks like usual mini-connector to RCA cable but I think the video/sound connector sequence is odd) for an iPod and play movies on a TV. I never tried it, the resolution was too low for even our crummy TV and my 4G iPod is clearly a music player first and a video player marginally. Besides, there wasn't anything for the kids, and we never have time to watch movies or tv ourselves.

Then word of iTV came out, and the fact that iPods output video seemed to have been forgotten. In this case Pogue does something very odd -- he mentions output from the iPod then forgets it a paragraph later ...

New at Apple: Smaller iPods, Bigger Ideas - New York Times: "Playback on a TV (for example, when connected to the computer or the iPod’s optional video dock) is exactly what Apple says it is: near, but not quite, DVD quality...

..Who, exactly, is interested in movie downloads at all? Compared with DVD’s, movie downloads offer limited selection, very little savings, no DVD extras and no surround sound. The files are huge, the quality isn’t even up to DVD standards, and you can’t play your movies on a friend’s TV or a DVD player in the car.

Huh? Is this some kind of bizarre supernatural mind control? If you carry the cable, you can play it on anything that accepts RCA inputs. The ultra-crummy DVD player our kids uses has a jack for that, and of course all semi-modern TVs do.

So what am I missing here? If iTunes has some of those Disney's kids movies you can't buy any more (Disney is weird/evil that way), we may even buy one. We'd need to buy the cradle though, a 4G iPod battery won't last through a movie without external power.

Update 9/21/06: The iTunes 7 update also updates the resolution capacity of the 5G iPods.

Data entry on the new iPods

iPods have long had some very limited data entry. You can modify ratings and build 'on-the-fly' playlists. I think that's about it.

The new iPods, however, have the ability to the ability to enter a string. Lower case and numbers only, used to search for songs. The data entry is rather slow, but since it auto-completes against a small string set it's probably quite practical. Hard to imagine entering phone numbers or addresses this way however.

iTunes 7 issues

Macintouch is compiling a list of iTunes 7 issues, mostly on OS X since this is a Mac site. A few that look like true defects:

- if your iTunes Library is in an atypical location you may have big problems. Consider relocating to a conventional location. OS X seems to be increasingly adopting the defects of XP.
- There's a big bug with OS X dual monitor setups. In some cases displaying the album images on a second monitor will lock up your system.
- Memory demands have risen significantly and there seems to be a lot of graphics processor demands -- presumably related to the album art. G3 machines are borderline for iTunes 7. Shame about all those machines Apple ships with 512MB of memory, much less the 256MB of only a few years ago.
- I was surprised to learn that iTunes 7 on OS X still doesn't support Apple's VoiceOver accessibility features. I'd like to see some ADA litigation going -- Apple has run out of excuses. They had the time, they have the cash, they own the US. Sue 'em.
- There's the usual odd mix of things not working on some machines. Some of these will be real bugs, some probably represent hardware or deep software issues that might have been exposed even by reinstalling iTunes 6.

As a side note the album art download feature, it's been pointed out, tells Apple know exactly how much music you have and what percentage is DRM vs. non-DRM.