Fast User Switching and big displays are very hard on VNC. I wish Apple would do something at the Quartz layer.
I'm told this VNC server software is a much better option. I'm going to try again and I'll report back here:
MacInTouch: timely news and tips about the Apple Macintosh:The critical item here in this release is full multi-user support and their specification of recommended clients.
OSXvnc 1.7 is an open source VNC server that provides remote access (with Rendezvous support) to the GUI, keyboard, and mouse using any VNC client. This release is now a full Universal Binary for improved Intel performance and adds a backward compatibility mode for older protocol versions, a flag to disable the screen saver while clients are connected, the ability to add as many listening hosts as you want directly from the GUI, and a number of bug fixes. OSXvnc is free for Mac OS X. [Source is available on SourceForge.]"
Update 1/10/06: in preliminary testing using a Windows client on a fast network this is FAR better than anything else I've tried. The connection from the windows client did not survive a FUS session swap, but it was easy to restore to the background session. Amazing to see that actually work. I may have my long missed remote control for iTunes.
Update 1/10/06b: Not as good with Chicken of the VNC. Whereas the Windows VNC client supported screen scaling, Chicken doesn't. Less useful since my iMac display is so large. Works, albeit slowly on a G3 client.