Reaching for Nerdvana: Integrating family and work calendars

Can I reach nerdvana? Can I manage calendars for every family member, keeping them all as private as digital can be, and also integrate my work calendar while allowing each calendar to be viewed and edited on a PDA?

Not yet, but we're getting close. The secret sauce so far is Google Apps for our family, Spanning Sync, SyncMyCal, Outlook 2003 at home and office, iCal, my wife's beloved but aging Samsung (PalmOS) i500, Missing Sync for Palm OS, and my battered and broken Palm Tungsten E|2. My demon-spawned Motorola RAZR is not invited, there's a seat waiting for the iPhone.

Here's the situation so far:

• Google Calendar on our family domain. The domain means I can easily share calendars internally without exposing them.
• In Google Calendar: a work and home calendar for me, a home calendar for my wife, child calendars to come. My calendar view includes my two calendars and my wifes.
  
• iCal is a natural fit to Google Calendar (gCal), so I create corresponding calendars on iCal for my wife and I on our shared home desktop. For example, my iCal desktop my personal, my work, my wife's personal, etc.
• Spanning Sync allows me to sync each of my iCal calenders to the corresponding gCal calendars, so we're in great shape on the Macs. gCal is the source of truth, but we can view the composite calendars on my MacBook as well.
• My personal Palm syncs to XP Outlook, so I need an Outlook to Palm solution. I'm testing SyncMyCal. So far, it's ok. Even handled some calendar name changes I threw at it. I need it because I want control over sync direction, only SyncMyCal offers that. Outlook is a weak match to gCal and SyncMyCal is not as sophisticated as Spanning Sync -- so I sync only my Personal calendar to gCal Personal and and to my Palm.
• The last step will be to instal SyncMyCal at work, so my work calendar will sync unidirectionally to gCal, bidirectionally to Exchange Server, and unidirectionally to another kludgy app I run on the Palm. [My work calendar uses every advanced feature of Outlook and Exchange server, I don't dare enable bidirectional sync to the much simpler gCal data model.]
  

If the last step works, I'll have achieved 90% of nerdvana ...

Update 4/30/07: I ran into a roadblock on my journey to nerdvana [1] while testing SyncMyCal. The problem sounds a bit like this one, but I don't use ActiveSync. I do use Palm's Outlook sync conduits (HotSync Manager) and I also use a no-longer-supported .NET-requiring Outlook plug-in called Lookout for Outlook.

In my case I had two problems occur around the same time:

1. HotSync Manager stopped responding. This happens every week or two anyway, so I don't give it enormous weight. I killed the stuck process in XP's process list and restarted it and it worked.
   
   
2. After #1 the SyncMyCal toolbar vanished from Outlook and a restart didn't bring it back.

This won't be easy to debug -- I wonder about one of those infamous .NET-version conflict problems. I'll contact the SyncMyCal authors and see if they're interested in tackling this with me. If not I'll contact the gSyncIt folks and see if they're interested in supporting unidirectional sync (a mandatory requirement for me).

I'll update this post if/when I make substantial progress.

Footnotes

[1] Hardly surprising. When one creates a dependency chain of unreliable software components the probability that everything will work starts to get pretty low.

Update 5/7/07: SyncMyCal flopped for me. The install on my home machine was troubled, but I followed the vendor's FAQ directions and I was able to enable the plug-in. It worked at home for a week, so I tried it at work (Outlook Pro 2003 w/ Exchange environment). It crashed Outlook 2003 on startup. I tried a few times w/ rebooting etc, but it crashed every time.

I have sympathy for the vendor -- installing this type of functionality into Outlook 2003 is, I wager, a nightmare. I use 'Lookout for Outlook' at work and at home, and that's an unsupported plug-in now -- but I deeply depend on it. Outlook 2007 is incompatible with Lookout, so when we switch to 2007 I might try SyncMyCal again.

In the meantime, I'll try gSyncIt at home ...

Update 9/8/07: A blog dedicated to calendar interoperability ...

Update 9/6/09: I did eventually get this all sorted out! Actually, several times, since it changes every few months. There are some hints in this 2009 post. I have a unified work/home/family calendar now on both my iPhone and Google. I don't use OS X desktop iCal at all any more.

Yahoo! Desktop (X1) is the new champion

Yahoo! Desktop Search Beta [updated 1/18/05]

Until recently AOL/Copernic was my choice on the XP platform for file search (not Outlook, for that I use Lookout). Copernic isn't ideal however.

Now X1 freebie, better known as Yahoo Desktop Search (YDS) has taken over. Roughly following the same format as my previous Copernic review, here are my comments. Bottom line: it smokes everything else.

Some key features with some caveats :

1. You can configure location of the indices. I store them in a folder that I exclude from backup. You really don't want to backup search indices. All my various search indices tools store files in this folder. The X1 index is 400 MB for a 14GB dataset, but much of my data are in large non-indexed databases.
   
2. You can control readily what folders are indexed. I turned off Outlook search since I use Lookout. NOTE: X1 does NOT appear to index Outlook Task or Note folders.
   
3. It indexes PDF and a wide variety of data types. You can preview files within X1. Big feature. It even includes viewers for obscure applications, like MindManager.
   
4. You constrain your search results by additional quick filters such as data, result type, substring on file name. You can readily sort search results by the usual metadata (file name, date, etc) and by file PATH.
   
5. You can tell it not to index files over a certain size (I used 10MB).
6. You can control when it builds the indices. However, control is limited. Indexing is not all that smart, since my machine is often active (backup, maintenance, etc) the index wasn't getting built. I had to turn off the default option of waiting for an inactive machine. The index did get built and it wasn't a big performance hit. You can't specify a time range for index building.
   
7. You can't index network shares with Yahoo's licensed free version. The commercial version of X1 does this. I may buy the commercial version for my home.
8. It indexes folder names and it treats folders as first class searchable items. You can constrain a search to limit it to folders. This is a HUGE advantage over Copernic.
9. When you find an item, you can right-click to open it in the enclosing folder.
   

Some defects:

1. No fundamental defects for file search so far. Big advantage over Copernic!
2. Toolbar is kind of dumb. I don't care about Outlook searching (I use Lookout for that) but those items still appear on the toolbar. They also appear on the desktop toolbar and take up a lot of room. However you can specify "Files" as the default to search (RMB, properties) on the desktop toolbar then hide everything but the data entry field.
3. YECH (Update 1/18/05). You can't get rid of the Yahoo toolbar in Outlook. Sure you can remove it, but it returns the next time you restart Outlook. This is very annoying, because (see next point) X1 is a crummy tool for searching Outlook. I have a longstanding problem with toolbars in XP that show up unwanted -- I'm still fighting an idiotic Adobe Acrobat 4.0 toolbar that's infested my OS for years. I think I have to figure out where these damned things live -- problem is I think they can live in many places in the OS.
   
4. NOT a good choice for Outlook search. I use Lookout for that, but this would be a huge problem if I dependend on X1. It doesn't index notes and tasks! I didn't notice if it indexed Outlook attachments, didn't care enough.
5. Minor defect: it doesn't seem to understand abstract entities, like "desktop" or "documents" -- only physical directories.
6. It doesn't "smart rank" search results (ie. explicity metadata > directory match > file name match > etc), though in practice the rapid sorting of results and subsearch capabilities mean I don't miss this too much.
7. It doesn't, apparently, search Eudora mail archives. For that you need the non-Yahoo version of this.
8. It doesn't search mounted drives. Again, the non-Yahoo X1 version of the app does this.
9. Update 6/30: It ignores basic windows metadata (subject, author, keyword) entered via document properties dialogs. It ignores PDF metada entered within Acrobat. It doesn't index text comments on Acrobat documents. All of these things make YDS a poor choice for indexing scanned documents. Quite disappointing, actually.
   

My configuration

1. Limited search to the folder that contains my data and the desktop folder. (Removed all other folders, for non-removables set to ignore via "modify" button.)
2. Moved indices to my "Cache" folder (no backup of this folder).
3. Max file size to index 10MB.

XP desktop and Outlook 2003 search: RIP Lookout, GDS, and YDS/X1

It's over. The big guy won.

About 3-4 years ago there were at least five serious contenders for full-text search of Outlook and the Windows file system. The best of the desktop search tools in early 2005 was a relabeled version of "X1" distributed as "Yahoo Desktop Search". Next was Google Desktop Search, though it was a distant second. Microsoft's search solution was weak.

On the Outlook front there was one great solution: Lookout for Outlook.

Microsoft bought Lookout, Google kept on going, Yahoo gave up, the others vanished. Now, finally, Microsoft has integrated the majority of Lookout's capabilities into a revised version of their desktop full text search tool. The result is very impressive.

I've been using WDS for a few weeks now, and at long last I'm removing Lookout for Outlook from my system. It hung on a very long time. I've removed X1/YDS from home and work, it was getting increasingly flaky with each Microsoft system patch (some things never change).

Eventually Microsoft will decide it's time to move their corporate customers to Vista and WDS/XP will die, but for now it rules.

Recommended, but as is usual with Microsoft these days, there are some caveats:

• Many of the Microsoft web pages referring to WDS are obsolete and have bad links.
• There are two ways to install. One installs only Windows Desktop Search, but no IE or Outlook toolbars. The other installs Windows Live Toolbar into IE and a separate Outlook toolbar. Windows Live Toolbar is also used to host my favorite blog writer. I recommend the second install, but I wouldn't be surprised if they were somewhat different applications.
• Toolbar installation into Outlook can be problematic. See the help links on the post-install page.
• Be sure to look at the help file (hallelujah, it's the OLD style Windows help) and look at the keyboard shortcusts and advanced operators such as "has:attachment", "before 10/1/2007", "filename:fred", "store:outlook", "kind:tasks", etc.
• Use the keyboard shortcut: windows-shift-F.

PS. Recently Google has sued to facilitate replacing Vista's search with GDS. Microsoft must be smiling about that. This is one battle Google isn't going to win.

Getting Things Done and Managing Email with Lookout for Outlook

GTD: The Fallow's summary

A month or so ago I posted about the Fallow's summary on GTD. (see link). Here's a revised update. (PS. Too bad Blogger doesn't support trackback links!)

Getting Things Done (http://www.davidco.com/).
See also:

Atlantic Online | July/August 2004 | Organize Your Life! | James Fallows
David Allen's book. (This is a bit dated, he needs a new edition.)

1. If you can do it in two minutes, just do it.
2. Get everything out of your head. Appointments, tasks, notes, contacts -- get it into one place (eg. Outlook).
3. Tasks have three important relationships:
the minimal context needed for the next action (ex: anywhere, phone, desk, computer, network, office ..)
the project(s) that contains the task
date of next action
4. Tasks always have a next action. Identifying and executing 'next actions' is critical.
5. Record tasks/ideas at time they are recognized.
6. Weekly review of about one hour. (This takes me at least 2 hours but I'm trying to speed my review.)
7. Tasks don't have priorities. (Personally I use priorities on tasks but Allen assumes if a task is scheduled then it should be done. I see his point.)

How I handle email (using Lookout)

(Note this works for any email solution that supports full text indexing. I think it would work for OS X Mail in Panther, it will definitely work in Tiger.

Lookout for Outlook:

1. Install Lookout. Note this is an early product and has some rough edges. I force a complete index rebuild every night. Still, value is enormous. It has not affected my system stability.

2. Read message. Follow GTD protocol as above (see book too). Then either:
- delete
- create task and save
- save

3. If a task is needed, I create a task by dragging the message to the Outlook task icon. Outlook creates a task that incorporates the message content (text only).

4. If the message is to be saved I
- edit the subject line to be more descriptive of the message
- rarely I edit the message text or subject line and add terms I'd use for a Lookout search.
- drop the message in my "Save" folder (that's it). I don't use subfolders anymore and don't spend time filing anything.

Update 3/2/05: This related article talks about environmentally-induced ADD.

Google Desktop Search bites the dust

I've posted frequently since January 2005 about various approaches to full text search in Windows XP in general and Outlook 2003 in particular.

My test environment is harsh. GBs of Outlook PST files. GBs of documents. Machine never rests. Hard drive always on the edge of meltdown.

The best solution, by an order of magnitude, was Lookout for Outlook. Fast, stable, and the bugs are manageable. Alas, it conflicted with some other Outlook Add-Ins (that environment reminds me of DOS TSR hell). Lookout also doesn't work in Outlook 2007 and it's definitely not supported anywhere, so I thought I needed to change. That was probably a mistake.

I used Yahoo Desktop Search for a while, but it was increasingly buggy and Yahoo finally abandoned it. It reverted back to X1 but it was even buggier when I tried their version.

Recently I decided Windows Desktop Search was my only option. It was slow and sucked performance, but it did all I needed and it incorporated Lookout's search syntax (see the help file). Unfortunately, my work machine has been possessed lately.

I'd removed about everything I could think of to try to stabilize my system, so it was time to remove WDS and try the only remaining contender Google Desktop Search.

Yech. This is what I wrote when I uninstalled it:

... I don't have enough control over indexing behavior. I don't have enough search syntax control - esp. for outlook tasks vs. email search. It was WAY slow to do a search against my multi-GB Outlook archives. It doesn't treat folders as first class search objects...

So now what do I do? I could buy a MacBook Pro and run XP in a VM, but Spotlight won't search PST files.

I need full text search - esp. of PST files.

I'm going to try Lookout again; I've gotten rid of most of my Outlook plug-ins / add-ins anyway.

Gee whiz, I miss my home OS X machines. I know people complain about Vista, but I'm not a great fan of any Microsoft products at the moment (though my problems, to be fair, may be hardware related). Even my beloved Windows Live Writer is buggier at the 1.0 release than it was in the last two beta versions!

Update 11/16/07: Joe Cheng (see comment) noted I hadn't submitted my WLW bug reports. They're really not awful, but they stood out relative to the perfection of the last beta. One is certainly a bug -- the category/tag names can be truncated randomly (last few characters are missing). The other two I need to prove are really bugs, which is a bit tough since my Dell laptop is definitely flaky.

I'm going to be more careful going forward about what I call a bug ...

Update 11/16/07b: The regressions (bugs) will soon be history. In the meantime don't use the Post to Weblog as Draft feature with Blogger.

Lookout for Outlook -- its time has passed

Lookout for Outlook has been the most valuable productivity tool I've used in years. Sadly, it was purchased by Microsoft. There was ongoing devleopment, but now it's been wrapped into MSN desktop. I don't know if MSN Desktop supports all of Lookout's functionality -- I rather doubt it. I'd also suspect there's no way to restrict MSN indexing to Outlook only. I'll keep using Lookout as long as possible. Not the first time a great tool has come and gone.

Lookout Discussion Forums - New version of Lookout ever?

As always, its great feedback like this that is really nice to hear and drives all of us to make our products better! Thanks to all of you for your support and helping make Lookout a great product; we're no geniuses - its only thanks to that feedback that so many people like the tool.

Now that the MSN product (http://desktop.msn.com/) is officially released, Lookout is not likely to have another release. The MSN product has outpaced Lookout, and that is where we're moving forward. Several of us from the MSN team do read these forums, and we take all feature requests to heart with the MSN product. (I hope it is noticed that the MSN product already has a number of features which originally were absent even from Lookout!)

Mike@LookoutSoft

gSyncit and SyncMyCal: not compatible with Lookout for Outlook

I'd tried SyncMyCal at home and work for gCal synchronization with Outlook, but it crashed Outlook.

I then tried gSyncIt. This time it installed, but Lookout reported that "another Outlook Plugin has installed an unofficial version of the Outlook libraries which breaks Lookout".

Sigh. Lookout may have been euthenized by Microsoft but I absolutely depend on it. I've never seen anything else half as good, including Microsoft's "replacement" windows desktop search.

I'll uninstall gSyncIt and try reinstalling Lookout. The gSyncIt install was suspiciously amateurish anyway (an installer named setup.exe?!), and I wonder if it really did install a hacked version of the Outlook libraries. Not to mention the lack of documentation.

One plus to this, I'm now reasonably sure it was a Lookout/SyncMyCal clash that crashed Outlook at work.

I'll have to wait and see what else comes up. Google's help file more or less promises a sync solution from them. I probably need to give up on Lookout (it's not compatible with 2007 anyway), but I'm not in any hurry to do so. I'd be in more of a hurry if I had more confidence that there was a great Outlook/gCal sync solution available.

Lookout CAN run on Outlook 2007

Lookout for Outlook was the best full text Outlook search solution ever to run on XP. It wasn't half bad for handling file system search as well. The last version had some bugs, but it was fundamentally excellent.

Alas, Microsoft bought the company and killed the product. Windows Live Search (aka Windows Desktop Search) is what I use now, though I fear it has deep bugs that might be killing XP. Of course I have a completely plausible paranoid fear that every new Microsoft product and patch is designed to covertly degrade Windows XP (the noble Windows Live Writer team doesn't know about the changes being made to their code after check-in).

Microsoft wrote the encyclopedia on killing rival products by creating subtle and gross incompatibilities, and Microsoft's most hated enemy these days is Windows XP. The Dark Arts are never truly forgotten.

But I digress.

Joel Spolsky tells us that the original developer is finding ways to extend Lookout's lifespan:

Getting Lookout to run on Outlook 2007 again - Joel on Software: "...the original author of Lookout, Mike Belshe, had just posted instructions for getting Lookout to work on Outlook 2007."

Incidentally, Belshe has a blog. Today's post has him praising Windows Live Writer and cursing Microsoft's web sites. I love reading people who agree with me, so I'll add his blog to my feedlist.

gSyncit for Outlook 2007 to Google Calendar and Contacts Sync

Looking through the archives I see I've written several posts about gSyncit (ex: gSyncit and SyncMyCal: not compatible with Lookout for Outlook). Each experiment ended up with me giving up on the topic.

Most recently I was trying Google Calendar Sync again with Outlook 2007. It worked for a week or so, then stopped with a cryptic error. I've a history of disappointments with Google Calendar Sync, so I decided to try gSyncIt again.

It's grown quite a bit since my earliest experiments. At one time I see I was asking for control over the sync process; they now provide a lot of tweaks and controls. More than most people could tolerate, but I'm a veteran of Sync Hell. I want 'em all.

I've enable unidirectional sync - Outlook Calendar to Google calendar and Outlook Contacts to a specific Google Contacts group. It will be a long time, if ever, before I dare to try true bidirectional sync.

So far, so good.

I registered this time, so they've got my $10.

Update 11/9/08: Not yet characterized, but there are time zone problems. I think Google Calendar tries to be "smart" about the time zone one is currently in. Big mistake. Correction -- this isn't a gSyncit/Outlook problem. I think this is a Google Calendar quirk depending on the time zone settings on the web client host machine.

Update 11/11/08: It's not exactly a quirk, more like a feature that's a bit too obscure. I've a f/u post on this dated 11/11/08.

Update 12/8/2008: gSyncIt has a bug with all-day events that span multiple days. It replicates them. I've seen this before; Outlook has a very nasty internal representation of an "all-day" event. I'm surprised gSyncIt hasn't fixed this bug, it's not subtle.

Copernic/AOL: current leader in the sponsored (freebie) Windows desktop search race

Copernic Desktop Search - The Search Engine for Your PC

Updated 1/5/05 with, unsurprisingly, some more of the negatives.
Update 5/10/05: I think Yahoo Desktop Search (X1 freebie) is the best PC desktop search tool -- except it doesn't index Eudora. X1 commercial does -- for $80. There was rumor of a Google Desktop Search plugin, but it's not there yet. So, for home, where I use Eudora still on my PC, I'm going to try Copernic 1.5 again.

Introduction

I really like Lookout for searching and managing Outlook. I'm sticking with that one for now. I tried Google desktop search and a few others, including Lookout's desktop search -- didn't like 'em.

Since MSN search uses Microsoft's built-in indexing, which I don't like, I haven't tried it.

I think Yahoo/X1 will be interesting, but it's not a freebie yet.

Copernic/AOL is my latest. It looks good at first glance, but it has some fundamental deficits.

Some key features:

1. You can configure location of the indices. I store them in a folder that I exclude from backup, including backup via ConnectedTLM. You really don't want to backup search indices. All my various search indices tools store files in this folder.
2. You can control readily what folders are indexed. I turned off Outlook search since I use Lookout.
3. It indexes PDF.
4. You can tell it not to index very large files.
5. You can control when it builds the indices, including time of day scheduling. It will do low level background indexing (not a default). Index builds seem very fast and "smart".
6. If you map a network share to a drive letter, it appears that it can index that drive. (Performance may be poor, I haven't pursued this capability as I don't need it at work.)
7. It indexes folder names. (A major flaw in Lookout 1.2's file system search.)

Some fundamental defects:

1. It's "stupid" in how it does search rankings. In particular it doesn't use NTFS file metadata it doesn't weight metadata >> folder name >> file name >> document content and it doesn't differentially weight aspects of documents (titles, early text, etc) .
   (One of the reasons Lookout works so well for email search is that there's so much reich metadata available in Outlook. The typical PC document filestore has very little metadata. I'm very interested in the 2005 OS X Tiger search because of the way it uses metadata.)
   
2. It doesn't return a folder or directory as a search result. The only search results are files. This throws away a lot of the intelligence and metadata that may exist in a file store.
   
3. You can't search within a result set easily.
4. You can't sort result sets by metadata (file size, type, date created, date modified, etc).
5. The Help and Submit Bug buttons are broken. There's no documentation.
   

My configuration

1. Limited search to My Documents folder only. (Removed all other folders, for non-removables set to ignore via "modify" button.)
2. Moved indices to my "Cache" folder (no backup).
3. Update index daily at 1am.
4. Max file size to index 10MB (I may shrink this further)
5. Various other small tweaks to enhance performance.

Conclusion

It's the best of a bad bunch, but I'm not impressed. I'll compare it to X1 when Yahoo launches it. For now I'm staying with Copernic (it beats Google Desktop!) but I still get more value from a directory string search I hacked together that emulates the immensely underappreciated Norton Change Directory feature of Norton Commander and Norton Utilities @ 1989.

Windows Search 4 broken by recent update causing MDAC corruption

My XP box index is complete, but Windows Search 4.0 returns nothing. The Event Viewer has no interesting Windows Search Service events; the indexer seems happy, but the search isn’t working. Rebooting didn’t help.

On any search I get "Nothing found in All Locations for query ...".

The only hints I could find wer ea recent post with a Vista problem: SearchIndexer.exe causing problems after Search 4.0 update on Vista Home Premium system. - MSDN Forums and Desktop Search help has no recent advice.

I’ll try doing a windows update, then if that doesn’t work a uninstall/reinstall cycle.

Windows Search was much happier when I was using Office 2003. It hasn’t been the same since I went to 2007.

Update 10/20/08: I miss Lookout for Outlook. Also, Spotlight and all of OS X. Anyway, the Windows Update and reinstall didn't work. This time I'll uninstall, track down my index and trash it, and try again. As before the index is built, everything looks fine, but searches return nothing.

Update 10/20/08: Still not working. I'm running out of ideas. Next step is to uninstall Windows Search 4 and install Google Desktop Search! Instant search works in Outlook, but desktop search doesn't work at all. From a post on MSDN that I wrote:

A few days ago Windows Search 4 stopped working in my corporate XP desktop (all updated). All deskbar searches return "Nothing found in All locations for query ..." regardless of the query. I can't indentify any precipitating event but this is a managed corporate desktop. Anything can happen to it.

Web searches work. Instant searches in Outlook 2007, which use the Windows Search engine, are also working.

The index is fine and it's being maintained correctly.

I've run Office 2007 Diagnostics. I've rebuilt the index. I've uninstalled Windows Search and reinstalled. I've reviewed the Applications event log. I've deleted the index and indexed only a small bit of Outlook 2007. I've relocated my index to a new directory.

Nothing makes any difference. Instant Search works, the indexer works, Windows Search 4 doesn't work.

My corporate desktop is encrypted (SafeBoot), but I've not run into any problems there.

I don't know what to try next. Actually, my next step is to uninstall Windows Search 4 and install Google Desktop Search. I need search to work and I can't go on a lot longer without it.

Later I also reinstalled Office Pro 2007 -- to no effect. I did try Google Desktop Search, but for me it was far too simplistic. It also crashed every day.

Update 10/25/08: The hardest problem I've solved in years. I finally found a 2006 post on an MSDN board about the same problem. Search worked in Outlook 2007, but not from the desktop. It turns out that desktop search uses Microsoft's data query infrastructure (MDAC), and that infrastructure can be broken.

Repair of a broken MDAC stack is occult. Fortunately Paul Nystrom had the answer in 2006

http://forums.microsoft.com/msdn/ShowPost.aspx?PostID=901078&SiteID=1

This generally occurs when you have a corruption in your MDAC stack. You can find instructions for repairing your MDAC stack here (note this solution is not officially supported by Microsoft):

http://www.pqsystems.com/kb/activekb/questions/165/

For some additional information:

MDAC stands for Microsoft data access components. These components allow WDS to query it's index for resutls. When MDAC gets corrupted WDS can not retrieve results from the index resulting in empty query results.

Paul Nystrom - MSFT

I followed the repair advice on the referenced page (I have XP SP 3 installed):

How to repair a copy of MDAC 2.8 SP1 on Windows XP with SP2 installed.
----------------------------------------------------------------------

Right-click on C:\Windows\Inf\mdac.inf and choose "Install".

. ..point to the i386 folder in one of these places:

1. C:\Windows\ServicepackFiles\i386 (it may not like this location, if not, go to the next one)

2. The \i386 folder on your XP installation CDROM.

This reinstalls and repairs MDAC 2.8.

I used the servicepack files folder first. When asked for a file that wasn't in there I used the \i386 folder our corporate IT had on my drive.

It worked. Thank you Paul.

Hardest fix in years. Without Windows Live Search and Google I wouldn't have had a chance.

So what happened? My guess is that there's a problem with the sequence I took, moving through Windows Search 4 on Office 2003, then XP SP 3, then Office 2007. Somewhere in that sequence I broke MDAC.

The Wikipedia article on MDAC is informative:

The current version is 2.8 service pack 1, but the product has had many different versions and many of its components have been deprecated and replaced by newer Microsoft technologies. MDAC is now known as Windows DAC in Windows Vista.

XP is starting to remind me of Windows 98.

X1.com -- search files and outlook -- a review

X1 instantly searches files & email. For Outlook, Outlook Express, Eudora and Netscape Mail.

I took another look at this since Fallows wrote of it recently in the NYT. It's a product of Bill Gross, a Caltech undergrad classmate of mine. He made his initial fortune on a Lotus Notes module (long forgotten).

It's not quite ready for primetime. Some odd bugs and behaviors. Sent my CPU usage through the roof. Every time I hit the letter 'e' when searching in the file tag it jumped to the email search tag!!

Now it's quieting down after I tweaked the options:

1. Turned off all outlook/email/contacts indexing. Lookout works great for Outlook, and X1 doesn't search notes or tasks (meaning it's not useful for me).

2. Removed all the quick key entries for email. That stopped the 'e' problem, so now I can search for terms containing the exotic letter 'e'.

3. Restricted indexing to a subset of directories and to files under 2MB. In some directories limited indexing to file and folder name. If nothing else X1 may be fast way to locate directories and files.

4. X1 ONLY does stem searching. It doesn't do substring searching. This is a reasonable compromise for document indexing, but for finding folders/directories substring searching is feasible and necessary. So it's not as good for navigating directories as WCD (for example).

5. I need more control over what NOT to index, preferably using regex to define directory paths and files to exclude. X1 is indexing all of the FrontPage index files.

Given the above it might be useable. I have a LOT of content to index.

Lookout for Outlook 1.3.0

Lookout for Outlook was one of the best pieces of productivity software ever created for the Windows platform. It makes Outlook, one of the shoddiest and crummiest piles of code ever sold, actually work.

Microsoft bought it and it disappeared. Allegedly MSN search has some of it.

I use version 1.2.8 dozens of times a day. By chance I see there was a quiet update to 1.3.0 last February -- almost a year ago. Worth trying.

Update 1/11/06: Hmm. I think 1.3.0 is not rebuilding my indices every hour as its supposed to. Alas, I fear I did this update once before -- with similar consequences. I withdraw my recommendation. I guess 1.2.8 is as good as it's going to get.

Lookout for Outlook: another outlook search tool

Lookout for Outlook Download

Lookout for Outlook Download: it's back!

Lookout for Outlook Download

I use Lookout every day. It was briefly unavailable, but as Fallows explains -- it's back!

In the face of bitter blog-world complaints, however, Microsoft reversed course and agreed to make Lookout available again. It is one of several intriguing free utilities in the Sandbox section of Microsoft's Web site.

This is a great application.

Google Desktop Incorporates a blend of Launchbar and Spotlight

Google Desktop 2.0 is big disappointing.

Update 8/24: Even bigger disappointment. After it finally built my indices (took days) I tried it out. I learned that:

1. It doesn't match on folder names. In other words, to GoD folders are invisible annoyances. ARGGHHH. The fools, the fools. This one just bit the dust.
2. It took forever to build the index, and you can't relocate it to a non-backed up drive without an unsupported utility.
3. Lookout search works, but I couldn't contrain what things were indexed and I couldn't restrict search to particular object types (tasks, etc).
   

I'm back to Yahoo Desktop! and Lookout for Outlook (now a zombie -- no further development going on post-Microsoft acquisition). I may try a look at MSN search sometime but I fear it doesn't include enough of Lookout's capabilities to be worth a switch.

Update: Biggest disappointment -- I can't find a keyboard shortcut so I can search and execute without a mouse. I'm sure one will be added shortly!

One of the most interesting features, however, may be a blend between OS X Launchbar (my favorite must have OS X utility) and Spotlight (which I'm still trying to find a use for -- if I didn't use Launchbar it would be more useful). That blend is precisely what I've been missing. They probably haven't incorporated, however, Launchbar's brilliant machine learning algorithms. (BTW, why doesn't Launchbar wrap Spotlight? Update 8/24: I'm told it will!!)

Quick Find makes launching applications and searching your desktop easy and fast. From within any application, just type a few letters or words into Sidebar's search box and you'll see the top results pop up instantly. You can use Quick Find to launch applications without having to deal with the Start menu; for example, if you have Microsoft Word installed, you can launch it by just typing 'wor' into the Sidebar search box and selecting 'Microsoft Word' in the list of results that appears. You can also use Quick Find from the Deskbar and Floating Deskbar, which are described in a separate section.

A few quick notes:

1. It's supposed to index tasks, calendar items, etc. We'll see. They mean Outlook but I wish they'd say so.
2. You can index networked drives and non C:\ drives. That's big, this was a major limitation of 1.0.
3. The Sidebar is an application deployment environment (aka an 'operating system' in which one can install other applications. Shades of Konfabulator and OS X Widgets. This is a shared desktop that can be accessed via any XP box (but NOT a Mac -- peculiar that Google should be driving their audience to Microsoft's platform)
   
4. You still can't move or relocate your index file, so you have to explicitly avoid backing it up. Annoying.
5. GMail indexing is most interesting. In my case, makes up for not indexing Eudora.
6. The deskbar does replace the Start menu.
   

Desktop search: why it has to be much smarter

Faughnan's Tech: Copernic/AOL: current leader in the sponsored (freebie) Windows desktop search race

I keep this blog primarily for my own uses as a place to keep notes on topics. Occasionally I do make editorial comments, but I was surprised recently to learn that at least two people read my posts on Copernic. I even replied to a comment explaining why I haven't yet bothered with MSN search.

So although I don't expect much readership, I'll expound a bit here on my thoughts on desktop search. If nothing else I'll link back to this in future.

I think desktop search has to be smarter than most people think -- at least for the 0.01% of the world that resembles me. (Caveat: I'm so far off the spectrum of users that no product manager with experience would use me as a representative user. On the other hand, I may resemble a "department" of typical users.)

I have thousands, maybe tens of thousands, of documents distributed across local and networked drives and, now, Blogger repositories. They go back about 15 years. I have maybe 10,000 images and they're growing fast. I have over 4GB of email in Outlook repositories and 2GB in Eudora. I have spreadsheets, databases, etc.

"Dumb" full-text search of my repository just returns noise: thousands of hits.

On the other hand, Lookout works great.

Why?

Because I use Lookout to search only my Outlook repository. And that repository has LOTS of rich metadata. There are date entries, subject entries, people entries, item-type (contact, task) etc. Lookout provides ways to constrain searches by metadata. It seems to use the metadata in its ranking (Subject >> text). It allows me to omit indexing attachments -- which adds more noise than value. I learn to edit email subjects/titles in Outlook (secret tip: this is very easy to do) before I throw my email in the Save folder (I don't use any other email folders). I can add an optional layer of on-the-fly metadata by adding categories. (Very poorly supported in Outlook otherwise this would work better.)

Desktop search has very little metadata to go on. Yeah, NTFS has LOTS of rich metadata support -- but it's ignored by almost all applications. Microsoft synchronizes (awkardly) Office document metadata with NTFS metadata, but even in Office support is weak. The workflow for adding metadata, even document titles, is very poor.

The biggest source of filesyste meaning-rich metadata on a PC is the folder/path name -- even more than the file name. I do better using a self-built kludged implementation of Norton Change Directory than I do with Copernic or any other filesystem indexing method. That works because I try to make my folder names descriptive.

Smart desktop search for someone as atypical as me needs to be smart about metadata. It needs to value strings in path names more than strings buried on page 50 of a 200 page document. It needs to value "Title" strings more than deep tex strings. It needs to value file name strings. It needs to rank recent above old. Heck, I could make a longer list (anyone want to pay me :-?).

Search results need to be very quick to sort and (me only -- subsort) and to allow additional subqueries (ok, I'm very data oriented.)

I think OS X Tiger search is going to knock the socks off the PC products I've seen so far -- including Google's disappointing offering. Reading their developer notes, they clearly understand the problem -- and, more importantly, they plan to deliver on their understanding.

Yeah, I know Microsoft has great stuff in the labs (Longhorn, etc) -- but so did Xerox 20 years ago.

Of course, as noted above, I'm really extreme. On the other hand, a departmental group of 10-20 people may have similar needs to mine. So someone building a desktop search solution for someone like me is building something that may work in an organization.

Getting Things Done: integrating the Outlook add-in with Palm synchronization across multiple machines

Getting Things Done Outlook Add-In Support Site

GTD is the latest geek fad. It translates "anxiety into action" and makes productivity very much like a flowchart. Geeks like flowcharts and methodologies -- esp. if they render the world of social interaction more programmatic.

Heck, I'll give it a whirl. Problem is, I sync Outlook with my Palm and I took ten years off my life getting all the outlook/exchange/palm bugs working. I use Chapura's KeySuite for synching my Palm to Outlook/Exchange at work, and PocketMirror for synching to Outlook at home. (I'll skip the part about using iSync to ALSO sync with my iBook -- I haven't dared go there. Yet.)

I also have all kinds of ways to use Categories that aren't compatible with GTD. Also, GTD expects to OWN Outlook -- it doesn't know anything about PST files outside of the core Outlook files. I keep all my data in local PST files.

Then it occurred to me -- the Chapura KeySuite Palm app can sync with multiple Outlook folders. I can use GTD without disrupting its operation.

GTD owns my OST (root folder). KeySuite sync is set so all task/notes/contacts from that folder overwrite my corresponding Palm folders. So no exchange issues. Those items are read-only (for now).

My PST folders (contact, tasks, etc) continue to sync with the Palm. They work the same as ever with their own categories.

And, of course, I continue to use Lookout voraciously, updating its index hourly and rebuilding completely at every opportunity.

The best revenge on Outlook is to crush it mercilessly with programs and techniques that make that abomination useful. Too bad Word is unredeemable.

X1 instantly searches files & email. For Outlook, Outlook Express, Eudora and Netscape Mail.

X1 instantly searches files & email. For Outlook, Outlook Express, Eudora and Netscape Mail.

---

Lookout works for me, but X1 searches file content as well: http://www.x1.com/products/viewable_file_formats.html

Reasons to install Google Desktop and thus Google Desktop search ...

Yahoo Desktop Search, when it doesn't crash and burn, is still the best full text indexing and search tool for XP (I so wish Microsoft hadn't bought and killed "Lookout for Outlook". Google Desktop, and thus Googe Desktop Search, however, is getting very, very interesting. The Plug-In modules are getting clever enough that they significantly increase the value of the product. Consider this one ...

Google Desktop Plug-in: HDDlife plug-in for Google Desktop

Worried about a hard drive failure? Get HDDlife - a real-time hard drive monitoring utility with malfunction protection and data loss prevention features. This hard drive inspector is an advanced proactive hard drive failure detection system that manages all of your hard drive risks. HDDlife runs in the background constantly monitoring your disks. It alerts you about possible hard disk problems before they happen and displays a disk health indicator in the Google Desktop Sidebar.