Converting Eudora email archives -- the end is near (and Norton Commander ncmail.exe in a footnote)

I have years of email messages locked up in a 1.2 GB Eudora for Windows archive [3]. Converting them has been a pending project for about 10 years. This archive is a part of my extended memory; it has the keys to remembering people and places now forgotten.

Now time is running out. Eudora Mailbox Cleaner is one of the best tools for this project, and it's PPC only. Since Lion doesn't have Rosetta, the app won't run on 10.7 or higher. [1]

I've downloaded a copy. I think I'll delay my primary machine's Lion conversion until the Eudora conversion is done.

There are other Eudora archive conversion methods, but most run on Windows -- and I think Windows VMs on Mac will fade away as Apple locks the OS down to both block malware and enforce DRM. Time is passing. If you have old archives, you may have only a year or so to (relatively) easily convert them.

I'm planning to create a user account to test the import, then backup up my current Mail database prior to adding this one.

See also:

• Gordon's Tech: Converting Eudora email
• Gordon's Tech: Converting Email from Eudora OS X to a modern format - the TidBITS review: Adam Engst's comprehensive guide

Update

It took a few hours to do the conversion; most of that was spent waiting for a bundled AppleScript to Rebuild Mailboxes. Post conversion the database is 2.4 GB and over 55,000 messages.

Follow the directions carefully. I didn't think the nicknames were worth converting until I saw them; the connections between Groups and Members turned out to have powerful memory value. I recommend testing first with a special user and an empty address book and email account. If you import into an existing Mail database be sure to backup ~/Library/Mail first.

I didn't import Filters. They'd just be noise now.

With OS 10.4 and later there's an essential 'Mailbox Rebuild' step. My Eudora had at least 478 folders across a deep hierarchy [2]. It would have been extremely tedious to try to rebuild them all. So I tried the 10.4-10.5 rebuild AppleScript, even though I am running 10.7 Snow Leopard. 

That script runs slowly, and because of the way UI Scripting work it must own the entire machine. It will steal focus. Don't try to even switch users. It took over an hour to complete. Thank Darwin it's there; the job would have been untenable without it.

As per the documentation (mandatory read):

Your imported mailboxes will be located in a new folder called "Import" at the top-level of your local mailboxes. The new files will be located at "~/Library/Mail/Mailboxes/Import" on your hard disk...

... due to the way the Address Book works, a new group will be created for each Eudora nickname which expands to more than one email address.

I've already leveraged the archive to rebuild memories of people I worked and corresponded with in the 1990s. Some have died, some I've lost touch with. My old Eudora email archive has now been reintegrated into my extended memory.

For now the email archive and my old Address Book are sitting in a distinct user account. I will decide over the next week whether I'm going to leave them there or, more likely, repeat the process. If I do a complete merge I may create yet another user account, copy both my current email archive and this archive there, and let the process run.

OS X mail has a virtually undocumented import Apple Mail Data feature. I don't think it will be practical to import the archive using that but I'll take a look at it. (Update: It seems to have worked quite well. I imported my old Netscape emails at the same time.)

- fn -

[1] A year or two ago our 8 yo G5 iMac hard drive died. I dithered about whether to replace it or not because of the infamous iMac capacitor problem and its slowly delaminating screen. (The G5 iMac was one of Apple's most troublesome machines -- they desperately needed to move to Intel. The heat problems were extreme. It was, however, extremely easy to repair. That was even a marketing feature. Perhaps the G5 iMac is why Apple doesn't design for fixability any more.)

I did replace the drive and it's still working. It can even, if I were to install 10.4, run Classic. It's now one of my most valued machines. I think I may turn our MacBook into the kid machine and put the G5 iMac on a very light duty cycle so I can keep it available for another decade. I'll also archive my software install library onto its now capacious hard drive.

[2] My Gmail archive has essentially 2 mailboxes: In and All. Full text indexing and search changes things.

[3] Created by Steve Dorner, Eudora was the preeminent Mac email software and was dominant for a time on Windows as well. This archive, during its lifetime, moved from Mac to Windows. I found some 1996 correspondence with Steve, he was a very approachable developer with a lot of correspondence!

Before Eudora I used Norton Commander Mail (ncmail.exe) with MCIMail (pre-inernet -- 4867991@mcimail.com was my email at the time.

Writing this post inspired me to look into those archives. I did have them -- zipped up from the days when 200K was a lot of data. OS X couldn't open the old zip archives, but Unarchiver did fine. I used Automator to attach a .txt extension to each file and assigned them to TextWrangler. That was enough to convince OS X to index my old NC email messages. For example:

I actually got a bunch of public domain and shareware Windows applications running SLIP under OS/2 over my 14.4K modem ... 

... The second thing I'm excited about ...) is learning HTML and creating my own home page on the SILS Mosaic connection...

...  for my next 'HOT tip'. Check out TeleScript. Ignore "General Magic" -- it's bogus. TeleScript is the secret pearl.

I didn't remember that 14.4K modems coexisted with HTML 1.x. TeleScript is huge in an alternative reality. GM was a copy co-founded by "Bill Atkinson, Andy Hertzfeld and Marc Porat". Historic names.

The next archive to crack is a set of email written using Netscape's email app. (Update: Mail imports these directly.)

Notes on converting a MacBook Core 2 Duo dual USB to Lion

Our vintage 2006 MacBook Core 2 Duo can, in theory, run Lion [1]. I just upgraded it from Snow Leopard, which it ran reasonably well albeit with lots of fan activity.

I didn't upgrade happily. There's a reason I've waited this long. I wanted to stay with Snow Leopard, but Apple's MobileMe migration was going to cause problem for Emily's Address Book/iPhone Contact sync. Yes, that was the primary reason. Sad.

I upgraded all the apps we use, said farewell to AppleWorks and many games the kids no longer use, removed all PreferencePanes and extensions, unplugged all cables, did a safeboot cleanup and two completely independent backups (one a clone). Then I upgraded.

It seemed to go well enough once Spotlight rebuilt its indices and I refreshed everyone's account. it does take a very long time to log out; Lion is saving a lot of state information and the MacBook doesn't like that.

It went well enough, that is, until my old account. Then things got slow. Even though I'd updated VMWare Fusion to the final "Lion Compatible" 3.x version, I suspected it was the problem. I tried running my VM -- that was a disaster. After power down and a safe boot I uninstalled VMWare 3 per directions.

That helped a great deal. In fact, the laptop is quieter than it used to be. I suspect Fusion has been causing problems for a while [3].

So will I try Fusion 4?

No, not on this old machine. I really don't use my VM's very often, and neither Parallels nor Fusion are sold through the App Store. To do their magic without Apple's help they must be hacking the underlying OS; and OS X is increasingly hard to safely hack.

For now the MacBook seems to have survived Lion - albeit at the cost of a little used VM I'm probably better off without and a dozen or so older games -- and AppleWorks.

So far, better than it could have been. I am, however, regretting obeying Lion's command to update my Airport firmware [2]. 

[1] That's almost six years ago! I'd forgotten how old it really was. Maybe I shouldn't be too upset it can't run Mountain Lion. [2] Lion really wanted me to undo my Time Capsule firmware regression, so I did. Now I'm seeing more problems with losing connectivity, i saw a -1 error again, and I'm again having to rebuild Spotlight indices of the backups. I have a strong feeling I'll be reverting again. I seem to be the only one with this problem though. [3] My best Fusion experience was version 2 with a Windows 2000 VM -- on that old MacBook probably with Leopard (10.5). It's never worked as well since. 

Update 6/17/2012: Logging out and user switching is much slower. It takes about 20-30 seconds to log out and 15-20 to switch. I think it's because of all the context saving Lion does; this old machine can't handle it. There are ways to disable saving of application states, but I'm going to wait a while on this one. Otherwise things aren't too bad. The machine is quieter than it has been for years, the fan no longer roars. I suspect that's due to uninstalling Fusion 3.x, but it could be a Lion improvement.

Update 6/17/2012: I went through each user account and turned off 'save and restore windows' in system preferences. Then I logged out and unchecked the restore windows on login option shown there. No logout and login is back to Snow Leopard times. Now I have to figure out what to do about #$@$ Google Software Update. it keeps popping up in managed accounts that don't have privileges to run it.

How well does Gmail archive via IMAP really work?

My Gmail account holds 64,800 messages. Of those perhaps a few hundred are of lasting interest, but by the time AIs sort out what they are we'll be long obsolete.

So, perhaps illogically, I like to hold onto the archive. That means a local copy. 

I recently had an opportunity to test how well an IMAP archive approach works [2]. I upgraded a machine I'd not used for a while and had to catch up on a year's worth of email. It took a few separate synchronization sessions to catch up, I think I ran into Google's bandwidth limits/transaction throttling behavior.

I ended up with 64,440 in my local "All Mail" synchronized folder, so about 360 have been lost in the bowels of IMAP. Perhaps they'll trickle in with future synchronizations.

The oldest message in Gmail online is dated 2/29/2004 [1], in Mail.app (Lion) it's 2/2/2004. Yes, that's weird.

To answer the post question then, I'd say Gmail archiving via IMAP is imperfect but not pointless. A C grade.

[1] It's not obvious how you can see this. If you mouse over the message count there's a hidden option to swap newest and oldest. It's not easy to get to show; navigating large email collections in Gmail is almost impossible with the "new" UI.
[2] IMAP archive is what Google's all-but-defunct Data Liberation Front recommends. I miss that gang, but there were of the Google- era. In the Google 2012 they're an anachronism.

Sparrow — How not to do marketing

Sparrow's value proposition is that it works with Google's email much better than OS X Mail.app (even Lion OS X Mail), but it also uses the OS X Address Book.

It's a great solution to problems I face.

They don't market this though. If they had, I'd have tested it out long ago.

Aperture 3.3 is iPhoto Pro. At last. Sort of.

After years of false claims of iPhoto to Aperture migration support, and four months after I realized iPhoto was going down the wrong road for me, and three months after I began my extremely painful iPhoto to Aperture migration, Apple has released iPhoto Pro (aka Aperture 3.3).

Imagine my joy.

The only saving grace is that I might have saved a few people by advising them to wait for Aperture 4, which has now come in the form of iPhoto Pro 1.0/Aperture 3.3.

According to Apple's marketing claims, the latest (Lion-only) versions of Aperture and iPhoto  share a single database model. So, in theory, both apps can work on the database.

I doubt it works as advertised, but it has to be an improvement on my experience! I'd strongly advise waiting until September before doing a major iPhoto to Aperture migration. Now that you know the end is in sight the wait should be tolerable.

Personally, I'm looking forward to, at the least, using iPhoto's Picasa uploader and iPhoto's superior UI for Event/Project and many common image management operations.

Assuming the inevitable bugs get sorted out, this is an extraordinary conclusion to what must have been a formidable software effort. At one point Apple had two completely incompatible photo management products. One was a natural Mac app with an elegant UI and some infuriating limitations. The other looked like a port from NeXTStep [1]. They had almost nothing in common.

Slowly, painfully, Apple turned these two disparate products into iPhoto and iPhoto Pro. To do that they had to reconcile very different functional models and data models. It would have been a very hard, very long, glamor-free slog. I hope the team was at least paid well.

Now, if you'll excuse me, I'm going to go cry in my Scotch.

[1] I was never able to learn where the heck Aperture came from. It's neither Windows nor Mac.

Update: I remembered my underused MacBook Air runs Lion, so I tested the new Aperture and iPhoto together. Unsurprisingly, they failed my first test.

First -- the good news. Annotations on an iPhoto Event are now visible in Aperture.

The bad news -- Album annotations (descriptions) are still not viewable or editable in Aperture.

Maybe in another year or two?

iTunes Store: "Error (-50)" - try GoogleDNS?

A few weeks after an intractable -100000 error iTunes error that only resolved when I uses SSH tunneling to download from a California server I ran into a iTunes Store: "Error (-50)" when downloading purchased content with a TV show my son wanted.

I cleared the download cache and deleted the partial download but I was stuck with the problem. I'd already ruled out all the other issues Apple mentions. I certainly didn't use a 'web accelerator' ...

But wait. "Web accelerator" gave me an idea. That suggests a networking issue...

So I used OS X Location switching to swap my CenturyLink DNS services for GoogleDNS (I had both configured). That did it. My download streamed normally.

I wonder if DNS issues with iTunes will get more common; after all, iTunes is a competitor for many DNS providers (though not, I think, CenturyLink).

Perhaps more likely, I wonder if this is related to Apples IPv6 transition.

I'm leaving GoogleDNS running for now.

Access 2003: Join expression not supported with VBA type conversion

One of the stranger bits of my life is that I often do data manipulation using Microsoft Access 2003 in a Windows 2003 server VM.

 

It's a long story. I suspect I'm not completely alone however. Access 2003 is unbeatable for certain kinds of data wrangling across disparate data sources. Access 2007 was a quite severe regression (post-2000, software  regressions are increasingly common).

 

So perhaps it's worth sharing the weirdest bug I've seen in a while. I was unable to do a "left" join on two subqueries where one of the keys involved a type conversion (string/long using VBA CStr() or CLng() - I tested both ways). I got a meaningless "Join expression not supported" error message.

 

I think this has worked fine for years. I wonder if a forced security update broke something. I found if I switched the query from 'Dynaset' to 'Dynaset (Inconsistent Updates)' it worked. Snapshot didn't work.

 

Now you know.

 

For me, it's a sign that I either need to switch to Access 2007/2010 or switch jobs. (I don't think 2010 is 64bit, so I still face the problems with ODBC drivers and bitness. I might be better to try Access 2007 in an XP VM.)

Update: I'm seeing other issues. It feels like something very bad has happened, probably as a result of a Microsoft security update, but I can't tell what.

 

Update 7/4/2012: I'm not quite sure what's going on, still having some issues. It might be an ODBC issue. I think the error message is a red herring.