Wednesday, October 10, 2007

Why we need something better than HFS+: bit errors are cumulative

I hope this analysis is not correct, but if it is then there's no debating that we need an HFS+ replacement. The geeks I read generally favor ZFS+, perhaps with Apple contributing improvements.

Recording Artist: ZFS Hater Redux

Here's a fairly typical Seagate drive with a capacity of ~150GB = ~1.2 x 1012 bits. The recoverable error rate is listed as 10 bits per 1012 bits. Let's put those numbers together. That means that if you read the entire surface of the disk, you'll typically get twelve bits back that are wrong and which a retry could have fixed.

Yes, really. Did you catch the implications of that? Silent single-bit errors are happening today. They happen much more often at high-end capacities and utilizations, and we often get lucky because some types of data (video, audio, etc) are resistant to that kind of single-bit error. But today's high end is tomorrow's medium end, and the day after tomorrow's low end. This problem is only going to get worse.

Worse, bit errors are cumulative. If you read and get a bit error, you might wind up writing it back out to disk too. Oops! Now that bit error just went from transient to permanent.

Still think end-to-end data integrity isn't worth it?

I wonder how NTFS compares? Too bad it's not open source :-)!

Tuesday, October 09, 2007

Ecto - the first good blogging app for OS X?

I've tried several OS X blogging tools, but none of them compared to Windows Live Writer. This post, however, was authored in Ecto 3 beta from infinite-sushi.com, and it comes close to WLW. It even supports OS X services! The link to the "alpha" version in the news blog actually downloads the beta version, I'm on 3.0b2.

I'll have more to say on Ecto as I test it, but it's great to see this emerging.

Outlook and automatic task lookup based on email subject

Microsoft Outlook is a great, shambling, brute of an application. In the fold of Office it has obliterated all traces of its betters, it stands unchallenged atop a mountain of the digital dead.

The beast is encrusted with the weapons of forgotten wars. Many barely work. Outlook 2003 [1] still lacks a cross-PST accessible object identifier, so neither Microsoft nor developers can create a robust object link. Sorting a categories view still undoes the view settings -- a bug dating back the 90s that was only fixed in Outlook 2007. Outlook 2003 tags (categories) are still stored in the registry and the master category list has been utterly broken since the dawn of time (happily, it can be ignored).

All the same, in a stupid and accidental sort of way, Outlook is powerful. If you can figure out where all the sharp edges are, if you learn to step around the explosive charges, if you have good backups and you know all the command line repair utilities and the caches and configuration files to periodically delete, it can be sort of tamed.

I have, over the course of many years of trial and pain, figured out how to make the combination of Outlook and Windows Desktop Search work. Yes, it periodically blows up in a very impressive fashion, but that's life with Microsoft. Only Excel is immune to this behavior.

The combination of full text search across GB PST files, the ability to effortlessly rename email subject lines, drag and drop transformation from one type (ex. email) to another (ex. task), the quick tag (category) and the ability to set metadata (category, due date) values by dragging and dropping between view "headings", the robust metadata query language and hierarchical view srtructures -- put them all together and one can painfully create a semi-stable workable PIM environment.

It only took years of work. I could write a book, indeed I've taught around a draft of such a book in my corporate life.

Apple could do far better -- if they wanted to. I don't think they want to. Microsoft could fix Outlook [1], but I sincerely doubt they care. There will never be a challenger to Outlook on a Microsoft platform. Google now ...

Well, I can dream.

Which comes, at last, to what I want.

We all know that managing work by email is the road to damnation. Work must be managed to tasks, which in Outlook are created by renaming subject lines [2] then dragging and dropping email to the task icon. Tasks get priorities, and depending on priorities they may be assigned due dates, calendar slots, and category attributes.

The process of creating these relationships between task, email and appointment could all be made much more fluid, but I'm asking for less than that. All I want is a quick way to find tasks based on the subject lines of incoming messages. Then I can update existing tasks rather than creating new tasks and then resolving the redundant tasks.

Put a button next to the subject line -- or on the email taskbar somewhere. When I redo the subject line [2] I'll click the button and the application will run a full text search using WDS to locate matching tasks. I'll then select the task I want.

I think an Outlook developer could probably do this. I'll pay $30 for this feature.

Go for it.

[1] I've used Outlook 2007 a bit, but we're still stuck on 2003. My sense is that 2007 has fewer bugs but probably no significant new features.

[2] I am running a one person campaign in a large publicly traded corporation for better subject lines. I think it's slowly working, but for the foreseeable future being able to effortlessly redo subject lines by clicking and typing in the subject field will remain one of Outlook's best features.

Monday, October 08, 2007

The iPhone is not all bad and Fortune's new Apple blog

Rob Griffiths has some kind things to say about the iphone: Macworld: Editors' Notes: Ten of my favorite iPhone things. He even pointed to an unconvincing, but not irrational, explanation for the stupid headphone jack.

It's a soothing story, it helps reconcile me to the bitter truth that I'm going to have to replace my wretched Palm Tungsten E2 with .... another E2. Not to mention that I will have to continue to live with my thrice cursed Motorola RAZR. [John takes another slug of scotch.] The iPhone doesn't do what I need.

I've largely given up on Apple producing the solutions for my n-of-1 market, though I have come up with a theory that suggests some missing features may have been belayed by the seven month slip of OS X 10.5. So I really need third party apps on the iPhone, because the long tail means even a market too small for Apple to notice can feed a hungry developer or two.

Which brings me to an excellent new Apple blog from (of all places) CNN/Fortune: Apple 2.0.

Apple to Open iPhone in particular smells like a leak from somewhere in Apple. It alleges that Apple is going to adopt a regulated development model for the iPhone similar to what Apple did for a few months to a year after the release of the very first Mac. I think it may have also been the development model for the Lisa. I think I could live with that, so I've another reason to hope for an iPhone in 2008 -- even if I have to buy 10.5 and spend $200 on yet another obsolete and increasingly flaky Palm device.

Continuing the theme of "things are not as bad as they seem" Apple 2.0 claims Apple's iPhone attack was manslaughter, not murder. It seems that iPhone 1.0 is held together by glue, bailing wire, and hope. Significant updates will destroy a small percentage of millions of unhacked phones, as well as a larger percentage of hacked phones. This is more plausible than one might think because Apple has a similar, but smaller, problem with OS X updates. Any major OS X update has a small, but real, risk of hosing the OS -- which is why I reboot my machines prior to an OS X update and don't touch it during the update process.

I think I've exhausted my iPhone patience now ...

Thursday, October 04, 2007

Winner of Slate's bluetooth headset review

The best Bluetooth cell phone headsets settled on the "Plantronics Voyager 520". I have a cheap Motorola BT that I really dislike. If I get another one I'll have this review at hand.

An RTF surprise: 850 KB to 40 MB

One of the reasons I really like Nisus Writer Express for OS X is that it uses Rich Text Format (RTF) as its native file format. If you're going to use anything other than Word the application must have the option of using either RTF or DOC as its native file format. Nothing else is acceptable at this time, though one day perhaps the OpenOffice file format will see wider use.

Today I discovered a surprising downside of Microsoft's version of RTF.

I have an 849 KB Word 2003 DOC file that contains a fair number of screen shots. I know Word is very good at bitmap compression, so I just pasted them in. I didn't bother creating PNGs and importing them (PNG is by far the best standard file format for screen shots).

I exported to RTF from Word and the output file was 40 MB. Obviously the images have expanded a bit, about 45 fold! I assume they're now uncompressed.

By comparison I created a PDF, choosing "High Quality" for the JPGs. The resulting file was 735 KB, but the images showed some JPEG compression artifact; they were not nearly as sharp as the original Word file.

I'm very curious to see how large the file will be that Nisus creates. Can RTF support embedded PNGs? Will Nisus convert the native Word images to PNG?

Update 10/8/07: I tested using Nisus Writer Express, opening the .DOC file and saving it as RTF. Nisus' RTF version was 1.4MB, so it was about 100% larger. That's a lot better than the 4500% increase in Word's RTF version. I'm not sure what kind of compression NWE is using. Incidentally, NWE could not render Word's Table of Contents for this document, and every time it starts up it nags me about a PAY upgrade. Great way to really annoy the customer.

Wednesday, October 03, 2007

The Fall of XP: Windows Live, Onfolio, WDS and emergence at work

I have not been happy with Apple lately. It is just as well, then, that Microsoft has chosen this time to remind me of that the "Dark Side" really means, as well as providing an interesting example of emergence at work.

I've written recently of how my work XP box has been experiencing accelerated entropic collapse. I've made progress on addressing many of the contributing factors, including realizing that Microsoft has two currently marketed products called "windows desktop search" with superficially identical interfaces but different functionality, bugs, and update schedules.

A problem remained however. Outlook was periodically crashing with varying error messages. Some of them, however, implicated Onfolio, my favorite Windows Feed Reader*. I couldn't figure out the problem, so I tried reinstalling. I downloaded the installer from the Windows Live Collection, but it quit with a meaningless "network error" (clearly a red herring). So I tried uninstalling, but the uninstall would hang shortly after asking me if I wanted to remove my feed list.

It took me a while to uninstall the damned thing. I had to restart a few times, then, when it hang, I went through every XP service running and, one at a time, I turned them off. After I was done with that it completed. I don't know if disabling all the services did the trick or if it simply timed out on the hung process and killed it, moving on.

I then reinstalled successfully from the Windows Live Toolbar "Gallery", searching for Onfolio and choosing "run" rather than download. We'll see if that works.

I simplified Onfolio's behavior as much as possible. In particular there's a "Windows Desktop Search" integration feature in Onfolio that allows WDS to search Onfolio Collections. Since I believe WDS, Onfolio and the Windows Live toolbar are all somewhat buggy, that kind of integration is just asking for trouble. I disabled it, I haven't done much with Collections anyway. I'll stop using them. Onfolio also installs an Outlook add-in I could remove, but I'm not sure if that won't cause more trouble.

Which brings me to emergence and the Fall of XP. Microsoft's Vista has not been well received. I'm sure SP1 will help a great deal, but it will still remain slow on older hardware. Microsoft really wants to migrate people off their old hardware onto new hardware and Vista. The problem is XP has been too good -- even though it's a crummy user experience compared to OS X.

The answer, of course, is to make XP unstable.

Is this a deliberate Microsoft strategy? I doubt it. It doesn't have to be deliberate. Microsoft has only to cut back on QA testing, increase the pace of software delivery (Windows LIVE), increase the rate of security patch delivery  and let nature take its entropic course. This is an emergent strategy, but it works just as well as a Machiavellian scheme.

XP will die faster than most people expect.

* OS X has great thick client feed (Atom/RSS) readers and lousy publishing tools. Windows has the world's greatest blog authoring tool and lousy feed readers. Shame.