OmniFocus for iPhone and OS X – a webdav odyssey and a really big problem with OmniFocus

(Note: Firefox 2/3/Safari 3/Blogger 2/1/draft is messing up my post formatting big time. I had to open this post in Windows Live Writer (yay!) to repair it. It’s quite an impressive set of bugs we got there!)

This post has more than a couple of updates. It gives a sense of how messy this iPhone transition is going to be.

Briefly, the two immediate problems are that OmniFocus can’t import tab or csv delimited data and iPhone synchronization requires a webdav server. The second is a hassle, the first is a killer.

Here’s how it all evolved.

--

The first post

I’ve been waiting eagerly for OmniFocus desktop/iPhone. It’s a key piece of my struggle to migrate off the Palm platform.

I'm a registered OmniOutliner Pro customer, so I get a good break on the full app price. The iPhone companion is $20. It sounds like there's a way to sync with a LAN WebDav server, I'm looking forward to directions on how to do that.

To learn about OmniFocus desktop and OmniFocus/iPhone you need to first download and test the latest Sneaky Peek Build. Then pay $20 for the iPhone app. The OmniGroup is great about free upgrades, so there's no financial downside to buying both apps now if you're sure they're right for you. I know the OmniGroup will make them work -- they have the talent. I'll have some early impressions in a few days.

The First Update: After checking out the pref files in OF, it's clear you need a real webdav server. Desktop OS X can be hacked to enable Apache's webdav services, but it's too much work for me. I'm shopping around for a nice Java webdav server I can run in my user account, but I think those are oddly hard to find. OS X Server has webdav services enabled, but that's too much trouble too. I probably just need to pay for Mobile.me. I might as well have Apple deduct a monthly stipend directly from my employer ....

Update: Ok, so neither XDrive nor Windows Live Skydrive provide webdav access (box.net might?). Free webdav access is pretty much nonexistent, and even commercial solutions are not common. Resin used to be open source with a good webdav service; whatever the owners may say it feels pretty closed source now.

The W3C has a WebDav server, but it feels dusty and is very XP/Unix centric.

I used to use a WebDav service in the 90s, it really feels like WebDav has slipped away since then. Really, MobileMe might be the only practical WebDav solution outside of OS X Server! Too bad GDrive has gone missing.

So why has WebDav become so unpopular? Too easy to abuse? Insecure? It's not like we see better file sharing services around!

Update: umm ... maybe webdav services with Plone? Here's Apple's download blurb on Plone, but contrary to the description it now downloads an Intel only unified installer version 2.5.

The Plone site (use versiontracker) has a UNIX installer for version 3.1.2, you need to run terminal to install it in your user account. Since I'm only interested for now in supporting webdav sync for the iPhone from my WLAN I installed it as a regular user rather than as a root user service. Be warned, however, after compilation the install is 250MB!

I think this may also not be a fully complaint WEBDAV implementation.

A thread in the OMNI Group Forums mentioned box.net, but that service is blocked by many corporate filtering services (which supports my suspicion that the reason webdav is so hard to find is that webdav services are often used illegally).

“Final” update: Well, I installed Plone into 10.4.11 with the universal UNIX installer and edited the .conf file to enable localhost:1980 webdav services. Incredibly, 10.4.11 finder worked perfectly. Unfortunately OmniFocus didn't. At first it gave a basic can't connect error message, on retry it crashed and I sent the OmniGroup a crash report.

Final update plus: After all of the above, I find in early testing that OmniFocus has basically no data import/export capabilities I can use.

Here's what I read from an OmniGroup Forum post:

1. If your organization has any development capacity, it is not difficult to write a specialised importer using Applescript. This would give you the most efficient workflow.

2. For a quick and dirty solution in the interim, you can use search and replace to convert your CSV text into tab-delimited text, and use Omnioutliner to import that. Omnifocus can import Omnioutliner files.

Hookaayyy. I can hire a developer to write an AppleScript importer to move Outlook tasks into OmiFocus?I don't think so.

So cross out OmniFocus! I'll keep the skeleton of my original post. At least I learned something about webdav services and Plone!

Looks like it will take longer than expected to come up with a good iPhone task management solution...]

Final-Final update plus: Ken Case of OmniFocus replies in comments:

Just thought I'd mention that we're planning to add the ability to import CSV files to OmniFocus, it's just not there yet. (The priority for 1.0 was to get the basic workflow and functionality working, and the priority for the in-progress 1.1 release is synchronization. Hopefully CSV import will make it into 1.2!)By the way, here's a pointer to the best tutorial I've seen on setting up WebDAV on your Mac.

The referenced webdav directions are for 10.5 workstation Apache and are very well laid out. The directions include lines like "be sure to use sudo", so they're not written for most users. Maybe OmniFocus will write a one-click utility to enable Apache WebDav? Lastly, I work in software development, and in my world "hopefully" means -- "if we're visited by space alien coders", but the OmniGroup may operate on a higher plane.

Bottom line: I'm going to be carrying a Palm around for a long time yet.

Uninstalling applications from the iPhone

I’m always curious about how application uninstallation works.

By way of comparison, the Palm never had a proper uninstaller.

I’m surprised I’m saying something good about the phone I’m supposed to buy tomorrow, but this is well thought out.

Syncing Applications from the App Store

… If you delete an Application from your iTunes library, the next time you sync your iPhone or iPod touch with that computer it will be removed from the device. If you delete an application from your device, but not from your iTunes library, it will be synced back to your device the next time your connect to iTunes.

If you normally sync applications to your device from one computer and then connect it to iTunes on another computer, iTunes will not attempt to sync applications from that library. Applications can only be synced from one iTunes library. If you are connected your iPhone or iPod touch to an iTunes library that you don't normally sync with, you can still transfer purchased applications from the device by choosing File > Transfer Purchases in iTunes.

It’s an extension of how music is managed on the iPod.

Remote control of iTunes and AppleTV: will AirTunes return?

Three years ago I fought a bloody battle with AirTunes, OS X 10.3, my AirPort Express, and iTunes remote control.

I lost. The microwave didn’t help …

… The devils of Digital Rights Management, AirTunes fundamental inadequacy, and the lack of a fast-user-switching compatible tool for remote control of iTunes finally defeated me….

… I was streaming some music using AirTunes. A rare event, but I do it on occasion. All was well, until the music vanished. I wondered what was up; then I realized the microwave was running. It's not all that old a model, but it is death to our 802.11b LAN…

Since then I’ve switched to a MacBook and a mixed b/g WLAN, but the MacBook is even more susceptible to microwave interference than the iBook was. (Though we also bought a new microwave, so maybe it’s just leakier than the old one.)

The post still gets read quite a bit, yesterday I corresponded with someone who’s just returned their Bose speakers because AirTunes failed.

Now Apple has, years late, added a remote control feature to the iPhone/iTouch:

Two Apple iPhone apps: Remote Control and Texas Hold'em - The Unofficial Apple Weblog (TUAW)

… 'Remote' [iTunes Store link] and oddly enough it lets you control iTunes on your Mac from your iPhone. You can stop, start, and pause music, and browse your library... all via your iPhone or iPod touch. Remote requires the use of a Wi-Fi network, and is free…

A bit late! This means more will try out AirTunes. I’m wary though. I suspect:

1. It won’t work with background sessions.
2. The AirTunes streaming will still be messed up by microwave use.

It’s not totally hopeless though. I haven’t tested AirTunes with 10.5 or the new AirPort Express. If the remote will communicate with iTunes running in a background user session on a 10.5 machine I might try testing again. The background user problem doesn’t apply to an AppleTV of course.

Update 8/18/08: Son of a gun, it works. It really, really works. I've been controlling my upstairs Library from my iPhone, streaming music to my AirPort express, and listening to speakers in two rooms. Years after the AirTunes hype died off, Apple finally delivered.

There's even intriguing/worrisome support for multiple libraries, which brings me to a comment from someone who's gone another step beyond me (note this only works if you wisely avoid the plague of DRM):

Jan sad ...

It looks like Remote with iPhones/iPod Touch and AirTunes is the solution for for the multi-room audio setup I was waiting for years to come.

I installed several AirPort Express boxes with AirTunes in the rooms and installed 3 users on a mac mini with fast user switching on. All users have their own iTunes setup and have access to a central NAS Server with all the MP3 files. This won´t work with Windows because Windows won´t allow fast user switching running iTunes !

With this setup every family member is able to hear their music independently on different AirTunes outlets. It really works !

Note Jan has one set of files, but 3 libraries. So each user can sync their iPod or iPhone with their own account and save their own Address Book and preferences, but share one set of music. I'm going to do this one day.

iPhone 2.0 pricing explained: two levels of "subsidy"

The new iPhone pricing represents a $360 2 year price increase for contracted non-iPhone AT&T customers compared to the old iPhone pricing.

This is because the old iPhone pricing model was a relatively good deal for contracted AT&T customers. AT&T has brought the iPhone in line Windows smartphones, the BlackBerries, etc.

This hurts. The 16GB model costs $299 for customers without a current AT&T contract, $499 for contracted customers.

Ok, but AT&T also extends the contract of an existing customer when they buy an iPhone. How can they do that if you're not getting some kind of compensatory value?

Well, here's the trick. I bet you were wondering what the "contract-free" $699 iPhone was about. Why would anyone every buy that? Why does that price point exist?

Well, think about it. Remember AT&T is extending your contract. What is it they're giving in return? The answer is $200 off the $699 price. In other words, a second level of subsidy.

AT&T may not intend to sell any phones at the $699 price point, but they need that price point so they can say customers are being compensated for their contract extensions.

Now, what happens when you lose or break your iPhone? Do you have to pay $699, or will AT&T offer you the $499 "subsidized" phone along with a contract extension? Might that depend on how much time you have on your contract?

The old iPhone plan, from a consumer perspective, was vastly better than the new, fully evil, AT&T iPhone plan. The good guys lost.

Sundry iPhone discoveries: none good

AppleInsider summarized 3 reviews.

There's no good news in the bunch; anything good has has been out for a while. My comments in parens.

1. With 3G enabled, the iPhone 3G's battery drained much more quickly in a typical day of use than the battery on the original iPhone. (Just as Jobs said a year ago. Many didn't believe power drain would be so bad.)
2. While iPhone Software v2.0 allows for both personal and Exchange email accounts to function simultaneously, synching Exchange calendars and contacts will erase any personal calendars and contacts. (Clearly there's a limit to Apple's genius. I'm definitely disappointed, but not surprised.)
   
3. Old iPod accessories meant to charge over Firewire circuitry (such as a Belkin car kit) don't work; iPhone 3G still only recognizes USB. (I'd never heard of this limitation. Annoying, but it's unlikely ever to change.)
4. The GPS receiver's antenna is too small to replicate turn-by-turn navigation of a full unit. (So the GPS companies should be ok.)
5. Battery replacement will still cost $86. (Still not user replaceable)
   

The worst news, by far, is that Exchange sync wipes personal calendars. In other words, the 2008 iPhone is no better than the 1996 PalmPilot (maybe a bit worse) at managing work and home calendars and contacts.

I will be buying my iPhone in the next week or so. I approach the date like a condemned man!

iPhone apps will not be able to synchronize with the desktop: OmiFocus

I'd asked again, recently, if iPhone apps would be able to synchronize with the desktop.

A month or two ago the Omni Group was writing about OmiFocus as though desktop sync would be possible.

Now they write: 

The Omni Group - OmniFocus for iPhone and iPod touch 

...Synchronized with your Mac via .Mac or WebDAV...

Yech.

Bad, but not surprising. This has been a hole for a long time, so we kind of knew Apple was going to short us on this one.

I assume Apple is guarding the sync conduits to ensure DRM of iPhone media. If so, it's a telling indicator of how DRM requirements will negatively impact iPhone usability.

Oh Android, I really do wish you were providing more competition.

Incidentally, Apple has given its own iPhone applications an enormous competitive advantage. Only they can synchronize with the desktop. So, Apple, any chance you're going to provide a task management tool?

No, I didn't think so.

Yech.

PS. How can Apple claim to be providing Exchange integration if the iPhone can't handle Exchange tasks at all?

What the heck happened to Canon?

As a rank amateur I’ve been asking about what the heck is wrong with Canon since they failed to boost light sensitivity in their low-end dSLRs.

Now the pros are beating on Canon for light sensitivity, auto-focus, and reliability (emphases mine) …

The D3, D700 and Canon - James Duncan Davidson

… I think the very next camera that Canon releases, which better be a 5D replacement at this point, is going to say a lot about how they intend to meet Nikon’s resurgence. If the 5D replacement is just a freshen up of the 30D to 40D variety and which doesn’t meet the challenge that the D700 brings, then Canon will be telegraphing that they’re happy with their market position selling the crap out of the Rebel XSi without worrying about the higher end. On the other hand, if they release a competent contender, then we’ve still got a two horse race.

What will be a competent 5D replacement? At a minimum, it has to have usable ISO 6400 that is as good as ISO 1600 on the current 5D and it has to have pro-level autofocus. I don’t care about more pixels at this point, though I won’t complain about a 16 megapixel sensor. Even with a larger sensor, it is the twin metrics of low to high ISO quality—two more stops at least over the current 5D—and capable autofocus performance that will tell the story. Anything below this threshold will say volumes about the direction in which Canon intends to take the platform…

Sorry James, Canon is in trouble at the low-end too.

Where did they go wrong? To me their biggest issue is sensor light sensitivity. Presumably they can fix the reliability issues, and some of their pro camera are felt to have good autofocus. The sensor though, is a scientific and technical challenge. If they can’t meet that challenge, Nikon will own dSLR business.

It's too late to short Adobe stock: Reader 9

As of today, Adobe's share price hasn't had a big recent drop.

Maybe insiders believe Google or Apple are really going to acquire them, just to get Flash.

Because looking at the latest release Adobe Reader knows Adobe is a disaster today. It's not just me, try Googling on "when did adobe go downhill"?

I'd guess they went off the rails a year or two before my adobe download manager post, so maybe January 2005. It would be interesting to know what happened then. Did some key people vest options and leave? What executive shuffle occurred? 

I won't be installing Adobe Reader 9 anywhere. I removed Adobe products from my OS X machines about a year ago, and life has been quite a bit nicer since. (Sure Adobe's photo editing apps are sweet, but they also show doom. How hard would it be to QA the app as a non-admin user?)

On XP I'm on Adobe Acrobat full (no reader). Eventually the gross security measures will force a reader update, at which point I'll switch to an open source alternative for ISO-standard PDF. I'm sure Microsoft will supply something, they're in far better shape than Adobe.

Digitizing a large CD collection

First I've seen this: Your Tech Weblog: Need your CDs digitized? Rent ripping robot. Too late for us, but worth noting.

Now I need something similar for my photos -- a high-end bulk load print scanner I can rent.

iPhone mystery: will Apple allow developers to do desktop sync?

I've been watching this for months. Despite all the SDK talk, there's no mention of whether Apple will allow app developers to sync with desktop apps (rather than with net apps).

This is a big deal for products like OmniFocus. Omni acts as though a solution is coming, but a week from go-live Mariner software doesn't know how this will work ...

Your Tech Weblog: Local firm making a spreadsheet for iPhone 

... He also needs [Mariner] Calc for iPhone to sync with Calc for the Mac, and is talking with Apple on how that might happen, but he has no idea when this critical hurdle will be overcome....

I assume that Apple wants very tight DRM and security for the iPhone, and that this has created their synchronization issues.

I wish Android were doing better. It would be nice for Apple to feel more competitive heat. As it is, they look ready to rule. For better and for worse! 

Precipitate: unify your online and local memories (files)

A few hours ago I wrote about my memory management meme. Things are moving even faster than I'd thought [1], because somebody at Google went and coded Precipitate:

Official Google Mac Blog: Precipitate: search your local and online docs 

... you're like me, some of your information is in the cloud and some is on your machine, and you don't always remember what is where. That can make it frustrating when you try to use your favorite local search tool to find something. Isn't the whole point of search that you don't have to remember where you put things?

That's where Precipitate comes in. After you install Precipitate, you can use Google Desktop or Spotlight to find files online (such as those in your Google Docs list) just as you would find files stored on your Mac...

Yep, that's what I need -- a tool to help unify my distributed memory. Precipitate currently supports only Google Docs and bookmarks. When they integrate Google Custom Search (as in my blogs) I'll try it out.

[1] joke.

My new number one Blogger request: fix backlinks with whitelisted URLs

I used the think that my #1 and #2 Blogger priorities were enhancing the BlogThis! bookmarklet and full support for Safari.

That was yesterday.

Today I thought differently about what my blogs are for, and where they are leading.

I've created a new category called "memory management" that will expand this idea, both here and in Gordon's Notes. More on that as I get to it, but it has nothing to do with "Quarterdeck Extended Memory Manager" (geeks of a certain age just had chest pains).

"Memory management" involves personal memory management and corporate memory management, private memory management and (this is new) public memory management, and an early (ok, so I was re-reading Idoru this am) version of gordon-google mind-fusion (one decaying, one growing).

Enough parens there?

Which brings me to my new #1 Blogger request.

Fix the backlinks.

First:

What are backlinks and how do I use them?

.... Backlinks enable you to keep track of other pages on the web that link to your posts. For instance, suppose Alice writes a blog entry that Bob finds interesting. Bob then goes to his own blog and writes a post of his own about it, linking back to Alice's original post. Now Alice's post will automatically show that Bob has linked to it, and it will provide a short snippet of his text and a link to his post. What it all works out to is a way of expanding the comment feature such that related discussions on other sites can be included along with the regular comments on a post....

Except backlinks very rarely appear on my blogs, and they NEVER include backlinks between posts on my domain.

Now you might think this is because Google never indexes my blogs -- which is how they claim to create the backlinks, but, honestly, Google is astoundingly quick to index all my blogs considering their negligible readership.

What I think happened is that the original purpose of backlinks collapsed due to fraud, webspam attacks, and search engine optimization. Google has given up on them for all but very high end blogs, and one of their defenses has been to block backlinks within blog domains (to reduce search engine optimization and link farm fraud).

Ok, that's fine, but backlinks are an aspect of what we used to call "backward chaining" in inferencing systems. In people-speak they allow one to explore semantic connections (insert obligatory semantic network, xanadu, memex, etc reference) to antecedent or precedent posts.

This capability is a strategic component of my personal memory management obsession.

So I want Blogger to create a new sort of backlink -- to posts that are within domains that I specify. I would create a set of whitelisted urls for my blogger account, and links from those urls to a specific posts would always become backlinks. I could remove them if I wished of course. To avoid linkfarm abuse Google would exclude this type of backlink from their value estimation algorithms.

This, then, is my new number one Blogger request: Create backlinks based on whitelisted URLs.

PS. As of first posting a search on "URL backlink whitelist" returns no meaningful hits. I wonder when that will change...

Update: 7 hours after the initial post the "URL backlink whitelist" search returns two meaningful hits -- this post and my secondary Gordon's Notes post. Actually it probably happened much faster than that, my embedded search had a typo in it. This sort of thing is really astounding, even though we increasingly take it for granted.

Blogger a mess with Firefox 3

I'm rewriting this post.

Every other post I've written over the past 3-5 days has had problems with lost line breaks. All my text runs together. It's as though Blogger had broken their age-old management of paragraph breaks. I've tried Safari 3, Firefox 3, ScribeFire, XP, OSX, Blogger-standard and Blogger-in-Draft.

My original post implied the problem was with ScribeFire, Blogger in Draft, and Firefox 3. Then I thought it was Firefox 3 and any version of Blogger. Now I think it's any version of Blogger with Firefox 3 and Blogger-in-Draft with Safari 3.

Basically Blogger is having a really lousy holiday weekend.

Be warned.

They'll fix this eventually. It's not hard to spot!

How to know it's time to stop reading a blog

"Mobile Opportunity" has been an occasionally fun read for a veteran of the Palm wars, but every so often it says something like this:

Mobile Opportunity: Symbian changes everything, and nothing

...Here's the weird thought for the day: Microsoft is the last major company charging money for a mobile operating system...

You might think he was excluding the iPhone OS because it's derived from a desktop OS, or you might think he excluded OS X because it's bound to hardware, but I've been reading MO for a bit. He really has no interest in the iPhone as a mobile platform.

That's just too odd.

On the other hand his link to the Register's Psion retrospective is party redeeming. The story reminds me of the sad tale of PenPoint -- I keep that book next to my OS/2 architecture book.

Dan's data reviews IDrive online backup, and mentions Mozy too

I suspect the "paid review" model will eventually produce the same results as PC Magazine of old, but it will take a while to corrupt Dan's Data. A recent review is very thorough and appropriately technical:

Review: IDrive online backup service

... The reason why I find IDrive particularly interesting, though, is that they're paying me via ReviewMe.com to write this review.

ReviewMe isn't one of those scummy services that allow payment to be conditional upon a favourable review, though. Whether I say IDrive is humankind's finest creation, or that it took both of my legs off at the knee, I get paid the same....

Dan's review of IDrive is pretty positive, though he only does XP stuff.

I've been looking for a good online backup solution for a while, to supplement my current Retrospect Pro USB disk solution. Apple's MobileMe might produce something, but it will be costly. I've about given up on Google -- they could deliver a service next week or next century.

DD says both IDrive and Mozy support XP and OS X. IDrive is setup for multiple accounts sharing the same space, that would work for me.

Incidentally, the primary reason to have offsite backup in Saint Paul MN is not fire, flood and quake, it's theft. True, smart thieves won't bother to steal hard drives, but most thieves are stone stupid. Elsewhere, fire and flood are right up there.

I wouldn't use these services as my primary backup solution, but as a supplementary solution they do interest me.

Update 4/11/09: Great comment below. At least some services (iDrive) don't back up OS X shortcuts/favorites -- which breaks some app data stores including some iPhoto and Aperture configurations. It's a good reminder to check how well security attributes are managed.