Sync heck: CalDAV vs Exchange Server - a Google Apple review

Two years after writing "Synchronization is Hell" I'm pleased to report it's been only a bit heckish of late for my iPhone and our family Calendar.

Three months ago I switched my iPhone sync to

• CalDAV to multiple Google Calendars, including my wife's family calendar, my personal calendar, and various school and sport and social calendars all shared via Google. (iCal also subscribes to these. It's useful when I want to see the most data, but it's not essential.)
• MobileMe for Personal Contacts. MobileMe manages Contacts sync for me across multiple OS X machines.
• Microsoft's "Exchange" (ActiveSync) for my corporate Contacts, Calendar and email.
• Gmail for my personal mail.

Yes, it's rather complicated. Life would be easier if the iPhone could handle multiple ActiveSync accounts - then I could also sync to Google using ActiveSync. I lost access to my Google Contacts when I switched my single iPhone ActiveSync client to the corporate server. I think I'm now ready to retry Spanning Sync, which I gave up on about a year ago and take another stab and "Project Contacts".

Despite the Contacts loss and some quirks, it's been a miracle to finally have work and personal and other calendars all in one place. I have a fairly full and complicated life, and being able to coordinate calendars this way has been genuine progress.

There are a few limitations with this I never see mentioned elsewhere -- so here's the exclusive list:

1. On my iPhone I can't move an appointment between calendars after I've created it
2. Exception handling is quirky. I can set an appointment in Google to workweek only, but it may not stay that way on my iPhone.
3. I can't invite people to appointments created on the iPhone CalDAV account, even though Google supports invitations. I can do invites with the ActiveSync account.

See also:

• iPhone sync to Google contacts - 3 methods and work/home implications
• Project Contacts: Now mixing Outlook/Exchange, PST file, Outlook/Home, MobileMe Sync, OS X Address Book and the iPhone.
• Using OS X 10.5 iCal with Google CalDAV - cleaning up import disasters
• Address book sharing with OS X and MobileMe
• Project Contacts: Integration across iPhone, Google and whatever
• Work home contact integration: Outlook to Google to OS X Address Book
• MobileMe: Integrating Work and Personal Contacts

Update 4/14/2010: Currently my iPhone can't edit CalDAV synced appointments that have invitations. Really annoying. iPhone OS 4 will support multiple Exchange connections so I'll try using ActiveSync instead when that's out. I don't know if this is a CalDAV, Apple or Google issue.

Access denied: VMWare Shared Folders on Windows 7

Between Dell machines shipping with motherboard disconnected SATA cables, a Clampi Trojan on my Windows 2003 server, a mysteriously vanished backup [1], Windows 2003 blue screening on a new Dell workstation, a failed Acronis disk image and the horror of 64 bit Windows 7 it's been another fun week in tech.

Today was a bit better. I installed VMWare Player on a 64bit Windows machine and created a 32 bit Windows 2003 VM - giving it all four cores and 3GB of memory. After VMWare tools installed and I enabled hardware graphics acceleration it felt faster than on the prior 3 yo workstation lost to the wretched corporate refresh cycle.

Mostly easy, until I dealt with the second drive in the box. Even after I used shared folders to map to drive E:, and the ancient DOS subst command to assign a drive letter to the shared folder, I still couldn't write to the new shared folder. I could read, but I couldn't delete anything. If I tried, I got an "access denied" message.

Worse, it seemed I could write to the drive, but the data wasn't there. VMWare showed files as having been copied, but in Win 7 they weren't there. On restart the VM didn't see them either.

The fix was to right click the drive letter in Windows 7, choose properties then security, and allow "EVERYONE" full control of the E: Drive. Then Windows 2003 in the VM could read and write.

I suspect there's a less severe fix. After I left work it occurred to me that I should study the read/write permissions on the C: drive. I suspect the vmware_user (__vmware_user__) group has special privileges on that drive, I just need to study them and replicate them for the E: and F: drives.

I'll update this post with what I find.

PS. It's unnerving that my Google searches really didn't turn up anything useful on this topic!

[1] Retrospect Pro backup on external drive. It vanished! Dir *.* and Attrib *.* showed nothing with 325 GB used. Retrospect could find the files though -- it restored from the backup drive.

--
My Google Reader Shared items (feed)

Time Machine, Time Capsule and offsite backup

Time Machine, among its other significant weaknesses, is ill suited to offsite backup – especially when it is used with Time Capsule.

If you attach an external drive to Time Capsule there is an “archive feature” that will shut down TC access and safely transfer the data to an external drive. Problem is, neither the drive nor the backup are encrypted.

Carrying around non-encrypted backups is not a good idea.

There’s a similar problem with a standard Time Machine external drive. If you swap them, you run into the same encryption problem.

The answer for an external drive is to use an encrypted disk image and mount that for TM backups. That doesn’t, however, work with standard Time Capsule archive behavior. I suspect one might be able to disconnect all clients and use the finder to copy the disk images to an encrypted disk image, but I’ve not tested that.

See also:

• Time Machine - Unable to Complete Backup bug on Time Capsule
• Freeing up Time Capsule space – and documentation for Time Machine and Time Capsule
• Time Machine- Fail first, then flaw discovered
• Time Capsule 10.5 Backup volume could not be mounted bug

Time Machine - Unable to Complete Backup bug on Time Capsule

I'm not a fan of Time Machine. I run into more TM bugs than I ever saw with cranky old Retrospect -- and I get less notification of problems.

Today I found a critical laptop hadn't been completing it's TM backup for about 2 days. It was showing a common, unhelpful, error message "time machine an error occurred while creating the backup folder".

David Alison's Blog has the answer for a standalone time machine. Turn off TM on the affected machine. Then navigate to your TM backup folders and look for something like "2010-02-01-134046.inProgress". Delete the "inProgress" file.

This doesn't work for Time Capsule though. TC won't let you delete the inProgress file. I couldn't find any report of a fix, save dragging the actual machine specific TC sparse disk image to the trash and starting over. (The official response to all similar problems, by the way, is to wipe the entire TC disk and redo ALL machine backups).

What worked for me was to turn off Time Machine on all the TC clients. Then I restarted the AirPort and then turned TM back on for the troubled machine. The backup chugged away for a while, and then it resumed.

See also:

• Freeing up Time Capsule space – and documentation for Time Machine and Time Capsule
• Fixing the Time Machine / Time Capsule 10.5 "Backup volume could not be mounted" bug

Windows 7 is OS X Warp(ed)

One of my work machines now runs Win 7. It’s the first time I’ve had to do more than play with it.

It helps to know OS X, but it also hurts. There’s a lot of stuff in Win 7 that’s a tasteless and ugly version of OS X. Take the desktop themes (please).

Hard to say if it’s really an aesthetic improvement even over XP. The XP interface feels light, sharp and clear by comparison.

Update: For example - "Program Files (x86)". Thousands of Google hits puzzling over that one. WTF were they thinking?

Update 2/4/10: OS X managed a smooth migration to 64 bit. I've had a few days of experience with the Win 7 mix of 32 and 64 bit ODBC, Oracle, Java, Microsoft Office, SQL Developer, etc. It's a train wreck. It brings back memories of early DOS experiences. This 2007 tech doc tells one part of the dreadful story.

Computing keeps getting more bizarre

At home I’ve retired my six+ year old XP machine. It lives on in a cloned Fusion VM [1] on my iMac. The dead hulk of the machine waits for anyone who might make use of it, but it’s most likely headed to recycling.

It’s a relief to be done with it. It worked well enough to the very end, but it was a flaming security hole (no antiviral software – that cure is worse than the disease) and it howled like a demented banshee.

At home the four Macs and three iPhones are quiet. So quiet I now notice the ever running fan on my G5 iMac, a fan I never heard when the XP box lived. OS X is kind to me. It all just works.

At work though, I still live with XP. Not just XP, but XP layered with monitors, automated maintenance systems, encryption software, automated backup software that isn’t useful, misguided and aborted security layers and only Satan knows what else. At work, computing is bizarre. I don’t think my workplace is atypical; I suspect this is true of many large publicly traded companies.

Consider this.

I reboot a Windows 2003 box after a failed disk cloning attempt to discover the boot disk is hosed. [2]. So I take a look at my personal backups (since the corporate backups are effectively useless) and find the disk has no files.

Nothing.

But Retrospect Professional (Windows) shows the backups have been working.

Nothing will show any files. Chkdsk reports no errors. But 325 of 350GB are in use.

So I try a restore from Retrospect – and it works.

The files are there, but invisible to cmd.exe. (No, not marked as hidden, truly invisible).

I suspect some side-effect of an cryptic corporate attempt to secure/encrypt USB peripherals. It’s not worth trying to debug this – I don’t have enough control over the pieces.

I have to assume we’re reaching some nadir of corporate computing – that things will improve somewhat with a migration to windows 7. It is ever more clear, however, that those of us who are cognitively dependent on our computers will need to have our own computers and network access at the workplace.

Which is good news for the iPad.

[1] Which is periodically slow and awkward on my quad core 10.6 machine compared to Fusion 2 on an older MacBook. Fusion 3 on 10.6 quad core needs work.

[2] Could be a side-effect of the Acronis disk cloning, but I doubt it. I suspect it would have been hosted on any reboot – that machine hasn’t been restarted for weeks.

Online backup – the security problem (it’s not the encryption)

Here’s how you lose everything.

First, someone gets control of your email account. It might be a security vulnerability, or a password attack (note: “tigger”, “angel” and “soccer” are not wise choices), or a password reset, or an inside job.

They then sell your email to someone who takes a look, and finds a backup report from, say, CrashPlan. They then reset your CrashPlan password:

Please submit your email address. Afterward you will receive an email with a link that will reset your password and securely display the new password to you. The provided link will only work for one hour.

Now they have access to everything you’ve backed up.

CrashPlan talks about their 128-bit Blowfish encryption (standard) or 448-bit CrashPlan+ encryption and how robust that is. As Schneier used to point out before he was overwhelmed by the boredom of it, this is rather besides the point. Their use of the industry standard “password reset by email” process means they’ve built a solid steel door on a house made of rice paper.

It’s not just CrashPlan of course. Google is little better. This reset problem is just one aspect of how broken passwords are (don’t get me started on “security questions”. Please.)

CrashPlan also offers a “data password” that encrypts at the client side. So even if someone gets control of your online backup they can’t actually do anything with the data.

Except … Well, CrashPlan’s FAQ dodges around this, but since the encryption is client side they can’t make any changes to whatever you’ve already backed up. So if you want to add, or change, your data password you have to wipe your online backup and start over. If you change it, but don’t start over, you better keep your old and new password since data may be encrypted with one or the other. In my home a full family CrashPlan offline backup takes about 4 weeks, so this is not a trivial change.

Note that I’m using CrashPlan as my example here because they’re the best in the offline consumer backup business, and they are the only offline backup plan I’ve considered. They just have the usual problem with their password reset procedure.

How could CrashPlan make the best of a bad situation? Well, in the unlikely event that they read this, they can research higher quality reset procedures (not #$!$!$ security questions). Those reset procedures often involve two factor authentication procedures, such as the procedure myOpenID almost got right. They involve more expense, so it would be reasonable to for CrashPlan to charge extra for a higher quality security service. They really don’t need more encryption, they need better reset controls.

In the meanwhile this problem has tipped me away, for now, from using offline backup. I’ll continue to rely on physical drive rotation for offline security and I may make use of CrashPlan’s (free, unfortunately – I distrust the longevity of free things) ‘backup to friend plan.

Update 2/4/10: For more on CrashPlan.

Update 5/17/10: Matthew Dornquast of ChrashPlan replies in comments.