XP doesn't like my OS X SMB share

XP really doesn't like my OS X 10.5.6 SMB share. I tried copying about 50 GBs of files from my XP box to the OS X SMB share over a 100mbps wired connection using xcopy and robocopy. I'd mapped a drive letter to the OS X share.

Problems included:

1. Time stamps aren't processed correctly. So even though date attributes appeared to copy correctly, both xcopy and robocopy interpreted the dates incorrectly (treated them as older than the XP source files).
2. After processing for a while the transfer would halt, and I'd lose all network connectivity (no ping responses). The usual error message was "the specified network name is no longer available", sometimes I saw "The semaphore timeout period has expired". It looked like the XP network stack had blown up. If I opened up a second terminal session, and just browsed the distant share things would restart.
3. Inevitably I'd start getting "Access is denied" messages with some files. They appeared to be file associated, but I could copy them in Windows Explorer. Once I got the message with a file it was persistent.

Wow. Really bad stuff. Following Microsoft's advice for network troubleshooting I tried setting the ethernet limit to 10mbps but the behavior didn't change.

I've had pretty predictitable problems with simply copying large numbers of files between XP machines using Explorer on many machines, but xcopy and robocopy are pretty reliable.

Something's definitely broken here, but danged if I can figure it out. My best guess is that the OS X SMB share is doing something that's cracking Microsoft's fragile SMB stack, but really it could be a network adapter hardware issue too.

I suspect I won't be able to solve this one easily. I'll try copying the files using the OS X machine and see if things work any better.

Update 2/18/09: I had no problems copying the files using the XP machine as a server and the OS X machine as a client. The date stamps on directories were also correct, I don't think the OS X SMB server sets copied directory dates equal to source directory dates.

My best guess is that the miserable behavior I'm seeing is not a hardware or network issue, but rather the result of bugs on both sides. My past looks into Microsoft's networking protocols have convinced me the "Heart of Darkness" has nothing on Redmond's legacy networking infrastructure. On the other side, I doubt Apple is putting a lot of energy into making OS X into a respectable SMB server.

Update 2/21/09: I’m now seeing crashes of the XP network stack, whether I use the OS X machine or the XP machine as the SMB server. Before the crash I sometimes see transfer rates slow. I can restore the stack by disabling my XP LAN connection then re-enabling it. I uses an new cable to directly connect the two machines (no router or switch) with no change – so it’s not a network problem.

I’m now copying files to an external USB drive which I’ll then move to the OS X machine manually. The process is exquisitely slow on the old XP box compared to throughput over 100 mbps LAN. The average real world throughput on the LAN was 6 Megabytes/sec, on my USB 2 XP connection it’s probably less than 1MB/sec.

I’d love to blame all of this on Microsoft’s SMB and Apple’s SMB implementation – that’s what I thought on 2/18. Alas, that’s too easy. This problem is too severe – and now I’m noticing problems restarting my XP box. It could be some nasty bug/virus/etc, but I actually suspect the motherboard itself is failing.

I’m not sure how old this machine is, I’m guessing about 2003. It is the first computer I’ve owned that died of old age while it was still quite useful.

Update 2/21/09: Now that I've moved everything and have been cleaning up the XP box, I notice I had Windows Search 4.0 set to index an iMac SMB share mapped to a local drive letter. Hmmm. That might put some extra stress on SMB. I'll see how things behave after removing Windows Share (I don't need it now for the XP box, I can use Spotlight on the iMac).

Update 2/22/09: I've been pushing GBs across the LAN using Retrospect Pro 7.1 (Windows) without a problem. I think the hardware is fine. I think the OS X SMB support is probably not immensely worse than native Windows SMB. Now I'm thinking I pushed the envelope a bit far when I had Windows Search 4.0 indexing a remote OS X SMB share.

Update 5/6/09: Replaced my NIC and things look much better.

VMWare Converter - turn your XP box into a VM platform

I love the competition between Parallels and VMWare. I very much hope they both thrive ...

MacInTouch: timely news and tips about Apple Macintosh, iTunes, iPhone and more...

... Along with last week's release of Fusion 2.0.2, VMware has released VMware Converter 4.0, a free standalone program that can create a VMware virtual machine from a physical Linux or Windows machine and can convert VMware virtual machines between platforms. This release brings support for converting new third-party image formats, including Parallels Desktop virtual machines, newer versions of Symantec, Acronis, and StorageCraft. Registration is required for free download...

My old XP box may be failing -- possibly due to a CPU fan issue (which would be fixable if I cared enough [1]). Time for me to move all the data off to the iMac server and then make a VM from what remains. Then, if the machine really does fail, migration to a new iMac or even the "legacy" plastic MacBook (by far Apple's best value) would be only a few hours work.

[1] It takes up a lot of space and power and it can't run most of the apps (OS X) I prefer. I'm not ready to throw it out, but I wouldn't mind replacing it with an OS X machine.

OS X Address Book to Google Apps Contacs using Google ActiveSync

I'm not the only one obsessed with integrating OS X, the iPhone and Google. I found a Google iPhone Help group thread full of fellow crazies.

From that thread I learned that OS X 10.5 Contacts, astonishingly, has its own private abilities to sync with Google Contacts, Yahoo Contacts and ... Exchange Server.

Huh? Why Address Book and not, say, Calendar?!

Good question.

Naturally, a fellow maniac is now using Google's iPhone oriented ActiveSync service to sync their OS X Address Book to Google ...

Google Sync iPhone = Mac OS X Address Book orphaned - Google Mobile Help

1) Go to your Address Book preferences, General Tab.
2) Then check the Exchange box and click on the Exchange button.
3) Enter your Google Apps username (first part e-mail in the E-mail field.
4) The password in the password field.
5) In the Outlook Web Access Server field, enter http://www.google.com/a/yourgoogleappsdomain.com.
6) Check the sync every hour box...

Holy Cow. I'm seriously tempted to try this. The catch is it's easy to backup and restore my OS X Address book, but it's very hard to restore my Google Contacts -- particularly their Group memberships.

Update: See Joe's comment below. I found similar comments elsewhere. This doesn't work with Google's version of Active Sync.

Update 7/1/09: Wade has an interesting comment. I may try this again with the current OS release but I'm also waiting for the new release of Spanning Sync 3 and its implementation of the Google 3.0 API.

Gmail iPhone web app has Google Contact and Gmail search. Who knew?

I can't find the post, but I know I've complained how weird it is that Google's iPhone optimized Mobile app didn't include search for Gmail mail or Google Contacts.

Wrong.

It's had Gmail search for months at least, maybe longer. Problem is, the search screen is in an odd place. It's way down at the bottom of the email display screen. You can double tap to to jump to the top of an iPhone screen, but I don't think there's a shortcut to jump to the bottom. This should either be at screen top or on the "More" menu.

There's also Contacts search. Go to Contacts from the main menu. Again, scroll to the bottom. Yes, there's a search button there too. Unlike the general Mobile App search function that runs against the iPhone's Address Book, this one runs against the Gmail Contact list. Like the newly improved web site search it searches all strings associated with a contact -- not just name and company.

None of this functionality is documented by Google. To be clear, these aren't related to the iPhone search function that Google actually markets -- that's an integrated search function that looks at the OS X Address Book as well as the web. These functions are much more useful. I'm exasperated that I didn't know about them.

Why so useful? Age and a congenitally bad memory for names. Age means that not only is my already bad name recall worsening, but the number of names in my Contact list is steadily increasing even as I'm expected to remember more of them on a daily basis. I'm over 1,500 contacts now; scrolling is not an option.

I'd love to have full text search across my entire iPhone, and it's aggravating that Apple doesn't provide full text search across the iPhone Address Book -- but again Google comes to the rescue. When I can't remember a name I can now search my full Google contact repository by city, by notes, by nickname, by any of a number of clues -- such as the names and keywords I attach to Contact notes. It's all part of my Google-powered prosthetic memory strategy.

Now if Google would be so kind as to move the $!$#!%#$ search screens to a more obvious and accessible location?!

Update 2/14/09: If you tap the "More" menu item you'll see a "Previous" and "Next" buttons in mid-screen. They hop to the top and bottom of the screen. Weird, but a good shortcut. I expect Google will soon split Contacts out into their own Mobile App screen; they're gradually doing that with Google Apps and Gmail. That will be handy.

Google books for the iPhone - not ready

Google has released an optimized version of Google Books for the iPhone. It sounds like quite a challenge. I tried it with Machiavelli's The Price and Sun Tzu's The Art of War (free only).

No luck. The search was surprisingly poor. Enter "The Prince" in Google and the first dozen or so hits are on the book, enter the same string in Book Search and you get an odd mix of texts.

Worse than the search, the OCR was pretty lousy, mostly because the underlying images were also lousy. Looks like the high speed scanner was having a bad day.

No threat to the Kindle here ...

Work home contact integration: Outlook to Google to OS X Address Book

One of the secrets learned on the wrinkly path is that simple solutions are very hard.

Obviously, not hard to describe or apply. They're hard to invent, because there are so many solutions, simple and complex alike, and most of them aren't good.

Lately, I've been starting to tackle a gnarly problem - managing my Contacts (address book). It's a big problem ...

Gordon's Tech: Project Contacts: Integration across iPhone, Google and whatever

... There are a lot of variables in the mix, from endpoints to transition tools. Gmail Contacts, Google Apps domain shared contacts, Google Contacts API and extended Contact Kind, OS X Address Book [1], iPhone Contacts.app, Microsoft Exchange Server, ActiveSync, SpanningSync, FileMaker Pro, Bento, MobileMe, gSyncIt, iTunes sync, Google iPhone Sync, vCal, hCard, GData, OpenSocial, Windows Live, Facebook and Outlook among others. (Yes, several others.)...

... Yes, this one is the big Kahuna. Big enough that, for this project, I’ve created a new Gordon’s Tech Label of “Contacts” to help me as I work through all the options.

In the meantime, there’s one big bit of good news. Contacts don’t change as much as Calendaring events. I can get by with manual export/import for quite some time...

It took me quite a bit of thinking and playing with the variables to come up with that last sentence. I explored a lot of very complex approaches, only to be stymied by problems with data structures and a lack of adequate OS X desktop tools.

Eventually I decided to scope the problem strictly to creating a pool of Contacts in OS X Address Book that would sync to my iPhone, and a slightly different pool of Contacts in my Gmail account. Once I did that, the problem became a lot simpler. I now have 1,600 Contacts on my iPhone.

A diagram would help here, but I don't have time to make one up, so here's a skeleton of what I'm doing. It's all manual, but I only do it every few weeks, because (this is the key), Contacts don't change as much as most things.

• Exchange Server/Outlook -> Export CSV file -> Google Gmail import into a "group" called "Work Contacts". Since this is a simple one way export I clean out this group before the import. I do this every few weeks.
• Gmail export -> vCal file -> OS X Address Book. When you do this Address Book adds items to the main group, but it highlights those just imported. You can then carefully drag them into a "group" called "Work Contacts". Incidentally, OS X Address Book duplicate management is very sophisticated -- the best I've yet seen.
  

I think I'm going to discover some other interesting nuances. In particular Google Apps now provide an intriguing pooled Contacts collection that may facilitate a family Contacts pool.

Still, this is substantial progress for me.

Update: BTW, the default vCal behavior for OS X Address Book does not export photos or notes. You must turn these on in options. The vCal from Google does export notes and photos. Address Book imports the notes, but not the photos -- or at least not during a merge/update (this may be a bug).

Update 2/18/09: Oops. Corporate exchange servers often use x.500 email addresses (aka EX format, for "exchange") internally. Those aren't useful to Google or my iPhone. I recall older Palm sync solutions substituted net-standard email addresses during sync; I need something like that. So there's still some work to do. I'm not the only one dealing with this ...

Update 2/19/09: Inspecting the output file I see that Outlook 2007 exports CSV file emails in this form:

E-mail address E-mail type E-Mail Display Name
fredflintstone@bedrock.zz SMTP Fred Flintstone
/o=Megacorp/ou=North America/cn=Recipients/cn=xdf415 EX Barney Rubble (Barnie.Rubble@Megacorp.com)

So the x.500 (EX) addresses contain an employee identifier ('cn') that maps to a current email address which is stored in parentheses in the E-Mail Display Name field. The SMTP style addresses do not display the email address in the E-Mail Display Name field.

So really, we want Google to be smarter about importing. It ought to "know" to pull the email address out of the [E-Mail Display Name] column when [E-mail type] = "SMTP".

This is rather annoying.

Someone must have written a utility that reads in this export file and post-processes it to extract the SMTP address and store it in the E-mail address column for EX type rows. I'll have to look for it. Meanwhile I'll ask Google to fix their import processing.

Update 2/19/09: I tried linking from Access 2007 to Outlook 2007, but this produces a very peculiar view of email addresses. The old data model problem no doubt -- the UI for doing this looks very old, so I suspect this function is really obsolete. I then exported as Access 2003 from Outlook 2007 and got something similar to the CSV file export. On inspection, however, it became obvious that Outlook's storage of SMTP equivalents for EX email addresses is pretty erratic. Indeed, it looks like even the EX addresses stored in Outlook (vs. on Exchange server) are flaky. So I'm beginning to understand why Google doesn't want to try to get clever with processing the CSV files -- the email addresses are unreliable.

I feel so sorry for programmers who try to work with this stuff. Outlook is really decrepit.

I'll need to think about this a bit more ...

Update 5/19/09: Major progress. Among other things, I learn that if you copy .OST Contacts to a .PST file Outlook resolves the X.500 EX addresses to SMTP addresses.

Project Contacts: Integration across iPhone, Google and whatever

I was just about ready to turn my iPhone into cyber-mulch when, unexpectedly, Google saved it. Calendar nerdvana has come to my iPhone, albeit at some cost to my iPhone battery. Again, I have hope. (Yes, Andrew, I am obsessed.)

I need hope. My PIM/PDA world is still a post-Palm mess. At the moment Google is my source of Calendar Truth, my iPhone is a sync client, and iCal is not involved. I’d like to have a copy of Calendar data that I “own”, so I’ll eventually re-integrate iCal. I’m no fan of Apple’s calendaring monstrosity, so there’s not a great rush. Sometime this year though.

In any event, it’s time for a refresh of my Palm Migration Status page:

Things have changed quite a bit over the past few months. It’s been a lot harder than I ever expected, and there’s still a way to go. (I hope the Palm Pre is a smash success, if only for vicarious revenge upon Apple for MobileMess.)

Today my old Palm Classic PIM data is scattered across my iPhone, Google, OS X desktop, ToodleDo, Evernote and Jott. That has to get simpler! I’ve made a “strategic” family and personal commitment to Google, so if/when they integrate their inadequate Tasks with Google Calendar (here’s how to do it) I’ll drop ToodleDo from the mix.

I could drop Jott fairly easily as well, and Evernote is improving nicely, so for now my focus is Contacts. This isn’t as critical as Calendar integration, but I’d sure like to get all my work and home Contacts on my phone.

Unfortunately, Contacts are hard, much harder than Appointments, Notes, Tasks, etc. (Ok, so Appointments have their own special non-technical problems too.)

There are a lot of variables in the mix, from endpoints to transition tools. Gmail Contacts, Google Apps domain shared contacts, Google Contacts API and extended Contact Kind, OS X Address Book [1] OS X Address Book Google Sync [2], OS X Address Book Exchange Sync [3], iPhone Contacts.app, Microsoft Exchange Server, ActiveSync, SpanningSync, FileMaker Pro, Bento, MobileMe, gSyncIt, iTunes Google sync, iTunes Address Book Sync, Google iPhone Sync, vCal, hCard, GData, OpenSocial, Windows Live, Facebook and Outlook among others. (Yes, several others.)

Bad enough, but “Contact” information is a big part of vendor lock-in strategy. Yes, Data Lock, big time. We’re talking Google, Microsoft and FaceBook fighting over ownership of contact relationship networks, and everyone fighting with banks, Amazon and the phone companies for identity ownership.

Complexity, corporate combat, rapidly shifting implementations, vendors focused until recently on Calendaring …

Yes, this one is the big Kahuna. Big enough that, for this project, I’ve created a new Gordon’s Tech Label of “Contacts” to help me as I work through all the options.

In the meantime, there’s one big bit of good news. Contacts don’t change as much as Calendaring events. I can get by with manual export/import for quite some time.

In the meanwhile, I’ll be experimenting carefully. Follow the Contacts thread forward to learn what I learn …

[1] OS X iCal is miserable, but OS X "Address Book" isn't all bad. If Bento weren't so crummy I'd put the OS X "Address Book" plus Bento at the core of my strategy.

[2] Update: Since my original posting, I discovered through this thread that OS X 10.5 Address Book has its own options to synchronize with Exchange, Google, and Yahoo -- in addition to the Phone to Google synchronization built into iTunes/OS X (available for 10.4 and 10.5). The mind boggles. So one could synchronize the OS X Address Book with Google Contacts directly, or through Google's Active Sync service ...

Update 5/19/09: Major progress - I have work and home integrated and options for Google too. The cost is I may have to subscribe to MobileMe.