Saturday, September 25, 2004

Rescuing a wrecked iTunes Library -- one approach

Usenet: Larry Bickford
... from the iTunes Help menu comes this info: 'If you throw away the iTunes Music Library file, your songs will no longer show in your library even though the song files are still in the iTunes Music folder. To make your songs appear in the library again:

Drag your iTunes Music folder (by default, located inside the iTunes folder in your Music folder) to your library in the Source list.' (They mean, to the open window where the songs used to appear)

'You will see the songs in your library again, but you won't see any of your original playlists (playlist information is stored in the iTunes Music Library file).'

Ok. I REALLY vaporized my iTunes Library. Miraculously, I was able to recover it. As bad as the following sounds, I did have good backups and could have recovered by other means, but because of my peculiar setup I went ahead as though I didn't have the backups.

My peculiar setup

1. I have two iTunes Libraries. The music files for each libary are located in separate folders on a Win2K machine. (Primary home file server.)
2. One iTunes Library is managed by iTunes/PC. Another is managed by iTunes/Mac (primary library).
3. In the past I've created references to the iTunes/PC files in iTunes/Mac by first disabling the "move imported to folder" advanced option then dragging and dropping the folder holding the iTunes/PC files to the Playlist of iTunes/Mac. This works and does not move files or create duplicates of existing entries.

The disastrous Act

In the process of debugging some iBook performance issues I created a new User, ran iTunes and pointed iTunes to my iTunes/Mac SMB mounted library. Oops. The Playlist and library was now empty! I returned to my regular user account and mounted iTunes. Uh-Oh. Playlists were out of date (curiously, not empty) and the tunes count was low. I knew I had about 2000-2100 tunes on the iPod, there were only 1984 in the library. Even this list was not correct, many were "dangling references" that could no longer be resolved. (Error message: not in library)

The diagnosis

I'd overwritten the XML file on the SMB share containing the Library metadata. I suspect some data is also stored in my user Library on the iBook -- that would be incorrect as well.

The initial recovery

I restored the XML files at the root of the SMB iTunes/Mac share from a backup that was about 12 hours old. I could have also tried using the backup XML file iTunes keeps around, but restores from my hard drive backups are very fast and easy.

[As is shown below, this didn't work as well as I'd expected it would. It took sometime to figure out what had happened. Retrospect lists its "snapshots" in order of oldest to most recent, and I took the top snapshot on the list! So my backup was a month old, instead of the copy that was 12 hours old. That explains why I had to go on to more involved repairs as noted below. SAD!!]

The residual problem

Ok, Now I had 1984 tunes listed in iTunes. I'd had over 2000 on the iPod however, so I knew I still had a problem. Furthermore, when I did sync to the iPod I received messages that some tunes could not be found by the Library. So the Library had a reference to them, but the reference was wrong. These showed up with an exclamation point next to the song name in the library; unfortunately one cannot sort a library on these first row "error icons".

iTunes/Mac did list an example of a missing tune, and I was able to see it was still safely in my PC SMB iTunes library folder. I just couldn't get to it.

The resolution?

First I did and drag and drop to refresh my iTune's Mac's copy of my iTune PC maintained library items. That added a few references.

Then, based on the above usenet post and my past experience, I navigated to the SMB share holding my entire iTunes Library. I dropped it into an empty area on my Mac iTunes Playlist menu. The iBook ran through all the files and then initiated a "copy". I was curious what would happen, since it would be "copying" back to a folder where the physical files existed. In fact it did NOT create any duplicates, and, based on the execution speed I was quite sure no copying occurred. Instead it added 53 new Library entries. So I was back up to 2037 tunes, which felt right. Evidently it added entries only for the Tunes it couldn't match on the server. The old "dangling" references were still there (exclamation points in column 0), but so were valid entries for the same tunes. I searched on Date Added to figure out what had been added, then sorted on name and filtered by album. That was I was quickly able to find and delete the danlging entries. I added ratings and other minor data that are not stored in the Library XML files.

But is it all resolved?

Well, not quite! After sync I had 2076 songs on my iPod and 2037 in my Library! Hmm. More detective work needed!

Update: After syncing again the iTunes/iPod gap shrunk to about 7 items. I painfully isolated the "dangling references" (later I found an AppleScript that does that automatically! -- See Doug's AppleScripts). I ended up with 3-4 more tunes on the iPod than my iTunes collection. Someday I'll clean the iPod totally and reload it.

No comments: