I dissected the remainders until I got the empty Library to 5.8MB. Here's what remained:
Thumb*.data: 125MB - when you delete images iPhoto does not necessarily clean up the thumb caches. These can be safely deleted.I'm going to take the 45 images I recovered and compare them to the images in my main Library. I'll provide as an update to this post.
iPod Photo Cache: 580MB - this is what goes on my iPod. It wasn't removed when I deleted all the iPhoto images.
2004: 75MB (41 photos)
2005: 2.3MB (1 photo)
I think these are left over folders from when I moved from iPhoto 5 to iPhoto 6. They probably hold images that remained after everything else was relocated. Interestingly iPhoto never produced any error messages about these residual images. I think they were all images I deleted that somehow didn't get removed properly.
Originals: 2 photos only remained in my Originals folder, apparent duplicates two of the images in the 2004 folder.
The final residual: 5.8MB