Restoring iOS deleted Contacts: the iTunes method and OS X Contacts.app method

Clark Goble has a solid iOS 9 wish list addressing longstanding half-implemented oddities like Contact Groups. It omits one longstanding product gap though.

There’s no iOS backup feature.

Yeah, I hear your scoff, but iCloud Backup is not Backup. It’s a system clone. If you delete Contacts accidentally, you can’t readily restore Contacts of, say, 3 days before. When my sister accidentally deleted most of her contacts she had no way to restore them from iCloud.

If she’d been synchronizing with iTunes she could have used a remarkably complicated hack: Recovering iCloud contacts, calendars, and bookmarks from an iTunes backup of an iOS device. Honestly, Apple, that’s just embarrassing.

In this case I had a Mavericks account for her on my primary machine that’s linked to her iCloud account (even though she’s never used it). I took the machine off the network and launched her account. Most of her Contacts were there. I created a local Contacts archive backup, did some cleanup, and put the network cable back in. I didn’t need to use my archive backup though — when sync was done she had a complete set again (which is weird, actually, but that’s iCloud).

I’ve read rumors of some sort of Time Machine/Time Capsule support in 8.x for true iOS backup, but that would still require a desktop machine. IOS needs more than system clone backup, it needs real backup.

Aperture: How can I tell my videos from my stills?

Is that a blurry still to delete or is it a precious video?

Aperture can store both video in addition to still images, but there’s only one way to tell the difference in the usual Project/Album views.

You need to have Badges enabled in your metadata configuration (see official docs or below):

and in Customize …

Badges are likely enabled by default, but I wasn’t showing them — perhaps because the UI is so wasteful. Unlike other metadata elements the Badge occupies an entire row of the UI, empty except for the Badge.

If you do enable Badges you’ll want to review the symbols; I don’t see any mouseover tooltip. In addition to Video look for Badges that indicate:

• Location assigned
• Referenced image
• Album pick
• Stack (number)
• Keywords applied
• Adjustment applied (1 or >1)

Make Gmail less painful for obsessives: set Promotions to "Read" status.

You know who you are. When you see a number badge you gotta clear it. When you open Gmail and see “5 updates” in Promotions you have to get rid of ‘em.

Yeah, we need drugs.

But I’ve got something even better. A Gmail filter that sets all email of type “Promotions” to a status of Read. Brief version:

Create gmail filter so promotions auto-read. | Gordon's shares:

enter the value “label:^smartlabel_promo” in the “Has the words” field, then mark as read. Ignore warning. No promotions are easy to ignore, delete at leisure.

For the full story see Stack Exchange. The Stack Exchange article is about recreating the Promotions filter after deleting it, but you can use the same filter logic to set status to Read:

1. Create a new filter and enter the value "label:^smartlabel_promo" in the "Has the words" field. When you do a sample filter search, that value becomes "category:promotions" automatically
2. Set action to  “Mark as Read”

When you do this you have to click through a warning message:

Yeah, Google doesn’t want you to do this. When you’re done you get:

You could also send all Promotions to the trash of course. I may yet do that, but for now I ignore them until I get the urge to select all and delete.

Now if only I didn’t have to do shift-click on edit subject then shift click to get Compose Window floating for every ding-dang email reply.

AVI video files in 2014 on the Mac -- you should probably convert them while you can.

In the process of slogging through an unexpected iMovie migration, I came across some old AVI files that iMovie 2013 basically swallows and hides. 

I dug them out of the Package and started searching for AVI conversion tools (bear with me, I’m out of video practice). Google returned pages of spammy looking hits marketing suspiciously “free” third party tools. 

That’s a clue folks. It typically means a market segment has died because Apple has made it part of OS X. Sure enough, FaceTime in Mavericks and later now converts most AVI files (AVI is a container, the real problem is the funky video compression standards inside the container) to .mov.

I experimented with one old AVI file — the original (low res) 40MB camera clip becomes a 64MB quicktime .mov file with H.264 compression and Microsoft ADPCM audio (I wonder if that was unchanged from the AVI file). The 50% growth is typical of migrating from one lossy format to another lossy format — it would be quite bad news if .mov file were the same size as the AVI file. That would indicate too aggressive compression.

I played the converted file back in QuickTime (Mavericks) and the original in QuickTime 7 and they didn’t look too different. Not bad. (As noted below I actually have Quicktime Pro 7. I’m not sure AVI files play in QT7 without the now defunct Perian plugin).

You can’t control the codec or parameters QuickTime Mavericks/Yosemite uses for conversion, for that you can try QuickTime 7 Pro ($30, I suspect part of that is for licensing codecs). Yes, it’s on the App store! First you download the Snow Leopard installer (works on Mavericks/Yosemite) then you pay. I was about to buy it, but then I thought I should check I didn’t already own it. I bought it in 2008 (!). I really should use it a bit more often.

You can also use Handbrake to convert AVI files, or VLC to view them.

I really need to convert those old files; conversion is only going to get harder. However, it’s weirdly good that QT Pro from Snow Leopard still works and is still sold on the Apple store. So we have a bit of time.

See also

General refs on AVI  

• OS X Mavericks: Some files may be converted when opened in QuickTime Player - Apple Support
• QuickTime will not play AVI files after updatin... | Apple Support Communities
• How to play .avi movies in Mavericks | Macworld - Play in VLC, QuickTime 7/Perian, or convert with Handbrake.
• Run QuickTime Player 7 in OS X Yosemite & Mavericks

Older posts of mine on the horrors of video codecs and compression — there’s been limited progress since 2008

• Why I hate video: Format, codecs, DRM and m4v vs mp4 7/2011
• How should I transcode my home analog video? 2/2012. H.264/MPEG-4 Part 10 (AVC) is as close as we have to a standard
• Video: The state of compression is not good. Blame it on Congress 2/2012
• FLIP Video Ultra camcorder: iMovie HD works, iMovie ’08 might 3/2008 (The short-lived pre-smartphone Flip Video camera taught me a lot about codecs. Specifically, a lot of bad things.

I migrate from iMovie whatever to iMovie 10/2013 whatever and of course it hurts

Dilbert, 4/14/94

In retrospect, things started to go wrong for Apple 9 years ago. Back then Apple’s iPhoto, iMovie and iTunes were all running pretty well on Leopard. Then Apple jumped from iMovie HD 6 to “iMovie ’08”, a complete reboot and major regression. Since then we’ve lived through a series of half-baked regression-heavy reboots all too reminiscent of SONY’s Spiderman. There’ve been reboots of FinalCut, iWork/Pages, iPhoto and now Aperture/Photo.app…

Oops. I forgot iTunes. It gets randomly rebooted yearly.

Three years after iMovie ’08 Apple recovered most of the lost ground with iMovie ’11. Which was then followed by iMovie 10, another reboot with feature regressions.

Yeah, Apple went from “’08” to “’11” to “10”, sometimes called “2013”.

I picked up iMovie 10 as an automatic update — I don’t use iMovie all that much (it’s been a discouraging ride) and I didn’t realize the product had had yet another reboot. So when I launched it yesterday to work on my daughters dance performance it took me a while to realize I was in a world of hurt. iMovie 10 has a new file structure and Apple cut a few corners on the migration tools - particularly for Libraries located on external drives.

The conversion and recovery process was too painful to recount here, but here are a few of things that might help if you’re trying to figure this out:

• Read these 3 Apple tech notes: 
• iMovie (2013): Updating projects and events from previous versions of iMovie. Note the classy way iMovie 2013 prevents old data being imported more than once — it writes a zero byte file “UpdatedtoiMovie10”. Big catch — the iMovie 2013 update process works only on iMovie ’11 files stored in the Movies folder or at the top-level of an external drive.
• iMovie (2013): Removing old libraries after updating. See my note below before doing this. Note there’s nothing in the update process to tell you that you have now doubled your movie storage budget.
• iMovie (2013): Work with multiple libraries
• The new file structure resembles Aperture and iPhoto; iMovie now uses Packages. You can right click on your Movie Library to see the internal folders; they resemble the old file structure.

For my conversion I did the “Update Projects and Events” twice. I first did it for the Projects and Events in my iMovie folder. Then I moved my external (large capacity) files to the root of an external drive and repeated it. Once the new Library Package is created you can move it wherever you want.

I then compared what I had in the old Projects and Events folders to what I saw inside the new Packages. When I was convinced those lined up I tested viewing each movie/video. I found a number of problems that I was able to fix with iMovie by changing metadata and creating events.

I also ran into a curious problem where one Project seemed to have pieces in two Libraries. The fix for that included creating a new empty Library and moving events one at a time into it from a damaged Library. There doesn’t seem to be any Library rebuild/repair function.

There’s a native menu option for moving Events between Libraries, so one can theoretically move them from an SSD to an external drive and vice-versa.

Good luck!

Wanted - A tool to chop Pinboard HTML export into Spotlight searchable HTML files

Pinboard provides a nice range of export options including XML, JSON and HTML.

The HTML export uses the ancient Netscape bookmark file standard (I love simple, ancient standards). My 10MB export file starts like this:

There are also XML and JSON versions:

I’d like to follow the example of my nvAlt/Simplenote archive [1] and have every bookmark as a Spotlight searchable file. The HTML file seems the easiest to process — wrap each DT/DD pair with HTML tags and save to file system with the file name taken from the Initial “Description” text field (truncate at nn characters, append ADD_DATE to make unique).

Seems like something somebody could whip out in AppleScript or Python and then sell on the Mac App store for a few dollars. Wouldn’t make a fortune, but I think Maciej Ceglowski would add this to the Pinboard Resources page. So maybe good for a few hundred bucks?

- fn -

[1] Simplenote is all-but-dead today. The beauty of the nvAlt (or Notational Velocity) and Simplenote system is that one can easily transition to Dropbox and something like Notesy, and all notes are available on my Mac and Spotlight indexed with all metadata all the time. That means I can hang with dead Simplenote, just like I hang with dead Aperture. It helps that Brett Terpstra’s (not free) nvAlt replacement is almost here…

 

dd, the Mac OS X (Unix) native solution to hard drive testing, cloning, drive recovery ...

App.net’s gaelicwizard posted this approach to hard drive testing back in Oct 2013. I’d lost it for a while but just found it again today:

$ dd if=/dev/disk1s2 of=/dev/null If it hangs indefinitely, replace the hard drive. If it I/O Error’s, replace the the hard drive. If it completes without problem, the hard drive has no bad blocks.

I’ve not seen this mentioned anywhere else recently, but I found a related MacOS X Hint from 2005 that explains what dd is, and how it can be used for drive recovery and as an alternative to “Norton Ghost”:

The Unix program dd is a disk copying util that you can use at the command line in order to make a disk image. It makes a bit-by-bit copy of the drive it's copying, caring nothing about filesystem type, files, or anything else. It's a great way to workaround the need for Norton Ghost.

Normally, in order to make a disk image, the disk you're copying from has to be able to spin up and talk -- in other words, it's OK to make a copy if the disk is healthy. But what happens when your disk is becoming a doorstop? As long as it continues to spin, even with physical damage on the drive, dd and Mac OS X will get you out of the fire.

We had a situation recently where a friend sent a disk to us that had hard physical errors on it. It would boot in Windows, but then it would hit one of these scratch marks and just die. We fired up dd, and it started OK, but stopped at the same physical error location -- complaining about a Hard Error.

So the workaround was to designate the dd mode as noerror -- which just slides over the hard stops, and to add the mode sync, which fills the image with nulls at that point. We did it on BSD Unix, but as long as you can get the hard drive attached to your Mac, the command is the same:

dd bs=512 if=/dev/rXX# of=/some_dir/foo.dmg conv=noerror,sync

… Once you’ve established the disk image (in this example, foo.dmg), you're almost home. Here's where your Mac OS X box is far and away the best thing to have. In this example, the dd output file is foo.dmg. You have to realize that this is an exact copy of a busted drive, but the "holes" are filled with nulls. As long as the damage isn’t to the boot sector, though, when you double-click on it, Mac OS X mounts it without breathing hard … who cares if it's FAT32, NTFS, whatever...

On a Mac or Unix machine “man dd” will provide the explanations of the parameters in @gaelicwizard’s example:

• if=file Read input from file instead of the standard input.
• of=file  Write output to file instead of the standard output. 

So the command reads every byte of disk1s2 and writes it to “null” (Nowhere). Why disk1s2? It’s a common boot drive. You can use “diskutil list” to find out the name of attached drives. On my machine the result looks like:

/dev/disk0
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *1.0 TB disk0
 1: EFI EFI 209.7 MB disk0s1
 2: Apple_HFS Stanford 999.3 GB disk0s2
 3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *2.0 TB disk1
 1: EFI EFI 209.7 MB disk1s1
 2: Apple_HFS Media 2.0 TB disk1s2
/dev/disk2
 #: TYPE NAME SIZE IDENTIFIER
 0: GUID_partition_scheme *3.0 TB disk2
 1: EFI EFI 209.7 MB disk2s1
 2: Apple_CoreStorage 3.0 TB disk2s2
 3: Apple_Boot Boot OS X 134.2 MB disk2s3
/dev/disk3
 #: TYPE NAME SIZE IDENTIFIER
 0: confidential *102.4 MB disk3
/dev/disk4
 #: TYPE NAME SIZE IDENTIFIER
 0: Apple_HFS BACKUP_B *3.0 TB disk4

I tested dd on disk4 (my backup disk). I found I had to run from my admin account, I needed to use sudo, and I had to unmount the drive first (or I got “resource busy” error message):

Admin$ sudo dd if=/dev/disk4 of=/dev/null

Please be careful what you type — especially for the of parameter. One little typo and you could be writing data anywhere.

It runs silently, you can stop with a Ctrl-C. It then reported:

1451352+0 records in

1451352+0 records out

743092224 bytes transferred in 48.218709 secs (15410869 bytes/sec)

I’ll let this run overnight on one of my backup drives some time, but it looks like an inexpensive way to test a hard drive. I’m surprised it’s not described more frequently. gaelicwizard knows the dark arts.