iTunes sync problem: Internet Audio Stream, the Duplicate tune bug and Playlist Voodoo

Rob Griffiths is the kind of Mac geek who gets tech support from Tim Cook, and even he is struggling to keep iTunes sync working. The list of presumed bugs and magical workarounds for sync failures and Metastatic Other (CloudAssets cache bug), continues to grow, though no fix has been quite as exotic as my restoring from iCloud when restore from iTunes failed or @gaelicwizard’s suggestion to sync from a foreign computer to fully reset sync.

I see all of the problems Rob sees — and more. I really do miss the reliable days of iPod synchronization to iTunes. [1] 

There’s rumor of some sync fixes coming, but what I’d mostly like to see is error logging in Console. At least then we’d have a fighting chance at debugging this kludged mix of Cloud, USB, OS X and iOS.

Today’s problem was failure to sync my daughter’s music to her often troublesome 4s. I tried all the usual incantations including a “restore” (wipe) and restore from backup [2], and as usual nothing worked. iTunes sync swallowed its errors as it always did, and I saw long lists of dashed circle icons next to her phone music list (means sync failed).

Given Rob’s article I looked for dupes and I found an odd set in the music library:

Internet audio stream dupes?! That’s another bug, they aren’t supposed to show in Music — they’re “radio stations”. I deleted them, but it didn’t help.

What worked was kind of odd — I think the Wipe/Restore helped a bit with this, but it wasn’t enough by itself. I walked through iTunes long music screen, removing all the albums, playlists, manually added tunes and so on. Lots of clicks, but I didn’t uncheck music sync. Eventually I got all the music off the iPhone. Then I created Playlists and consolidated them with another “union” playlist that merged every playlist. Then I set that, and only that, playlist to sync. It worked … almost. Of 433 songs in the consolidated playlist, 432 actually show up on the iPhone. Turns out one of them is “Explicit” so was blocked, it shows as “dotted”.

I think this phone has some Flash problems, but maybe the single playlist sync source helped workaround what sounds like some nasty bugs with reconciling duplicates. Or, more likely, it’s all voodoo [2].

Update 11/26/2014

It’s dead Jim.

I thought maybe I’d gotten a bit more life out of this incredibly problematic phone, but my daughter reported that much of the music on the phone wouldn’t actually play. Music.app skipped the piece.

I had another 4s a friend just gave us. I wiped that one, restored by daughter’s last backup to it, and it works perfectly.

This phone is done. I assume the Flash is widely defective.

I’ve set up a service repair by mail — I don’t have time to trek over to the Apple store (not to mention I tried that 6-12 months ago and the phone passed testing). If Apple can’t find out what’s wrong I’ll give it to charity and take the tax break (I couldn’t inflict this thing on anybody else).

See also:

• iPhone with sync error: restore from iCloud fixed when restore from iTunes failed 7/2014
• iOS 7.1 movies (video) sync from iCloud but not iTunes - a bug and a fix 4/2014

[1] Synchronization is Hell. Gotta admit though — Calendars5.app/Google Calendar and Contacts.app/iCloud still mostly work, Toodledo/Todo is very reliable, and Simplenote/AltNV hangs on by its wee finger nails. [2] I wonder if the voodoo depends on whether the phone stalls out on a bad bit of memory. We need those missing Console diagnostics. 

Time Capsule fails again -- and Mavericks has the old OS X sparsebundle delete bug

I’d say that I’m paranoid about backups, but, honestly, I’m just realistic. Our entire computing architecture is designed to destroy data — because it’s designed for performant writes and reads. We aren’t writing on stone tablets [1], we’re writing on sand.

Realism means that for my most important machines I always have two backup systems with as much independence as I can manage. At work I use Retrospect/Win (historic reasons) and HP/Autonomy’s “Connected Backup”. At home I use Carbon Copy Cloner 3.x [2] and Apple’s infamous Time Capsule [3].

But maybe I’m not realistic enough. As of this moment three of my four backup systems have failed.

At work Retrospect/Win stopped launching — probably the victim of a corporate system/antivirus update/bug. “Connected Backup”, when I did a routine restore test, turned out to have made a mess of my backups — old data, missing data, etc.

At home meanwhile Time Capsule failed a routine restore test [4]. I could navigate my backups, but from my primary user account I couldn’t interact with the Time Machine UI. My clicks were ignored. I did some experimentation with different accounts and, as best I can tell, Time Machine wasn’t completely dead. It was just responding extreeeeeeeeeeemlllyyyyyyyy sloowwwwwllyyyyy - and missing my mouse clicks (maybe in the timeline it inhabited my clicks were too fast to be perceived. I used GB ethernet of course.).

I wasted a few hours trying to delete my main machine’s 1.4TB sparse bundle disk image; I tried all the tricks in the book to work around OS X’s inane file deletion limitations [5]. Nothing worked - perhaps because the drive had deeper hardware or permissions or file system problems (Time Capsule is an opaque system, probably running NetBSD.)

I gave up on the fix and wiped the Time Capsule drive, restoring new backups from each computer one at a time (via ethernet for initial backup). 

I’m going to take a look at Synology NAS options. In the meantime I’ll up my Time Capsule ‘single file restore’ checks; I wonder if I’ll have to wipe the Time Capsule every 6 months or so to get more reliable results.

- fn -

[1] At one time in the ancient world writing was done on clay tablets - designed for rapid erasure. It’s a lousy way to preserve data— except that in Nineveh great fire backed the tablets.

[2] Mostly a clone, but it moves updated files into an archival storage. It isn’t a great way to recover data that was corrupted months ago, but since I rotate 4hard drives to 3 locations (work, home, my van) I do have older versions of my data.

[3] Infamous for the world’s least practical user interface (LSD-fueled trip through a star field), awful performance, and cryptic failure modes.

[4] Every month I try restoring a small file; I’ve a Toodledo task for this that I just keep bumping ahead.

[5] Finder and most terminal deletes need to a reference to EVERY file to be deleted, and getconf ARG_MAX shows Mavericks has the same and limit as Mountain Lion: 262,144 arguments. Since a sparse bundle is a collection of 8MB files (“bands”) big bundles hit this limit.

Update 11/24/2014 - Twist

After wiping my Time Capsule drive and recreating my backup, I tried a restore.

It worked well with 2 of the 3 accounts I tried — but, again, not on my home account. The Time Machine window didn’t respond to mouse clicks. As shown in the attached screen photo there’s a pencil icon in bottom left with a slash through it (can’t write).

So I tried repairing permissions. It’s never worked before, and it didn’t work this time.

Next I verified my (SSD) drive — that hung about midway through. No error, it just didn’t complete.

So I booted from the recovery partition and ran Repair Disk — which did complete. Without any kind of error report.

Then I removed all peripherals and tried Time Machine. This time if I switched to a recent backup and waited a couple of minutes it DID respond and I COULD restore a test file for my problem user.

It seems that there’s something about that user’s account (by far largest and most complex) that causes Time Machine to take a VERY long time to be ready to interact with the user.

Update 11/25/2014 - I think it might have been the SSD.

At the same time I was wrestling this problem I was also preparing to apply Samsung’s fix for my increasingly slow Samsung EVO 840 SSD. As noted above I was seeing some odd behavior with Disk Utility — no error was reported, but Disk Utility seemed to hang waiting for a directory structure operation to complete. After I repaired the Disk Time Machine seemed to work.

After I had a working backup I ran the Samsung repair. Following that my startup times went from 20 seconds (faster than hard drive) to 2-3 seconds (SSD speed). Time Machine was also quite a bit faster.

My guess is that read failures on the directory structure for my problematic user account was causing Time Machine to hang. If so the problem wasn’t with the Time Machine backup, it was with the drive I wanted to restore too…

Samsung SSD 840 EVO has a slow read bug: information on the fix

Over a year ago I switched my 2009 iMac from yet another dead hard drive to a Samsung EVO 840 1TB SSD. It was a calculated gamble that has worked fairly well so far, but SSD technology has proved less mature that I’d thought (Maybe this is why Apple has been slow to transition). For example, the net is full of contradictory recommendations on TRIM use. For another, the Anandtech coverage of this complex bug ends with “None of the big SSD manufacturers have been able to avoid widespread bugs”

Recently Samsung admitted they had a problem with slow reads — something I’ve been wondering about. They’ve put out  Windows fix and a bootable DOS version that’s supposed to work on a Mac.

I haven’t tried it yet, but I’ve assembled my references…

• Samsung SSD Downloads | Samsung SSD: Windows/DOS “Performance Restoration” / firmware update
• Problem description: “when reading an old file (e.g. 6 months old), it may give you very low reading performance (e.g. 20MB/s), this tool is to upgrade the firmware to fix this issue and restore the current data's reading performance.
• AnandTech | Samsung Releases Firmware Update to Fix the SSD 840 EVO Read Performance Bug
• Using the Samsung 840 EVO SSD Performance Restoration Software on a Mac - I love this article. I noticed Samsung’s “bootable USB” version wasn’t actually a bootable image — not too reassuring there Samsung. FileVault must be disabled.

Thanks to Conrad’s Chavez’s generous blog post I feel ready to do this - once again I appreciate the CD drive in my old iMac. After BOTH of my backup systems are completely current of course. Actually, maybe I’ll do a 3rd back up just in case.

Update 11/25/14

Wow. I did the fix and the speed improvement of my 1.5 yo SSD is amazing. I’d noticed some operations were slow, but the gradual decline hid how bad things were. Startup time is maybe 2-10% of what it was.

I think extremely slow reads on something related to directory structure were causing my Time Machine/Time Capsule restore failures (more on that in the post, later).

A few notes/extensions of Conrad’s fine post:

• I did 2 backups beforehand and tested a file restore from one. Then I did an additional Aperture specific backup to another drive.
• I recommend booting into and running Disk Utility disk repair prior to the fix — just as a general safety measure.
• Unplug all peripherals — keep it simple.
• I didn’t have to rename the .ISO file to .DMG as Conrad mentioned. Disk Utility burned CD normally.
• On my Bluetooth keyboard I hold down the ‘c’ key immediately after startup tone to boot off CD
• Firmware fix is very quick — but the full repair took about 4.5h. The UI writes a dot every 15min or so (see below).
• After the fix was done I got the old (I remember DOS) prompt. I pressed restart and got a heart attack as OS X was stuck in an exotic failed boot cycle. I had to press and hold power for a few seconds to force a proper shutdown. After that all was fine.

iOS 8, AT&T MVNO and the endless searching Personal Hotspot bug: toggle Bluetooth

We’re on 8.1.1 and I’ve discovered our first significant bug - the Personal Hotspot icon is stuck in endless “searching” spin on two of the 3 children’s H2O Wireless (AT&T MVNO) iPhone 4s and 4.

It’s a new problem, likely related to the new Instant Hotspot features of 8.1 (even though it’s not supported on these older phones), but there was a variant of this in older iOS versions. The phones use the H2O “Locus” configuration Profile though, so I can’t follow old advice to edit APN settings.

I think there’s also a contribution to the bug from restrictions on making changes to cellular data (restricted on the children’s phones).

There’s a workaround though, one that I came up after realizing one of the 3 phones didn’t have the stuck Hotspot search. That phone had Bluetooth disabled, and Bluetooth is needed for Instant Hotspot functionality.

So I disabled Bluetooth on the #3’s phone and the endless search stopped, instead I saw “Set up Personal Hotspot”. I reenabled Bluetooth and the bug didn’t return. On #2’s phone Bluetooth was off, and enabling it stopped the search.

So it seems all I have to do is toggle Bluetooth. 

There is still weird stuff here. All 3 devices have the same Profile and H2O plan, but one 4s has the “Personal Hotspot” functionality and the other two say bring up a “Call AT&T to enable” dialog. I’m still playing around to see if I can fix that.

Subscription to Facebook Events calendar now possible, including Google Calendar - for the moment

I just added a subscription to my Facebook calendar to my Google Calendar.

Is this new?

I can't say, but knowing how it works I found documentation updated in Sept 2014. So it might be new. I briefly scanned Facebook's marketing infested 'newsroom' and didn't see any notification there.

Parenthetically, this kind of discovery is a real Facebook issue. I've looked for this feature several times over the past few years. The last time I looked I found several old hacks that were either obsolete or insanely high risk.

Why is Facebook such a mystery? Partly it's lack of documentation and feature churn, but, more importantly, geeks ignore Facebook. So my usual notification systems fail.

Anyway, this is something I've really wanted. Look for the cryptic three dot icon next to a Facebook event:

Choose "Export Event". You can email a .ICS for the event, but, more importantly, you'll see webcal URL with an embedded unique identifier (token).

Copy that URL and paste it into Google Calendar "Add by URL".

Suddenly Facebook Calendar is useful.

Force Chrome to remember iCloud password - how I ended up with LastPass

I don’t use iCloud for much - Apple’s cloud functionality is almost as lacking as its reliability. Our family has used Google Calendar for 7 years [1], I use Simplenote and nvAlt [3] as an information store, Gmail for email [4] and Toodledo/Todo.app for tasks. [5]

That leaves Contacts in Apple’s badlands — they’re too tightly coupled to iOS and OS X to readily migrate. So Contacts are the only bit of iCloud I use; I have to admit that they have been relatively reliable.

Hang on — this does get to the part about forcing Chrome to remember an iCloud password. But first I need a bit of setup. I have to explain something about corporate life and personal data.

Fifteen years ago many employees mixed their personal and corporate data on business laptops and workstations. It wasn’t unusual to use a single email for both work and personal use. Ever since then the two worlds have been dividing - driven by legal and security concerns. Even thumb drives are encrypted on insertion now; data on the increasingly locked down corporate laptop belongs to the corporation.

Which is fine for email and work documents - they should belong to my employer. Contacts though — they’re a problem. They don’t divide neatly between work and personal — and my work Contacts are pretty important for my future employment and family food. So, when it came time to decide where my Contacts should live, I moved them entirely into the personal sphere.

Which is why I need to use a corporate browser (Chrome [6]) to access iCloud — that’s where all of my Contacts live. I need ‘em when I work. 

Ok, so we’ve established I need to use Chrome to access iCloud. Now the problem — it makes me enter my password way too often. And my passwords aren’t easy to type or remember. There are extensions that once forced Chrome to store this password, but they don’t work any more.

So today I broke down. LastPass has a freemium model for online credential storage; the web app and Chrome extension are free. (LastPass charges for mobile services.) Unlike 1Password, which I use on iOS and OS X, there’s no need to buy a Windows client — and I don’t want to put all my credentials in the Cloud anyway. So I signed up for free LastPass, and created an account with a single stored credential - my iCloud ID and password.

It works fine. So one of my longstanding annoyances has been fixed — I can quickly bring up iCloud Contacts.

- fn - 

[1] Calendars 5.app is essential for the Google Calendar power user — we have it on every phone [2]. My native iOS Calendar syncs to my corporate calendar, Calendars 5 reads the iOS calendar database so it appears inline with my other 17 calendars, including 1 for each family member and one ‘all family’ calendar. Our family grandfathered into free Google Apps accounts, but if we didn’t I’d probably pay for the business service. Free has been nice though.

[2] It’s $7 a user. Since it’s not funded by in app purchases I believe iOS family sharing would allow one purchase to support five users. If it did use in app purchases that wouldn’t work. FWIW we still share a single Apple Store only AppleID, I use a different AppleID with iCloud.

[3] nvAlt is in maintenance mode for now — but so is Notational Velocity. Brett has a commercial replacement in the queue, as of today “it’s amazing and will probably be released”.

[4] Google’s broke Gmail usability with their last UI redo. I use Mail.app on iOS and sometimes Airmail on Mac, but mostly I gnash my teeth and weep and use Gmail.

[5] Nobody would do this by choice. Is a legacy choice.  Works and I hate to change things that work.

[6] The corporate standard is IE 9 — thanks to legacy apps. So IE for corporate apps, Chrome elsewhere.

Taking advantage of AT&T's promotion: 2GB to 3GB for same price (exp. 11/15/2014?)

AT&T is offering 3GB of MobileShare data for the $40 price of 2GB - part of a price battle with Verizon. It’s the default for new customers, but existing customers can get it by using the web site as though changing data plan. This is what I saw on that screen:

If you do this be careful not to miss the Submit button on the final screen, it’s lost below the formal fine print agreement.

Based on my reading and what a rep told me, there’s no known catch and no contract. Since Emily and I routinely use 1.9 GB I considered going to 1GB and paying the overage fee, but there’s no new discount at the 1GB level.

AT&T used to charge us $15 for an extra GB when we occasionally exceeded our 2GB limit, so this promotional plan should save us $45 a year — and we don’t have to be quite so obsessive about our iPhone data use.

I’m not sure how long this offer will be in place, the fine print says you have to make changes before November 15, 2014:

We don’t have any contract with our MobileShare (all our iPhones are unlocked), so I’m less worried now about AT&T treating me badly. I can leave at any time.

The change was applied immediately. Instead of 1.8 GB left this billing cycle we have 2.8GB.