iTunes Mac to iPad photo sync is broken

I had a longish holiday break this year, so naturally I spent part of it fighting with another technology regression. 

This regression was about getting photos from my Mac to my iPad; I use the iPad as a digital photo frame randomly displaying my Aperture (running on Sierra) images tagged as “slideshow” (this drains the battery fairly quickly, so I usually have the iPad charging). Years ago this was painless, but over the last few months I’ve found that only a fraction of my tagged photos make it over to the iPad. Often the process seems to die.

This go round I tried an old Aperture tip to delete the iPod Photo Cache (Apple approved) [1]. That had one effect — image transfer died completely! Instead of telling iTunes to transfer all my Slideshow tagged images (9,019) I had to select year-specific smart album slideshows one at a time (incremental add):

That generally worked, though the ghostly counts shown above in the iTunes 12.7 UI came and went inexplicably. Sometimes a mouseover seemed to trigger an appearance. If they didn’t show up that was a clue that something had gone wrong with the sync. (Infamously iTunes does not log errors to Console. Perhaps because they’d overwhelm console?)

Some part of the OS (or Aperture?) has to regenerate the .ithmb files [2] that are shared, that seems to be a slow process. I think the build process might make use of some data Aperture saves on exiting; that data is also used by the old OS X media browser (itself related to Apple’s old iLife code) [3]. I’m pretty sure some of the .ithmb creation is by an OS thread thought, and that seems to have a low priority. Maybe part of what causes a complete failure with large numbers of files is that iTunes has an arbitrary timeout placed to work around some other iTunes bug [4].

Once I broke down my slideshow into smaller bits and added them incrementally I seemed to have some success. The count (erratically) displayed in iTunes matched what showed on the iPad album counts and the Aperture smart album counts. Then things went south. The numbers transferred started to be a fraction of the Aperture counts. When things settled out I seemed to be stuck at 7,811 on the iPad (same count in iTunes if I mouseover the iTunes content type distribution bar) out of 9,109 in Aperture [5].

I did some half-hearted testing to try to figure out if this was related to filename characters or the like, but I haven’t gotten very far. I did confirm that RAW files don’t have an obvious problem.

The great news is that I learned today that I’m not alone and it’s not an Aperture specific problem! There’s a Macintouch thread on this by riley (quotes others, emphases mine):

So here's an update on the issue with syncing large Photos databases between iOS devices and iTunes.

After posting a number of followups to my Apple bug report, the response I got back from Engineering was that this is a known limitation in iOS, and "we are working to improve this in a future iOS release."

Their suggestion was to sync by starting with just a few albums, and then repeatedly re-sync, adding a few more albums each time. "On subsequent syncs, iOS will not add additional photos unless the sync preferences in iTunes are different from the previous sync, which is why it is necessary to keep adding selected albums each sync, instead of all at once."

… There is a huge discussion of this over on the Apple forums (see “Photos app on iPhone not syncing in iOS 11 ").

Various people have been told various things. Some have been told it is an iOS issue, some an iTunes issue. Personally, I think there are issues in both that are interacting, which are preventing reliable syncs with large photo albums over USB between iTunes and iOS devices.

Two years ago, our then ~40k photo library in Photos would sync the selected items (maybe around 30k photos in various albums) without problem. Perhaps around the time of iOS 10, the syncs needed to be restarted to complete, but eventually it would work. Since iOS 11, some sync, but many do not, and the sync just cancels with a variety of error messages at various points. Sometimes it is a device error, sometimes nothing. Sometimes many of the photos will be on there, but the albums they should be associated with are empty…

… The last update I heard from the person who was helping me was that it was an iTunes issue (at minimum, perhaps iOS too). She said that the Apple engineering team is aware of the issue and working on finding the problem so it can be fixed.

… I also see that since the latest tvOS update and the latest iOS 11 update, many albums are no longer being shared with the Apple TVs.

Hmm. This suggests to force updates one should add or remove some album with every sync. That did not work quite as expected. After I tried using the Media Finder code [3] and the ‘add new album’ trick iTunes said it was synchronizing another 1000+ files (that’s good!) but then the counts in iTunes and iPad dropped from 7,811 to 7303. So that’s just great.[6]

Since I now know this is not just an (abandoned) Aperture problem I added an iTunes feedback comment about the problem. Rumor is Apple processes these, presumably doing some algorithmic text extraction based on volume. So add your voice if you’re interested.

- fn -

[1] It took hours to empty the trash after deleting iPod Photo Cache — despite files being on an SSD. As of 7/2020 the links are all broken, but this is what you do: "find your Aperture or iPhoto library in the Finder. Right-click and select “Show Package Contents” then find “iPod Photo Cache” and delete it."
[2] I changed extension of a copy to .jpg and it rendered as expected. They are fairly high res images, in a small sample I examined one was a 3MB jpg. 
[3] In Sierra’s Open dialog the Media Finder still shows in the left nav pane! Maybe invoking it might trigger some updates? Worth a try to open it and let it run for a while.
[4] It’s widely assumed that iTunes source code is an impenetrable mess with bits left over from pre-Apple days. 
[5] Incidentally “Stacks” are a problem. One of the many bad things Apple did in bridging iPhoto and Aperture was to treat an old iPhoto Edit as an Aperture Stack. This threw away key information — that one image was the new version of another. There are lots of downstream implications. Years later I’m recreating thousands of iPhoto edits in Aperture. One of the problem is that when you tag an image as “slideshow” then transfer to iPad both versions go. I think this can cause the iPad count to be higher than the Aperture count, but of course the problem I haves is in the other direction.
[6] I removed the extra album and then the count went UP to 7,811 again! That album did reference photos that were in other albums, so maybe part of the bug has to do with photos that are in more than one album. I do think there are multiple bugs here though.

Update 12/31/2017 I

There are some JPEG images in my older files that do not show up in either OS X Media Browser (old iLife code) or Sierra’s Screen Saver Library browser and do not transfer to the iPad via iTunes. However, if I export the original and drop it back in again then the image appears normally — this doesn’t appear to be problem with the JPEG. They also render normally in several different apps and EXIF viewers.

Rebuilding the Aperture database does not make a difference.

So this is curious, and potentially an important clue, but to go further I need to understand how Media Browser works.  In my case the bug doesn’t only impact image transfer to my iPhone; it impacts Media Browser as well. I’m now poking around at ~/Library/Application Support/iLifeAssetManagement and ~/Library/Application Support/iLifeMediaBrowser, especially ~/Library/Application Support/iLifeAssetManagement/state/albumshare/Daemon.sqlite.

Update 12/31/2017 II

Finding more “invisibles” but no clear pattern to what’s omitted. Some are quite small - from the days when 100K was a good image size. It’s not size alone though; of a pair of similar old images a 64kb one made it across and a 70k one didn’t.

Update 1/1/2018

Same sync, same everything — but now iPad acts as though it’s been syncing to a different computer. All images removed! It took several tries to get one album to sync, but after it started working I could do about  @1,200 at a time. Now have about 7,700 on iPad out of 9,100 on Mac. I won’t sync iPad to iTunes again until there’s word of a bug fix. Just stick with what’s on there now.

Update 6/9/2018

I thought this was fixed — but it’s broken again. On latest versions of iTunes and iOS 11.3.1, macOS Sierra.

Update 6/9/2018b

I was able to sync 7777 of 7771 (different apps count photos differently, not least due to Aperture’s brain-dead “Stacks” handling of iPhoto imports) photos after installing iOS 4. I don’t know if the bug is fixed or if it’s just that process of installing an iOS update cleared something out. In the latter case it might return.

Update 7/4/2020

It's been a year or two since I last tried syncing. At this time I'm still on Aperture and Mojave. The sync was worse than ever. I wiped my old iPad and restored from iCloud backup and sync actually seemed to work! It saw only 7992 of the 9413 images in my Aperture slide show, but it did import those.

I tried the trick of deleting the iPad Cache in Aperture [1] but on resync it was still 7992/9413. Consistent at least. 

I think there's a bug in Aperture with Sierra and later with generating the images in the iPod Cache. Maybe something iTunes or OS X was supposed to do.

 I suspect the iPad flash memory is also in bad shape. 

The lack of logging or error handling is appalling.

How to maybe stop Sierra from preferentially auto-connecting to XFINITY WiFi.

My Mac insists on connecting to my neighbors Comcast/XFINITY WiFi rather than my home wifi. The usual recommendations didn’t work. (Note, if you don’t run as admin, and want to play with WiFi settings, you need to first click the unlock icon on the Network Preference Pane before you click “Advanced”. Bad UI.)

Despite deleting XFINITY from preferred networks I could still immediately join my neighbors network. Sierra didn’t seem to honor the order of WiFi networks I’d set. I searched Keychain but couldn’t find Comcast or XFINITY left anywhere.

The new trick to look to see if you have a Profiles PreferencePane. There, if you’re like me, you’ll find a “User Profile” called XFINITY WiFi. (I dimly recall installing per XFINITY directions, but now they advise us to use an app — I think the app may allow one to prioritize a local WiFi).

My XFINITY WiFi Profile description said “Automatically connect to XFINITY WiFi hotspots”. It had an Enterprise Profile ID and I suspect MacOS is prioritizing it. So I deleted it.

And … I still connect to the XFINITY WiFI without specifying a password. It appears to need no credentials (or the Profile is still lurking around). When I toggle WiFi on/off I don’t auto-connect to XFINITY though, so maybe removing the cert deprioritized it.

I think this is more a MacOS bug that a Comcast bug.

Can't remove last "App" from the App Bar in iOS 11.2.1 iMessage.app? Turn screen to landscape mode.

On my iPhone 8 with iOS 11.2.1 I was unable to toggle off the last App (YouTube by alpha sort) in my plug-in list (what I mean). I turned phone to landscape mode and my touch was recognized. It’s not a screen touch bug because after doing this I could turn back to portrait mode and toggle it on/off at will. I suspect an array count or layout bug.

My Instagram account was hacked -- the follow-up is interesting

I signed up for Instagram before Facebook bought them. Back then it was a curiosity of low value and I used my throwaway password. That’s the password I’ve used for over 20 years for things I don’t care about, usually things that don’t even make it into my 1,867 item password database (ok, so only 488 are likely useful).  It’s an 8 character alphanumeric — not bad by the standards of 1997 but obviously insecure now. It’s also certainly been added to many dictionaries as various services have been hacked.

After Facebook acquired instagram I think I authenticated through my Facebook account. I forgot about the old password.

Today when I launched Instagram.app I was notified of a login from Rio de Janeiro. When I answered that was not me I was sent to a password change screen. Evidently, like Google, Facebook/Instagram considers a valid password only a modest marker of identity (it might help that I never use the Instagram password and, in particular, I never login with the email address associated with that account, only with a username).

So no harm there — but it means someone is testing the throwaway password together with my gmail address against a range of accounts. I checked my database and there may be a few low value accounts I should clean up. Very few though …

Overcast.app: the upload feature in premium is not very useful

I like to listen to medical and historical podcasts on my commutes. I have years of material in iTunes and limited listening time, I don’t need new subscriptions. I just need to queue up selected podcasts on my device. I prefer not to waste cellular data and I’m not interested in listening from newer to older (in fact many medical podcasts need to be sequenced the other way).

iOS Podcasts.app worked for this, back when I could choose a Playlist in iTunes and sync it. Today’s iOS 11 Podcasts.app is pretty much worthless.

So today I tried Overcast. I can select the podcasts I wanted in iTunes and drag them to the desktop. I subscribed to Overcast.app premium so I could upload them for download into Overcast (no iTunes support).

That’s when I found uploads are one … at … a …. time. Not to mention slow, but really I could live with slow. Overcast.app is even less useful than Podcasts.app

I wasted $10. I’ve canceled subscription renewal.

Simplenote for Mac search has been fixed

Two years after Simplenote.app for Mac was released with broken search it is now fixed in version 1.2. A product I rely on is not only alive, it’s even improved.

This was a significant rewrite and there is still lots of basic work to do. Tags don’t autocomplete (I think they used to in the prior release and they do on iOS). You still can’t apply tag changes to a group of notes. There’s no export of notes as text files from App. There’s no markdown support (not a priority for me but iOS version has it) and of course no support for images. 

I haven’t used it long enough to spot any sync errors — recent versions of Simplenote were very reliable.

I hope this is a sign that Simplenote has a future. Now if they would only figure out a way for me to pay for it…

Sending multiple Contacts to an iOS user from iCloud Contacts

Found this works

• Select multiple in iCloud
• Click gear bottom left to export VCF
• Email VCF to iOS device
• Click on attachment - get option to Add all members

I’m sure there are other means. Note this does seem to require a computer. I imagine works same from Mail.macos.

iTunes cannot sync voice memos ...

“iTunes cannot sync voice memos … because the Voce Memos app is not installed … You must download the Voice Memos app …”

Got this error message in iTunes today. I haven’t used voice memos.app in a long time (I use iTalk.app) and didn’t notice Voice Memos.app was no longer on my iPhone.

So why am I only getting this now … and why did a search on this phrase only find some phishing/spam page? Why does searching on Voice Memo Apple find only a bunch of third party apps? Why isn’t Voice Memos.app on my iPhone?

1. I don’t know why only now … maybe an iTunes update?
2. I don’t know why Voice Memos.app didn’t get installed when I updated to iOS 11 — might be a bug there.
3. It is on the app store, but Apple’s app search/discovery is an unholy mess. You have to search for “Voice Memos” exactly. Shame we can’t search iOS App Store from the #$#@% Mac any more

One good thing — I ran across Apple’s Music Memos.app — which I’d missed. It does sound useful.

Canon EOS Rebel SL2: switching the paired iOS (or Android?) device (WiFi, bluetooth)

Older Canons cameras had a lovely IR remote. Worked great.

Now they use a Bluetooth remote — or you can use a complex remote app on an iPhone called Canon Connect.app.

It’s awful.

If you do ever get your camera paired to your iPhone you’ll run into trouble when you switch phones. How do delete the old pairing?

I think you don’t - at least, not directly.

Go to Wireless Communication Settings. Then Bluetooth Function (not Wi-Fi).  Their you check/clear connection info and then do Pairing again. The Wi-Fi info will then get set when you use the Cannon Connect.app to connect to the Camera’s built-in hotspot.

The trick is you start with Bluetooth, the Wi-Fi settings are stored thereafter.

Facebook's disappearing Pages feed

I remember when you could get an RSS feed for a Facebook wall. Years after that when away there was an RSS feed for Page posts, but I think that’s gone now.

Up until a year ago there was some obscure way to see the Pages Feed (all Pages) in Facebook Mobile, but I think that’s gone too. 

AFAIK the only way to view a page feed on mobile is to save the URL to the Home Screen. Go to https://www.facebook.com/pages then click on tab feed then save. Safari will sync un/pw via keychain across Safari on devices.

Looks like Facebook only wants Pages to show up in the news feed, and then only for a price.

Spotlight bugs in iOS 11.2: Empty contact on search.

I think the spotlight/contact bug was introduced in 11.1. It’s not fixed in 11.2.

If I search in spotlight for a Contact name, or search on Contacts field of phone app, I get an empty (Null) Contact as a result. It has no content.

Turning Search (spotlight) off then on for Contacts restores search, but only transiently. I have 2,162 Contact Cards, I did find one that was empty and deleted it.

I’ve now removed all Contacts from my phone (iCloud) and restored them. I’ll see if that works.

Google's phishing vigilance and the risk of blog comments

I got a notice from Google yesterday. tech.kateva.org had been quarantined as a phishing vector. Visitors would be warned away. I had a short time to fix the affected page that was listed below …

But there was nothing there. Same thing with a link to a downloadable spreadsheet of issues. That was empty too.

Elsewhere in the message a page was identified. It sure looked benign, the only link was to an Apple support site. It’s a (Google) blogger site and all the widgets were default Blogger. No extra HTML. No comments.

Also, despite Google’s warning, the blog was not quarantined.

A puzzle.

Oh, I left something out. The day before a published spam comment showed up in my comment tracking feed. I get a few spam comments every day or two, but this was the first published one in a while. I went to delete it … but by the time I got to Blogger’s post management menu the comment was gone.

This is what I think happened:

• This blog was set to allow comments without approval from authenticated posters for posts less than two weeks old.
• A bot created an authenticated identity and created a phishing attack comment.
• Google spotted the comment, quarantined the site, and sent the notification email.
• Blogger spam detection identified the comment author as a spammer and deleted all comments by that identity — including the one on my site (why it was gone).
• Google rechecked my site and lifted the quarantine — but couldn’t retrieve the notification email.
• The notification email was partly empty because that it was a query — that returned Null. It had one part that was written at time of email generation and that contained the link to the once contaminated page.

I changed comments on the blog to require authorization at all times — no two week window for authenticated users. Clearly authentication is no longer a sufficient barrier. I don’t want phishing attacks on my blog, and I don’t want to get quarantined.

This reminds me what a strange fish Blogger is. It works fairly well, though there’s a longstanding problem with CR/LF handling that reminds me too much of DOS 2.1. It gets very few, but still some, updates. Google has switched their blogs off Blogger, but they haven’t used their new proprietary RSS/Blog platform to replace Blogger. Blogger is neither dead nor alive, and Google RSS is similarly quantum.

Apple's Apple ID fiasco is getting worse -- declining support for Apple Store ID that is not iCloud ID

Like many veteran geeks I have a different iCloud ID and Apple store ID. This used to be a supported configuration. In my case it was essential because of some complicated history with Apple’s .mac precursor to MobileMe and iCloud. (In an unrelated matter I have another 3-4 Apple IDs that aren’t connected to anything but, depending on the vagaries of Apple’s hacked together legacy databases, sometimes pickup Apple Store hardware purchases.)

This is what Apple’s support document says now (emphases mine) …

Sign in with your Apple ID - Apple Support

… We recommend that you use the same Apple ID for all Apple services on your device—including the iTunes & App Stores and iCloud …

… If you have multiple Apple IDs, you can’t merge them …

I went looking for this document because I think iBooks.app doesn’t work properly with an iCloud ID that’s different from the Apple Store ID that can be used to purchase iBooks. It looks like this will be a trend.

Note what Apple says here. Your Apple Store ID and iCloud ID should be the same. You also can’t merge them [1]. So you either need to abandon all your Apple Store purchases or your iCloud storage purchase.

Anyone remember when Cook promised to fix Apple’s original sin of  botched identity management system? Apparently the problem is harder than building spaceship headquarters.

Apple should bite the bullet and come up with a process to merge Apple IDs. I fear they aren’t going to bother though. I really miss class action lawsuits.

- fn -

[1] There is a possible workaround. You may be able to use your iCloud ID as an Apple Store ID and then make it a family member of the original Apple Store ID. This will run into rules about changing device Store IDs and constraints on family member size as well as issues with the total number of devices that are part of a family (10). It isn’t an official workaround and I suspect it has irreversible problems of its own.

Podcasts.app - sync Stations between iTunes and iPhone via iCloud, not via iTunes sync

Apple removed playlists from iOS 11 podcasts.app. I loved playlists. They were a beautiful user friendly query building tool.

Instead we have Stations, which are more limited. I created some in iTunes and some on my iPhone. They didn’t sync even though I transfer podcasts from iTunes to iPhone via WiFi/Lightning cable.

It turns out Stations do sync — but only via iCloud even though files may travel via local sync.

Once I enabled podcast sync in both iTunes and iOS my stations appeared in both. 

This is good, because one advantage of stations is that podcasts play consecutively. At least I think that’s why mine started playing consecutively instead of stopping after each podcast as they did earlier in iOS 11.

There are lots of weirdnesses in iTunes when it tries to combine sync via iCloud/Store with local file based sync from iTunes. It will be interesting to see if Podcasts does better at this than Movies does.

Apple is clearly heading towards the world where we have all have 2TB iCloud Drives, a 256GB local machine store for frequently used data, and special folders in iCloud for personally owned files that sync separately from Store files (ex: personal images, ringtones, podcasts, PDF, etc). There’s no resisting this one.

Update: It is a mess. Of course. Stations need a setting to only show downloaded episodes. One problem with Stations is that while they can filter out Played stations, they only know a station is Played as long as the file is around. If the file is removed and the show is in the Cloud then it is treated as unPlayed.

Progress of sorts - integrating my simplenote memory extensions with gDrive

Nine years ago I wrote What my blogs are for: memory management and the Google-Gordon geek-mind fusion. It’s hard to believe now, but in those days we did not yet fear Google. Back then, we liked Google. Google was good.

That was a long time ago. Google has since transcended mere human morality (Galactus is a much better name than Alphabet btw). Unfortunately I haven’t transcended my need for artificial extensions to my working memory. To the contrary, that need has grown. Now in addition to my old technical expertise a change in employment means I manage more organizational and political knowledge. Oh, yeah, and I’m back to being a practicing doctor. So now I need to know adult male ambulatory internal medicine. Also, I am old and my brain is crud.

So, yeah, I need those memory extensions. I still use Google’s tools, though the Custom Search Engine I rely on to integrate my blogs with my link share stream and my ancient (hidden) web pages is all but forgotten. Since 2008 though I’ve moved away from the Google dominated web back to files and file systems as a memory store. Carrying a connected computer in my pocket made 1980s style files useful again. I rely in particular on plain text Simplenote to carry around fragments of memory. For regulatory/security reasons I have one Simplenote account for work memory and one for non-work memory — with some notes shared between both using Simplenote’s (simple!) sharing mechanism.

Spotlight indexes simplenote files on my iPhone — sadly only for one account. On my Mac nvAlt syncs to my personal simplenote account — so the text files are indexed by Spotlight there too. Today, guided by a @simplenoteapp tweet I installed Simplenote Electron (which Automattic is abandoning for a future fully native Mac upgrade [1]) and tied that one to my work account. Then I used the improved text export to create a local read-only copy of that memory store — which Spotlight Mac now indexes. So the stores are in one search space at last.

The last twist is that Simplenote local text stores are in a Google Drive folder, so synchronized to gDrive with appropriately stored folders such that appropriate text stores are searchable on both devices. Sadly gDrive does not support “Available offline” at the folder level — so when I don’t have data access I also lose some memory access. (I’m hoping Google will fix this.)

The gDrive integration is important because there are limits to plain text - particularly for medical topics. I have PDFs, Word Docs, Google Sheets and other detritus of decades of notes. I can put all of that in gDrive alongside the text notes and let search figure it out. (This is what Evernote is supposed to be good for but they don’t have a clear exit strategy. It’s trivial to walk away from Simplenote. I need that exit strategy [2].)

It’s all one heck of a kludge, but so is my memory. So kind of fits.

See also:

• What my blogs are for: memory management and the Google-Gordon geek-mind fusion 7/2008
• Snippets Explained (Version 2) 2/1999
• Beyond Simplenote: I still want a graph layer (concept map) overlay for my memory augmentation notes collection 5/2016. Sigh. This would be wonderful.
• Project Memfail: Tackling my search space problem 9/2013. My work space problems got sorted out when the company died. I’m much more cautious now about creating new memory pockets.
• Thinking tools 2014 - holding steady but future unclear 5/2014. RIP App.net.
• Full text search and digital prostheses: new email, new mind 7/2008
• Strange loops: Google custom and customized search - and a memory blog 3/2008. Non-evil Google.

- fn -

[1] It has the same search bug as Simplenote/Mac!

[2] For example - if they don’t fix that damned search bug [1].

 

Selling or retiring an iPhone -- I hope you disconnected Google Authenticator and Google Prompt first.

Wipe that iPhone to give to your child or sell? I’m sure you remembered to launch Google Authenticator and remove the device from your trusted device list before you erased all …

Using a new phone to receive 2-Step Verification codes - Google Account Help

… On your old phone, open the Google Authenticator application…

Oh, you forgot about this step? You are clearly inadequate.

There’s hope though. Assuming you have a computer, there’s another option hidden away …

Add or remove trusted computers - Google Account Help

… Under “Devices you trust,” select Revoke all...

Except that’s not a bit misleading. There are more options once you sign into you Google Account and dig through the “Sign-in & security” section, select Signing in to Google, and Choose 2-step verification.

There you can remove the “Google Prompt” iPhone that manages authentication via Google App. That flips authentication to an Authenticator app (OTP authentication with RFC 6238 and 4226). The Authenticator app might be Google’s, a 3rd party, or 1Password or another password manager. I use Google’s because I started with it and I’m lazy.

You can also “change phone” on Google Authenticator. Authenticator is working for my new device though — which was restored from a 6s backup. So I didn’t do that immediately. 

Instead I removed the Google Prompt device, since that was still going to my (since erased) 6s. Then I added it back to my new iPhone 8. Google had the 8’s name so I authenticated there. 

Then, because I’m a paranoid sort, once I had Google Prompt working, I went into Authenticator on my i8 and removed my Google account then setup authenticator again from the 2-step verification page (scan barcode).

I’m sure everyone knows to do this. Otherwise why would pundits keep telling us to enable 2FA on every service we care about?

PS. I think when you authenticate within Safari for iOS Google can’t identify the host device. So my https://myaccount.google.com/device-activity list shows both “John8” (my current iPhone) and “Apple iPhone” — even though I believe they are the same thing.

PPS. I think if you want to be very careful you really should do the “Revoke all” as I suspect the old device Safari authentication can still be an issue (except I erased it, but if you’re paranoid …)

See also

• Things to do when selling or retiring an old iPhone 9/2015. Except its really worse than this … 

The Internet Lied: Apple's 3.5 mm to lightning adapter does support audio in -- for Apple's EarPods.

I was deceived.

Prior to buying my iPhone 8 I read that Apple’s 3.5mm to lightning adapter didn’t support audio in for earphones with a microphone. Not so! It works quite well with Apple’s EarPods.

It probably doesn’t work with other earphone microphones though. Apple’s EarPods have a different arrangement for the audio-in connection from some other earphones. I looked into this years ago and it wasn’t clear to me how much of a standard there really was. I dimly recall that Apple, shockingly, was different from most.

Did iOS 11.1 (partly) fix iTunes photo sync?

I’m back to wired sync for photo transfer after giving up on iCloud Shared Albums. The simplicity is a relief — except with 11.0 it wasn’t working well. Out of 8300 images in Aperture I was lucky to get 6200 to sync. Sometimes repeated sync brought over a few more, other times I was stuck.
With 11.1 I got them all.

I still can’t get iTunes WiFi sync working though.

PS. I finally found some use for my iPad Air 2. I set it up at work without a network connection and play on device music and randomly display family album images using LiveFrame.app. I use MindNode.app to organize projects. It’s still marginally useful but that’s progress.

Update 11/3/2017: I think WiFi sync is working now.

Update 6/9/2018: I had to do a restore to my iPad and the bug is back again. I might try iOS 4.

Repairing Aperture database: 1061 recovered images ...

Aperture is an artifact from an age where Apple made ambitious software to handle big challenges.

Those were the days.

It’s not, however, flawless.

Periodically I rebuild the Aperture database. Sometimes all is well. Today there 1061 “recovered” images.

I’ve been through this before so I didn’t panic. I sorted by size and chose 10 version names from the largest 50. I then searched on each name and all of them were in the proper place at a larger size as well as in the “recovered” project. The recovered images were thumbnails.

When I get 10/10 I assume they are all thumbnails (most are a few hundred K vs. images of 3MB to 30MB). So I delete the Recovered Folder.

It is unsettling how often I have to do this.

Moving the family to 2TB of iCloud shared storage (Family Sharing)

We have five family members with 7 devices using iCloud storage. Four of us are paying $1/month for 50GB of storage, one is fine on the free 5GB. Mostly this suffices, but my daughter’s photos have been near the edge for a while [1]. Instead of updating her quota I decided to update the family to 2TB ($120 a year) and not have to worry about this for a while. First, of course, I confirmed that there was a way to unwind this decision.

I wondered how it would go since we all use a single iTunes account for purchasing DRMd material (video, movies, etc). We’ve done this long before family sharing was a feature and switching from to the new approach is a bit worrisome. (Family Sharing of DRMd material does have one advantage — whoever buys things gets to keep them when they separate from the family. As the kids tastes have diverged sharing is less useful.).

The iTunes account turned out not to matter. Family iCloud sharing runs off the family owner iCloud account, not the family owner’s iTunes account (Of course for most people these are, at least superficially, the same thing. I suspect they aren’t all that integrated in Apple’s famously messed up back end servers.)

I followed Apple’s support guide to update myself to 2TB. The guides aren’t that well done, they don’t make it obvious how the process works. I couldn’t enable the family sharing of my iCloud quota in Sierra (probably needs High Sierra), but it was easy to do on my iPad. Once that was enabled an iMessage went to family members for them to transition.

The experience for family members was a bit glitchy. For two members I could just tap on the iMessage, enter iCloud credentials and accept the switch. Users get a notice their current plan will end and they’ll receive a prorated reimbursement. The family member who was on the free quota didn’t need to opt in, he just go the expanded quota (an interesting security-convenience tradeoff). For the fifth family member I ran into several failed attempts on an iOS 10 device. I got as far as entering his credentials to access the upgrade but then the process failed. On his iOS 11 device I was able to make the switch from this iCloud manage storage settings. His iOS 10 device then also showed the correct settings.

Two TB is overkill. I might drop back to 200GB in a month or two — especially since I’m using less iCloud storage now. (I’ve gone back to Aperture-iTunes and a lightning cable for getting photos to my devices).

Update: After doing this I realized our family total is now just 67GB. So I downgraded to 200GB. Interestingly this didn’t take effect immediately — I wonder if it will take until the end of the current month. I’m not that concerned about the delay but it is a bit annoying. Note that we currently pay $4 for 200GB for each of four users, but will pay $3 for 200GB shared between 5 users. The 25% cost reduction is nice (I’m cheap) but the real deal is that a shared 200GB is far more efficient. My daughter will end up using 150GB and the rest of us will be fine with the residual 50GB. I’ll move the family to 2TB if/when we need to. Perhaps by summer 2020 Apple will have enhanced Photos.app and iCloud Photo Library enough for me to switch off Aperture. I’ll need the 2TB for that.

Update 11/8/2017: I checked with Apple Support via Twitter. We prepay for storage. When you go up it’s an immediate change and you are billed in a prorated fashion. When you go down the change applies to next billing cycle.

- fn -

[1] On the one hand the 50GB cap does motivate her to edit things. That’s good, but really she has enough stuff to deal with.  She doesn’t need one more painful discipline. Sometimes I gotta walk my Dad mistakes back …

How to do a simple random image picture frame type slideshow in iOS 11 on an iPad.

You know how you look for something on the web and you can’t find it?

That’s because it can’t be done with the base OS and nobody talks about the features that Apple has removed. There’s no ad revenue in that.

But I don’t take ads, so here you go.

This post exists to tell you that as of iOS 11 you can’t create a random picture-frame like slideshow on an iPad without a 3rd party app. Yes, the iPad used to be able to this. Once upon a time you could set a random image display up as a lock screen. Later this was moved to the Photos app.

With iOS 10 it died. In Photo albums on an iPad there’s a slideshow button (top right), but it only plays linearly. Which I loathe.

There are slideshow options by the way. They are insanely obscure. Start a slideshow. When an image appears, tap on it. There are a few options. No “shuffle” though.

There used to be an excellent third party app to do picture frame slideshows called Picmatic. My father loved it. He died before I updated his iPad to IOS 10. Good thing, because iOS 10 broke Picmatic and the developer never updated it (damn thing needed #$!$ subscription pricing).

There’s one “Picture Frame” app left on the App Store — LiveFrame.app. It worked with my Google shared libraries, but even though it could “see” my iCloud Shared Libraries it would hang when I tried to use them.

I’ve been fairly disgusted with iCloud Shared Libraries so I decided to try life without them (to be fair all image sharing except Instagram seems to have died). I turned iCloud Shared Libraries off on all my devices. I’d already given up on iCloud Photo Library. Then I went back to the stone age. I connected iTunes 12.7 to my iPad with a Lightning cable and had it sync 8,300 images from my Aperture “slideshow” smart album.

Of course the sync didn’t go easily. The image transfer aborted 3 times — without any notice. Mercifully the sync restarted where it left off. Unlike iCloud Photo Sharing I think iTunes supports a true 1 way sync; updates are relatively painless.

With the photos on my iPad, and no evil iCloudness, LiveFrame.app works. It’s no Picmatic, but it’s there. The developer should go to subscription pricing so they have an incentive to keep it around. There’s zero competition and this app is a perfect fit for subscription (no data lock, nothing to prevent switching).

Note — this slide show doesn’t need a data connection. The images are on the iPad.

So today one Apple thing worked — albeit an old thing. Sort of. That’s pretty good for Apple in 2017. (I can’t believe people are buying the iPhone X. Are they insane?!?)

iCloud Family Sharing storage: What happens to Alice's photos when Bob drops her from the Family?

Alice and Bob are a “Family”. Bob, the Family Organizer, pays 2TB of iCloud shared storage. Alice has a 1TB iCloud Photo Library.

Alice and Bob split. Bob drops Alice from the “Family”. What happens to Alice’s photo library?

Apple doesn’t discuss this in their support article on leaving family sharing [1]. So I asked on Apple Discussions. I received several responses that seemed suspiciously knowledgeable [2].

There’s nothing written down, but I think both of these responses are correct …

"Apple is allowing a grace period to transition without issues, but that is an unstated, voluntary policy. AFAIK, Apple makes no promises of any kind that this policy won’t change, or even that it will be applied to all users.”

and

"iCloud keeps all of the information associated with your Apple/iCloud ID for 30 days whenever you have a payment issue or change.

When you joined Family Sharing, your Photo Library did not move or get re-associated with the Apple/iCloud ID that "owns" that data. It is tied to your ID even if you went with the Family Sharing plan. All that does is move the responsibility for paying for the storage from you to the Family Sharing organizer.

All you need to do is leave the Family Sharing plan, and then upgrade your iCloud Storage. Anything that was stored under your Apple/iCloud ID will remain in iCloud for 30 days, so if there is a gap between when you leave (or were removed) from Family Sharing and when you upgrade your iCloud Storage, as long as it doesn’t exceed 30 days, you should be good to go."

My takeaway is:

1. Apple needs to write this down.
2. Alice probably has 30 days to up her storage before she loses her photos (or, if Alice is geeky, she can move them locally).
3. Alice should probably up her storage before she’s dropped from Bob’s account (assuming she has warning).
4. Alice should always have a local full res Photos.app Library that’s backed up to a local drive (probably not by Time Machine, Apple is shockingly unclear about whether Photos.app can be safely backed up by Time Machine).

- fn -

[1] This is worth reading. I thought that children, on reaching 18, could retain a copy of DRMd material. Either I remembered incorrectly or policy changed. Effectively any FairPlay DRMd item has only one iTunes account owner.

[2] I think some respondents on Apple Discussions have inside information. I don’t know if they are contractors or employees or what.

My latest attempt to reclaim my wife's stolen email address from Facebook

You cannot reclaim a personal email address used by a stranger’s Facebook account. Facebook’s procedures do nothing. 

Today I tried something different.

To review, a year or so ago my wife started getting Facebook notifications from “Ding”. Her iCloud email address was used by “Ding Ling” (https://www.facebook.com/ding.ling.98031) to create a Facebook account. I don’t know if my wife accidentally validated it or not but we’ve been unable to retrieve her email address.

Since we control the email we can reset the account password, but Facebook’s poorly documented procedures to reclaim the address did nothing.

So this time I again reset the password but then I followed Facebook’s directions to upload a legal ID (a way to take control of the account):

I used my drivers license as a template to get past Facebook’s image size test then overlayed a text image requesting release of her email address.

I’ll update this post if it works. 

Yeah, there should be a law.

Update 10/29/2017

It worked. It took a few go rounds. Eventually I was directed to the same kind of form I’d submitted a year ago, but this time it got to a human. I had to send a summary response email from Emily’s iCloud account. I think it helped to point out that “Ding Ling” was obviously a fake account. Once the email was retrieved I added it to her account.

I wonder if somehow it mattered that Emily has both an iCloud and a “me” address for the same account (due to Apple’s old mobile.me migration). Her .me address was already associated with her Facebook account.

iCloud calendar invitations to non-iCloud accounts are still broken

It’s been over a year since I first posted that iCloud invitations to a non-iCloud (ex: Google) account have been broken since 2011. Briefly, if someone sends an invitation from iCloud to my gmail address I’ll never see it. Somehow Apple looks up one of the four (that I know of[1]) iCloud accounts that I have, presumably one that references my gmail address, and uses that one instead.

In late 2016 Apple introduced an obscure workaround, an advanced iCloud (only) Calendar setting to receive event invitations by email — “if your primary calendar is not iCloud”.

I was working on a book chapter today so I revisited the old bug to check out the workaround. From my son’s (unused) iCloud Calendar I sent myself an invitation. Despite the setting nothing appeared in any of my (unused) Apple iCloud Calendars.

I waded through my Apple IDs to identify which one was associated with that Gmail address. I had to answer Apple’s “secret” questions [2] several times, but I found an Apple ID of mine associated with an iCloud account that did had the “receive by email” option enabled and had iCloud mail services. I tried from several iCloud accounts; none of the invitations appeared anywhere. They didn’t show as email, they didn’t show up on my iCloud calendar. They went into the ether.

Apple iCloud calendar invitations to non-iCloud addresses are still broken.

[1] Multiple iCloud accounts, some with email services and some without, is a longstanding Apple fiasco. Cook promised to clean it up several years ago and quietly abandoned hope. I periodically read hints from insiders that Apple’s identity management is more screwed up than even the most cynical outsiders can imagine.

[2] Also known as a hacker’s best friend.

 

Photos.app flailing during sync with iCloud Photo Library? Maybe it's a permissions problem.

My daughter uses Photos.app and a 50 GB iCloud Photo Library to manage her videos and images [1]. She edits on an older Air with a small SSD, in that environment Photos.app caches scaled res images and only downloads full res when editing.

In addition I run an instance of Photos.app for her that stores full res images. The Photos.app Library is stored on an external SSD that hangs off an Elgato T2 Hub attached to my beloved Air. The hub has been very reliable under El Cap and Sierra.

I have a user account for her on my drive, and in that account the external library is the Photos.app System Library. My Time Machine [3] and Carbon Copy [4] backups include that Library.

All was well under El Capitan. A few months ago I upgraded to Sierra [2]. Yesterday I decided to update her Photos.app library — only to discover I was a few months behind [5]. Her user account hadn’t been updated to Sierra; when I opened Photos.app her Library had to be updated.

Things did not go well. Photos.app said it was uploading @8,000 images (really it shouldn’t have uploaded anything, but Photos.app sucks), then @2,000, then @11, then … You get the idea. It did that when I went to bed, and it was doing it in the morning.

After a bit of playing around I discovered that a Sierra bug meant that she no longer had write permissions to the external SSD, even though macOS said she did. I switched to an admin account and there she had no permissions, so I added her. After that she could write to the SSD. Photos.app “stuck upload” was because it had no write permissions at all.

I decided to create a fresh Photos.app Library for her. To do that I turned off WiFi and did option-Photos.app startup to create a new Library. I copied the old Library to an external drive and deleted it. I then opened the new Library, made it the System Library (interestingly it showed images from a cache!), turned on WiFi and enabled iCloud Photo Library. The images then downloaded from iCloud (source of truth) and restored my local backup copy.

- fn -

[1] She is chronically running against the limit — which isn’t all bad. It enforces some editing. I might switch to sharing a 200GB plan, but I’m not sure how that will work with our current family use of a single iTunes password. Future experiment needed.

[2] I like to wait at least 8 months before accepting Apple’s dangerously buggy macOS updates.

[3] Our two Airs do Time Machine backups up to a Synology NAS. After some initial issues that has been utterly trouble-free. The NAS has two RAID 1 drives, if one fails the other survives. This is another reason I wait for macOS bugs to get fixed; I also need things like VMs and NAS to be updated.

[4] CCC backups to a 4TB low heat drive in a Voyager cradle with Firewire 800 connection to the Elgato hub. I rotate 4 drives. Rotation is every 2-3 weeks, 1 drive is across town, the other in my Van. A Yellowstone eruption would take them all out unless the van outran the pyroclastic flow. It is a shame that offsite internet backup has failed.

[5] Only automated backup ever works — and no form of backup is reliable.

Mobile device management and Apple Classroom for home

Contrary to Ziff-Davis (ok, it was 2013) there are several vendors who provide MDM solutions for home use. MMGuardian and Qustudio are two of them.

I wonder though if it’s possible to cobble something together at home that would work with Apple Classroom. OS X server ($20) includes Profile Manager, Apple’s MDM manager (support). Joshua Jung has written a nice tutorial on getting Profile Manger working. In theory Apple Classroom should be able to work with this …

Anyone try it?

Quicken for Mac moved our financial data to their servers and we can't remove it

From Quicken.com support:

What is the Quicken Cloud?

 …The Quicken Cloud data cannot be deleted, although there may be an option to delete it in the near future.

Quicken Cloud is used to sync data for mobile devices. We don’t use Quicken mobile, we only use Quicken for Mac. We did not enable Quicken Cloud sync, we were careful not to enable it.

It appears an update turned it on. Our financial data is now on Quicken’s servers. The servers of a company that clearly has its head deeply buried in an orifice. I’m sure they’re just great at net security.

Anyone know of any good lawsuits against Quicken.com we can support?

Update 10/29/2017

Via Twitter: FAQ: How to remove cloud data in Quicken Mac 2017 4.6.x. I followed the directions. As best I can tell data can flow from most of our transaction partners either directly or via the Quicken cloud. When this procedure is applied transactions revert to Direct where possible. Some don’t work Direct, they stay in the Cloud.

Life with Apple: Podcasts move to streaming only

In a move that feels as inevitable as death and taxes, Apple has made podcasts effectively streaming only. The same thing is likely to happen to all media, Podcasts just went first.

Until recently macOS iTunes and iOS Podcasts.app supported both file based sync, including Playlists, and streaming based distribution with an unreliable sync of podcast metadata.

With iOS 11 iTunes playlists are no longer represented in Podcasts.app. You can still create Playlists in iTunes made up of local files and/or cloud references, but they stay in iTunes. The Playlists were the main way I organized listening to my large collection of file-based In Our Time podcasts and my medical education podcasts. No more. I’m now dependent on the very limited (ok, crap) organization abilities of Podcasts.app.

Apple has long had problems with video that moved to an iPhone from both iCloud and iTunes. iBooks synchronization is a mess too if you mix non-Apple store ePub with streamed iBook.

This is ugly and going to get worse. We’re moving fast into the DRMd hard data-lock rental-only future.

Update: Many good threads on this in Apple Communities. They reminded me to leave feedback. Stations are not a substitute for playlist sync, but even on their own they are missing a key filter — limit to on-device items.

Photos.app is as stupid about JPEG Export as iPhoto and Aperture.

Try this experiment.

1. Import a camera produced JPEG into Photos.app. Check the size. Let’s say it’s 6.8MB.

2. Don’t edit the image. Export it as maximum quality JPEG. Check the size. It will be something like 17.2MB.

Even though the image was native JPEG, and it wasn’t edited, Photos.app decompressed and then recompressed it. Adding 11MB of non-value.

Unchanged from iPhoto and, for that matter, Aperture.

There should be a better way. (No, unmodified original is not a better way — because if editing were done, or the original were not JPEG, then you would want the transformation.)

Annals of iOS inconsistency: Contacts vs Notes vs Reminders - backup and sharing

https://www.icloud.com/#settings currently shows an “Advanced” subsection for restoring Contacts. It provides options to restore an iCloud data set “archive” from iCloud (not to be confused with restoring an entire iOS device backup):

Notes aren’t on the list though. They have their own note-specific backup restore option, but it’s at the level of an individual note and there’s no version restore, only the ability to undo a delete for 30 days by restoring a Note from “recently deleted”. (BTW, if you Share a Note only the Owner can “delete” — but anyone with Edit privileges can remove all content — and since there’s no version undo that means anyone who can edit a Note can delete it without a recovery option.)

Sharing is another area of odd inconsistency. Notes must be shared one at time, but multiple Reminders can belong to a set of People.

I’d like to see Notes add Google-style Note-specific version save/restore and share by container (folder) as well as Note, but there’s no rumor of that in iOS 11. I’d pay for a third party solution for iCloud, similar to what CloudPull does for Google App docs, but I fear the demand is too small (for example). An Apple iCloud Drive folder view of Notes [1] would be a big help; I’d then be able to restore an individual Note from a Time Machine or Carbon Copy Cloner backup …

Anyone have an AppleScript to create a local daily snapshot of Notes? (There is this, but in Sierra Apple omitted AppleScript dictionary support for PDF creation).

The world moves in unexpected ways. We seem to be converging on a form of backup that’s a regression for people like me, but a big improvement for most. There’s probably some kind of futurist principle there — the good-enough mass solution will drive out the elite ideal …

- fn -

[1] The main reason I’m still on Simplenote is that nvAlt on my Mac maintains a synchronized file store that works just like this. Perfect data freedom — but almost nobody appreciates this …

[2] As of Sierra at least some parts of Notes are in /Users/[username]/Library/Group Containers/group.com.apple.notes. This location has changed a few times. Note content is distributed between media files (PDF, etc) and text in a sqlite database, so recreating an individual Note document as, say, an RTF file, is a non-trivial task. For example (sqlite browser):

I suppose Time Machine backups of this folder might be a kind of ‘restore all notes’ option, but restoring a version of an individual Note would be tricky…. (There’s something deep here about the ways in which we assemble bits to create something our brains perceive and our tools manipulate, but it’s beyond my ken. Once upon a time a BYTE article would have traced the roots of the Notes sqlite store back to database file systems of the 1980s…)

Facebook won't let you use the email '+' extension as a new email address

Gmail, and some other systems, support a very old email standard that needs a name. Gmail will treat myname+123@gmail.com as though it were myname@gmail.com. It’s handy for filtering email lists.

I wondered if it could use it with Facebook accounts.  As I discuss in my book there are many reasons to have Facebook related email for a vulnerable user go to their parent or “Guide”. Facebook doesn’t allow an email address to be associated with more than one account — maybe the + feature would work …

Except it doesn’t. I tried adding a + variant of my personal email to one of the kids accounts and Facebook told me it was in use.

Bummer. Now you know not to try.

Incidentally, iCloud support up to 3 aliases, so you can do this with an iCloud email alias. Alas, regular Gmail does not support true aliases — only the + suffix trick. Google Apps does support aliases, at least if you own a domain, but that’s strictly a geek or business thing.

Getting around Koodo's (Canada) password reset bug - use incognito browser

Americans think Comcast customer service is bad.

It is true that Comcast is not great — though it is better than it was. Compared to Canada’s mobile carriers though, Comcast is bloody Apple.

My sister uses Koodo, a Telus subsidiary. She was unable to pay her bill for a couple of months, attempts to pay by credit card were met with a nonsensical error message. Customer support couldn’t help. Somehow I figured out their security system was balking on an address mismatch between her bank and her Koodo account — and producing a red herring error message. Of course changing her address on the web site didn’t suffice…

Eventually Koodo changed her address. Since her only computer is an iPhone, and since Koodo doesn’t have a mobile solution (but, you say, “they are a mobile service …”) I did the transaction online.

Or rather, I attempted the transaction online. Koodo wouldn’t recognize her password. They’re happy to do resets though, but the reset passwords wouldn’t work. Koodo wouldn’t let me reuse them though — it was storing the password hash correctly (or, knowing them, the password in plaintext).

I kept getting this message:

Something went wrong!

Sorry, the username and password you entered does not match our records. Please try again.

Warning: Five (5) unsuccessful attempts will cause your online access to be locked for one hour. If you can't remember your password, reset your password now before your account is locked.

The fix?

Well, I knew the username and passwords were correct, so I figured the real error was again unrelated to the error message. My bet was on some cookie state.

So I tried with a Chrome incognito window. That worked.

There’s no way Koodo tech support would have figured that out. What a hopeless company.

Take heart America. We may have the worst leader since Andrew Johnson, the greatest threat to civilization since Mao, but Canada has Koodo.

Ok, forget that. We’d rather have Koodo.

After macOS 10.12.6 a lot of apps are doing the "quit unexpectedly"

Scrivener is the 3rd app today to “quit unexpectedly”. It’s been like this since 10.12.6 went in.

Restarts aren’t helping …

How I plan to test my Aperture Library against Sierra

It’s almost time for me to move to macOS Sierra, now that the usual post-release beta testing is drawing to a close.

This is what I am doing to test my Aperture Library on Sierra:

1. Export a small Project as a Library.
2. Open the small Project and delete its contents. Now it’s an empty shell with my settings in it.
3. Import my existing Library into it. So everything is written with the latest version of Aperture.
4. Test with #3.

Apple Discussions have lots of fix and workaround suggestions for early Sierra, but I’m told they are no longer necessary. The things broken in El Capitan and broken in Sierra, but nothing new.

How to delete your iCloud account and Apple ID

First, invent a time machine.

Second, go back in time and force Apple to add account removal.

For now - you can’t.

You can remove your Google account. You can remove your Facebook account. You can’t remove your iCloud account and your Apple ID. They are eternal.

I’ve run into this little oddness before, but I was reminded of it when cleaning up my deceased father’s online presence.

A 2013 Apple forum post says: “Access can be stopped by Apple if they are provided with your Death Certificate.” I bet you have to fly the certificate to Cupertino. Even then it’s not clear if any data is deleted. I wonder if anyone has ever done this.

Apple gets away with a lot.

PS. I did set his email to forward to me.

OS X (macOS) installer sizes - Mountain Lion through Sierra

Sierra is almost ready for release now, so I’m preparing to install.

Interesting to compare installer sizes from the download era:

Release	Size (GB
Mountain Lion	4.46
Mavericks	5.33
Yosemite	5.68
El Capitan	6.21
Sierra	4.97

Sierra isn’t much bigger than Mountain Lion.

Curious.

Apple drives me into Google's arms - using Google Photos with iPhone and Aperture

Apple has broken me. I’ve left iCloud Photo Stream shares for Google Photos.

First I lost the ability to share from Aperture to Facebook. I think that was probably a Facebook change, but of course Aperture isn’t getting updates any more.

That was annoying.

Losing Apple Photo Stream was much worse. Photo stream wasn’t great, but it was simple for my daughter, sister, and other users to subscribe to. For a time I could use iCloud Photo Library on Photos.app alongside iCloud Photo Streams on Aperture [1].

Then Aperture retched and I lost my shared photo streams (but not, happily, the originals). I played around with restoring iLifeAssetManagement from backup but, despite early promise, I couldn’t defeat Apple’s black box sync infrastructure [2].

That’s it. I’m toast. I surrender. Google’s inexplicable aversion to album creation on upload is the lesser evil now.

I’ve installed Google Photos on my iPhone and enabled backup and sync. I’ll use that to cull and play with photos before I transfer them to Aperture.

I’ve freed up 14GB from my Air’s SSD by deleting iLifeAssetManagement and I’ve installed Google Photos Uploader.app. I pointed that to a folder on an external drive, when I want to share from Aperture I export there for upload. I do my post-upload organization and sharing through the web UI.

Since Google nicely migrated images when it closed Picasa Web Albums my new shares are reunited with my old Picasa web albums. I’ve come home again. Though I’m still puzzled by Google’s weird album aversion.

It’s far from ideal, but Apple has burned me yet again. They seem to despise my data.

[1] Though I gave up on iCloud Photo Library when I realized it was more or less incompatible with importing images from iPhone photo roll to Aperture.

[2] Apple is famous for sync that disallows any kind of troubleshooting.

iLifeAssetManagement

Partial restoration of lost Apple iCloud photo stream shared albums (updated: didn't work)

Something went wrong. It always does.

I had thousands of images distributed across over 60 shared photo streams. One day I rebuilt Aperture’s database and all the iCloud images were in one recovery folder. I deleted them and then most of my iCloud shared albums vanished.

This is a quick summary of how I recovered most of them from backups. I don’t know how this truly works, but it seems that this folder in my user account was a source of truth for iCloud photo streams:

/Users/[my user name]/Library/Application Support/iLifeAssetManagement

I copied what was there to an external drive then deleted it, logged out (necessary to close open databases) then logged in. With Wifi on when I launched Aperture it showed no images at first then downloaded what was in iCloud. So there was some kind of sync.

Next I did the same thing (closed Aperture, deleted, etc) but this time copied a backup of iLifeAssetManagement from prior to the bad event. I then turned off wifi.

On relaunch Aperture showed about 6100 images in “Shared:iCloud”. It rebuilt thumbnails for them. Then I turned on Wifi. Next I saw the count rise briefly as albums I’d shared previously came down from iCloud. Alas, the count started falling again, stabilizing at 5600.

I had most of my streams back — though one stream was much smaller than it used to be. Still, about 80% recovery and I didn’t lose a few I’d done post-disaster.

Better than nothing.

Sync without controls is truly hell (and Apple never provides enough control).

Update: Aperture shows 56 single owner photo streams (one is empty) and 5 shared. iOS Photos.app shows 20. At least one of the iCloud albums not seen in iOS photos.app cannot be found at its public link. The iCloud library and the Aperture iCloud library are not in sync. So I’d call this a failure.

MarsEdit tables: create in TextEdit and paste

MarsEdit is a great app — but I wish it were a rental product. Then I’d pay every year and Daniel Jalkut would be incented to add support for image resizing and table editing.

In the meantime I’ve discovered I can get good results by creating a table in TextEdit then pasting it into MarsEdit rich text editor.

iTunes purchased movies showing only a handful of my purchased movies

iTunes on my Mac shows 138 movies of which 135 have purchase dates. Those 135 were purchases from Apple starting in March 2009.

If I visit those 135 in iTunes some allow me to download, some invite me to purchase again. One old SD movie allowed me to download and that worked on my iPad and showed in the Purchased list after download.

My iPhone seems to show about the same number (though it doesn’t count for me).

My new iPad shows only 24 movies as purchased. Both devices have the same iTunes account. When I view my devices in iTunes (we’re at the 10 device limit) both my iPad and iPhone show up under the same Apple Store Apple ID.

One possibility is that the purchased list on my iPad is only showing HD movies. I can’t tell from iTunes which are HD and which are SD; we almost always buy SD when it is available. Some 3+ GB files are probably HD and they don’t show up.

Something is broken in Apple DRM land. I have a private message into Apple Twitter support and an Apple Discussion post …

Update: The “something that is broken” thing is the user interface. Apple Twitter support pointed me in the right direction.

When I view Movies in TV.app on my iPad there’s a subtle top left drop down called “Library” that on tap shows a hidden filter that defaults to Recently Added.

When I change that to Movies I see them all.

I was fooled by the "See All" link to the right of "Purchased Movies" that displays when the hidden Recently Added filter is active. The “Purchased Movies” heading should really be “Recently Purchased Movies” when viewed in this mode. It shows 4 recently purchased movies, and “See All” shows all recently purchased movies.

How to upload images to a specific album in Google Photos

How to upload images to a specific album in Google Photos:

1. Create the album. You have to choose an existing photo to create it.
2. Drag and drop the photos you want to upload onto the album you’ve created.

Despite years of customer requests you still can’t select a folder and upload it into an album.

I’ve read that if you work with full res (not reduced) images in Google Drive you can organize them in folders and turn those folders into albums in Google Photos. These count against storage costs. I have not tested this.

Aperture crash - sad day for my iCloud Photo Share streams (shared albums)

Aperture locked up when duplicating an image. I had to force quit, when I restarted I rebuilt the database. 5,300 images showed up as recovered.

Turns out they were all thumbnails for iCloud shares, but they’d lost connection to iCloud. When I deleted them I found most of my iCloud share streams were empty.

I believe I have my images, but it is sad to lose the relationship to the shares. Aperture is no longer supported by Apple of course. I’m running El Capitan, for what that’s worth.

Backups are no help of course. Even if I could recover the relationship to photos shared in iCloud I’d lose other work.

Update: 

/Users/jfaughnan/Library/Application Support/iLifeAssetManagement/assets/pub has 7.5GB of files holding 2,634 items including some photo stream temp files. It’s not clear if this can be deleted, but it may be Aperture doesn’t use it…

• tacos - 53 GB in iLifeAssetManagement folder but only 5 GB in Photo Stream - Ask Different
• tacos - What is iLifeAssetManagement, and can I delete it? - Ask Different
• After migrating to iCloud Photo Library can the iLifeAssetManagement folder be deleted? - Ask Different

There may be a fatal flaw in my backups. (actually, no)

I’m leaving this one up as a reminder of how scary the world of secure backups is, and how important it is to actually do a dry run of a disaster recovery scenario.

This is the original post. It’s wrong:

Don’t every tell me backup is a solved problem.

I have offsite backups of my data. Two offsite and two onsite Carbon Copy clones that I rotate. In addition to my onsite Time Machine backups.

All encrypted of course, because otherwise that would be terrible.

Great. All set. If the house burns down we’ll have our data (assuming we still need it).

Except those drives are whole drive encrypted with FileValue 2. So each has a unique recovery key. A recovery key that is different form each backup drive and can only be known at the time of encryption. A recovery key that is stored in a keychain on my MacBook. A device that can be lost.

I’d be better off if that recovery key were in iCloud, but I don’t think it is. Or I could follow Apple’s complex directions for managed recovery keys. Or I could have created encrypted sparse image folders for CCC, I’d know the image password then. Or maybe created bootable encrypted disk backups.

I have a bad feeling I don’t really have backups at all.

There’s a fine line between security that makes data inaccessible to bad actors and security that makes it inaccessible to everyone.

I hope I am wrong about this.

It’s wrong because FileValue 2 whole drive encryption actually behaves like the disk image encryption I’m familiar with. I was confused by the Recovery Key complexity. Doing a dry run of disaster recovery shows what happens.

I mounted one of my encrypted backups using my Voyager cradle and a USB 3 to UBS 2 cable with an old Air. I was asked for the password I’d used to encrypt the drive, not for the recovery key. I was able to mount my backups just as I would on any foreign Mac.

That password is the same for all my backup images and it’s stored in 1Password as well as printed. I’m going to add it to the Dead Man / post-mortem document I keep in Google Drive that’s shared with several trusted people.

False alarm. Need more coffee.

See also

• The curse of iTunes encrypted iPhone / iOS backups: the password is NOT your device password (and how to recover it from OS X Keychain) 8/2013. Another surprise.

Touch ID security issues are less than i thought.

My Touch ID security post of a year ago didn’t get any reaction. So today, while working on a bak chapter, I decided to retest my concerns.

On my own phone, with iTunes Store Touch ID purchase enabled, I added a new fingerprint of mine. To do this I only needed my iPhone unlock code. I then purchased a song (Cheryl Crow’s Heartbeat Away fwiw).

I wondered if iOS would let me complete the purchase — in which case there would have been a risk issue. It didn’t work that way though. Despite my having Touch ID enabled for iTunes purchases iOS requested by Apple ID password — even though I used an old fingerprint. Adding the new fingerprint seems to have put my phone into a ‘enhanced risk’ category, so Apple ID credentials were required for purchase.

Once I’d entered my Apple ID the first time though I was able to use my new fingerprint for the next purchase. So entering that Apple ID password “blessed” all fingerprints.

So there’s still a way to “sneak” a fingerprint into the cue that might be exploited for unauthorized purchases, but it’s a smaller window than I thought. I tried “gifting” an app to see if I could exploit that, but abruptly my iPhone stopped being able to gift at all. So I couldn’t test. (Bug?).

Something wrong with Google's Device Activity page?

Google’s device activity page used to let me remove devices that I no longer used:

https://myaccount.google.com/device-activity

Today it lists 3 iPhones for me. I think they are all actually one device, showing up with two different names — none of them current.

There used to be away to remove these devices, but today I can’t. I am able to Remove Account Access, but not the devices.

I removed access to all devices, then reentered credentials on my phone. So now I know that ‘iPhone” is my phone (phone name is John6s).

I think something is broken somewhere….

DreamHost remixer - a brief trail

I tried DreamHost’s remixer web site dev tool. I was able to create a page with it. Reminds me a bit of Apple’s old iWeb. 

It doesn’t output static files though. It is mapped to a folder on my DreamHost site, but there’s some redirection behind the scenes.

That means it’s transient. When Remixer dies, so will all the content in it. It’s not portable either, I can’t move my Remixer work anywhere else.

Might be a good app for something transient, but the little I do on the web today I like to keep portable. WordPress is as far as I’m willing to go into things I can’t readily move and backup.

Why is Apple's mysterious two factor authentication better than a strong password?

What would I do if my home burned down and Emily and the kids get out alive but I’m dead?

That’s what I think about when I read about Apple’s “two-factor” authentication (vs. the now obsolete but similar “two-step verification” they used to have). Particularly the scary procedure you need to follow if you’ve lost your authentication devices …

Two-factor authentication for Apple ID - Apple Support

…. If you can’t sign in, reset your password, or receive verification codes, you can request account recovery to regain access to your account. Account recovery is an automatic process designed to get you back in to your account as quickly as possible while denying access to anyone who might be pretending to be you. It might take a few days—or longer—depending on what specific account information you can provide to verify your identity…

… With two-factor authentication, you don't need to choose or remember any security questions. Your identity is verified exclusively using your password and verification codes sent to your devices and trusted phone numbers.

 and

Regain access to your Apple ID with two-factor authentication account recovery - Apple Support

… You might be asked to verify other account information to help shorten your recovery period. After you verify your phone number, you’ll see a confirmation that your request has been received and you’ll be contacted when your account is ready for recovery...

… We’ll also send an email to your Apple ID or notification email address to make sure you’re the person who made the request. You can click Confirm Account Recovery in the email to help us shorten the account recovery period. …

Scary indeed. It’s vague as hell. Even control of a confirmed email account (presumably different from the iCloud account) only “shortens” the recovery period. There’s nothing in Apple’s process comparable to Google’s inactive account manager. There’s no secret recovery key I can store in an encrypted repository on an offsite drive with a password known to 3 family members.

Apple’s 2FA either makes my data too hard to recover or too easy for someone to steal … or both.

I don’t see the advantage, yet, over a strong password used only on a secure device. Google does this better — and even Google 2FA is too complex for me to manage for multiple family members.

I’m staying with a strong iCloud password for now — until Apple forces me to change. (The way they’re forcing 2FA with the 10.3.1 update makes me wonder if iCloud really was thoroughly hacked.)

PS. As best I can tell if you use Apple’s new 2FA when you change your iCloud password on one device you change it on every authenticated device. Better be sure you have them all.

PPS. At least they got rid of the secret questions … but only to replace them with some mysterious, fully automated, no humans involved, identity validation process.

PPPS. Ok, we’re traveling. Both our iPhones are lost. What do we do? hmm. I think Charlie Stross had something about this in a story … accelerando?

See also

• Gordon’s Notes: Preparing for the inevitable - Google Docs for the “Not available” letter 2/16/2013. There are certain passwords in this document that useful to people who have physical access to one of my backup drives. I have a task that triggers a reminder 3 times a year to my trusted persons that this exists.
• Gordon’s Notes: Inherit the Cloud: Who gets your Google Docs when you die? 2/17/2013

Migrate Contacts from Outlook/Exchange server to OS X Contacts

I last wrote about migrating contacts from Outlook/Exchange server to OS X in 2011.

Back then one of the issues with migration then was that Exchange uses X400 format email addresses. One way to make Outlook convert to internet is to drag contacts into an email — that produces .vcf files (undocumented) with internet email addresses. In 2011 I wrote that dragging into a local PST did the same thing. Not sure that still works.

Here’s what I did recently …

1. Drag and drop from Outlook/Exchange into email (undocumented export)
2. From email save to desktop.
3. Drag and drop the .vcf files into OS X Contacts (undocumented import.

From OS X contacts they can go to iCloud as usual.

Incidentally, i couldn’t see any way to readily import a set of single contact .vcf files into Outlook 2016. It only wanted to import them one a time. Drag and drop of the files into Outlook resulted simply in Outlook opening each one until Windows 10 die.

Exporting multiple emails from macOS OS X Mail.app on 10.11 (El Cap): only PDF works.

An email search retrieved 100+ emails. I wanted to copy them to a separate folder for archival purposes. I know one can drag and drop a message and create a net standard ‘.msg’ format but when I tried that only worked for 1 message at a time.

Save As allegedly worked at one time, but when I tested both rich text and .mbox export they produced one large file that wasn’t very readable.

What worked was File:Export as PDF. That worked.

pnut.io - an app.net heir

There are several heirs to app.net — I know of 10Centuries.org, pnut.io and Manton Reece’s micro.blog kickstarter.

Each of these efforts is a largely one person project. All good people. pnut.io’s creator is  33mhz/Robert. I believe the API is based on the app.net API, which means the many apps created for app.net can have a second life. I’m using ChimPnut.app for iOS and the https://noodle.s3rv.com web client to access pnut.io. I’ve been told the app.net client Riposte.app may reappear.

pnut.io has an interesting approach to social networking …

Our community network is a small, sustainable hobby. We are tied closely to our users and developers through their Patreon pledges and small opt-in features. When you invite someone, they are associated with you in the user tree and you may be responsible for them if they abuse the network.

 I made a Patreon pledge at the $5/month level but anyone can use it for free. The spam protection is the user tree. It will be interesting to see how well that works.

If you know me from app.net days or elsewhere and need an invitation send a note to me at jgordon@kateva.org. Best to mention how you know me.

 

 

Escape from Outlook Notes - export as text files, import into NvAlt or other

Back in 2010 I wrote Gordon’s Tech: Escape from Outlook Notes - ResophNotes, Simplenote for iPhone and Notational Velocity.

Seven years later ResophNotes, Notation Velocity (I now use nvAlt), and Simplenote are still around — despite lack of a revenue stream for any of ‘em. Not only are they still around, but it’s still possible to keep notes in plain text or RTF — which is as future proof as computing gets.

They are still around — but not in great health. ResophNotes was last updated in 2012 or so and it is donation ware (always was). Simplenote was purchased by Automattic (WordPress) and is now open source and apparently run as some kind of charity operation [1]. nvAlt is ancient but Brett Terpstra recently updated it to run on Sierra (a notoriously buggy version of macOS).

These apps are old and kind of worn — but so am I. So we’re a good fit. 

Recently I had another set of Microsoft Outlook Notes files to move to Simplenote. Talk about old and kind of worn! Outlook Notes is old, odd, and useful. It’s a winner in a category of one. Functionally it’s a lot like Simplenote — though you can’t print from Outlook [2].

The problem with Notes isn’t that it’s old and odd, it’s that everyone has given up on it. Microsoft tries to make it invisible. Apple dropped support for Notes sync via iTunes/iCloud — though I think Exchange sync may still work. Google ignores them too.

Which is why I needed to again move a data set of out Outlook Notes. I think export to Outlook CSV them import to ResophNotes is still the best bet. From there to Simplenote and from Simplenote to nvAlt, etc.

There’s another way to go though. You can use VB to script export from Outlook to c:\notes:

Sub NotesToText()
    
Set myNote = Application.GetNamespace("MAPI").PickFolder
  
For cnt = 1 To myNote.Items.Count
        
noteName = Replace(Replace(Replace(myNote.Items(cnt).Subject, "/", "-"), "\", "-"), ":", "-")
        
myNote.Items(cnt).SaveAs "c:\notes\" & noteName & ".txt", OlSaveAsType.olTXT
   Next
End Sub

The key thing is this script creates file names with the note title. It’s not a perfect result because the top of each file looks like this:

Modified: Thu 1/12/2017 2:36 PM

accidents and injuries

In this case ‘accidents and injuries’ becomes both the file name and the third line of the note. The “Modified: …” bit is just annoying. I suppose it could be removed using regex and a text editor that can iterate over a set of files … or script the removal.

I imported the plain text notes into nvAlt where they got the title from the file name so it looks something like this:

accidents and injuries

Modified: Thu 1/12/2017 2:36 PM

accidents and injuries

A bit of redundancy in there, and, of course, the Modified string is still around.

Overall this doesn’t work quite as well as the ResophNotes method, but it’s helpful to have options.

- fn -

[1] Automattic recently released a redo of the macOS Simplenote client. I haven’t tried it, but I hope it fixes the perennially broken search of the current client. In any case, Simplenote is not dead yet.

[2] Outlook 2013 broke Notes by essentially removing the list view — I think this might have been fixed in Office 365.

Update 3/13/2017

Speaking of ResophNotes, the current version has an impressive set of import options. Outlook CSV, Toodledo Notes CSV, text files, single file with note separator …

I donated years ago, time to send another donation.

Things I learned connecting my upgraded Comcast modem (Arris TG1682G)

Comcast sent me an email offering a modem upgrade that would enable higher speeds. I figured I’d have to do it sooner or later so I accepted the offer. Higher speeds would fit with Comcast’s “pay to play” network non-neutrality strategy [1].

I got things working, but it was a bit of a cluster. Comcast tried to make this self-serve, but they didn’t quite hit the mark.

I’ll skip the boring details of things that didn’t work and chat advice that was misguided and just list the useful bits I wish I’d known:

1. I replaced a simple, compact wall mounted ARRIS CM820A/CT (no wifi, coax in, ethernet out) with a much larger standing ARRIS TG1682G (download manual from link. Has 4 ethernet, VOIP phone, wifi, xfinity hotspot).
2. You can disable the WiFi and the xfinity hot spot and you can run it as bridge or router. See below.
3. XFINITY support can remotely configure changes, but they can take hours to be implemented. I’m guessing there’s some sort of slow queue.
4. The TG1682G ships with two high quality coax connector cables. My original Comcast installer had trouble with excessive signal strength on the original device and installed an inline damper (resistor?). I reused that cable.
5. You have to activate it before you use it. Even after all lights are green and it seems to be on the Net it’s really not. It’s locked out by Comcast until you connect to the device (wifi or ethernet) and navigate to xfinity.com (if that fails, try http://register.xfinitiy.com). You should see an activation prompt. The ’text code’ method didn’t work (message arrived much later) and the login method failed too. I entered my account number and address. You have to accept a new web services agreement. I don’t want to know what I agreed to. 
6. Once it’s activated go to http://10.0.0.1/ to configure the device using credentials of un=admin and pw=password (yes, “password”). You will be prompted to change your password. I suspect 99% of users don’t because the standard setup skips this step.
7. From the web interface you can disable wifi (I do wifi via Airport Extreme), adjust firewall, and go to bridge mode. For now I let it in router mode (so I have two firewalls - Airport Extreme and Arris). I don’t use P2P so I upped the firewall to standard security
8. To disable the XFINITY “hotspot” service (used to provide roaming wifi services to their customers) you can use the xfinity app on your iPhone or go to https://customer.xfinity.com/#/settings/internet.

I pay $80/month for Performance 25 Internet with Blast Pro; allegedly 200 down and 10 up. To test I connected by Gb ethernet directly to the router and  http://speedtestbeta.xfinity.com gave me 238 down and 12 up. Which is what I pay for, but not faster. Maybe the higher speeds are coming?

 

- fn -

 

[1] Shortly after the start of Crisis-T Comcast capped our network capacity at 1TB/month. The timing might have been coincidence, but maybe they were waiting for the end of net neutrality.  Carriers do bandwidth caps so they can extort funds from media partners (ex: Netflix) who pay to be excluded from the cap. Higher speeds make economics sense then, because they enable more media consumption.

 

Update 3/12/2017

 

A few weeks after I’d installed the Arris a comcast technician came by my home with a new inline filter. According to this gentleman the new Arris modems chat with each other and this was somehow unwanted.

 

Knowing Comcast this all sounds ominous. On the other hand, I don’t see how a dumb filter would block a typical digital signal even over analog cable wiring.

 

After installing this additional filter I had no net service. I had to power cycle the modem and wait about twenty minutes to restore service.

Samsung sells a security cam DVR, but not a TV DVR.

I’m old enough to remember when it was inexpensive and convenient to time shift football games. In the US and Canada that died with the analog to digital conversion. In some other countries there’s push-button record to USB from every TV, but not in the US. (I blame a VW-diesel class conspiracy, we now know those can happen.)

Periodically I look to see what’s sold without a monthly fee. Today Amazon pointed me to a $140 Samsung security cam DVR. There’s no tuner of course, so it can’t be used to record OTA TV.

The time-shifting story seems a small thing, but it convinced me markets don’t work the way people imagine they work. And the world doesn’t work the way I once thought it worked.