Managing sync issues in the Apple Notes app - what I do

I use Apple's Notes app fairly often. It's come a long way from early days but it has its share of bugs. The two I run into most often are formatting bugs and especially sync failures.

Formatting bugs seem to be related to frequent edits to a note using Notes app styles and character formats. Sometimes editing stops working in odd ways. I have to create a new Note and select all/paste into the new Note.

Sync bugs are more common and more serious. There are two kinds of sync bugs - global application sync failure and note specific failure.

Global sync means no notes are synchronizing and perhaps no iCloud content. I've seen that with iCloud password changes. I recommend never changing an iCloud password, Apple really doesn't want it to change. I had to recently because trying to use the Stanza calendar on the Saint Paul Saints web site with Apple integration triggered an instant Apple account lockout and mandated password reset (be afraid).

If iCloud sync doesn't work after a password change and iCloud seems to be properly authenticated first try restarting your iPhone or Mac. Then, on an iPhone, try sending an iCloud email. That last may require you to YET AGAIN enter your iCloud password. With luck the credential problem may resolve for all iCloud functions.

Single note sync failure may be something I see because I'm using Notes across the latest version of iOS and Mojave. I suspect it's actually fairly common even on Monterey/iOS though.

If it's just one note that won't sync between two devices you can check out Apple's web client view of Notes. That will tell you which device is off but it doesn't help with the fix. As far as I can tell there's no sync failure resolution built into Apple Notes. If Apple's sluggish iCloud sync means you have edited one note on two devices when they weren't in full sync then that particular note will no longer sync. No error messages, no offer to sort things out, no creation of two versions, it just quietly sits in isolation.

The workaround is to create a new note and confirm that syncs. Then on each separate device move the content from the now isolated notes into the new syncing note. Then delete the notes that didn't sync on each device.

Synchronization is a problem some seem to have solved (Simplenote for example), but Apple is not among this elite group.

Universal Clipboard (Handoff, Continuity) not working - Mojave and Monterey, regenerating the authentication token/password

 Recently I've had two issues:

1. Mojave Universal Clipboard stopped working between Mojave and iOS 15.4.1
2. Mojave notes iCloud synced but always showed a spinning activity icon

I found that I could make Universal Clipboard work again by creating or editing a note on my iPhone or on Mojave. Once that synced then UC worked until I rebooted.

If Universal Clipboard isn't working first reboot your devices and confirm connection to WiFi with Bluetooth enabled. Then try toggling Handoff off/on on both devices. Then try:

1. Changing my Mojave location DNS settings from Cloudflare (1.1.1.1) to Google (8.8.8.8). (Based on this post)
2. Toggling Notes iCloud off then on again (restores notes)

Now the spinning is gone, and UC seems to be working.

Update 5/15/2022: UC is back to not working unless I create a note that's synced between iOS and Mojave. Notes doesn't show the skinny activity icon. So this is an open problem.

Update 8/30/2023

Continuity has worked since May 2022 on my Mojave machine, but I'm now switching to a 2023 M2 Air running Monterey. And, of course, the Universal Clipboard didn't work again.

A few hours later my devices notified me that iCloud services would not be available until I reauthenticated. That tedious and annoying process requires entering the iCloud and device passwords; I think it creates a new token/password behind the scenes. After going through this continuity worked again. Signing in and out of iCloud is a common workaround for Continuity issues, I suspect it speeds up regeneration of the new token/password.

It would be nice if Apple improved this process.

PS. Typical things to check when it doesn't work: bluetooth on, devices on same WiFi, etc.

ToDo apps: Microsoft's solution

I've used Appigo's ToDo app for about 12 years (with Toodledo at first). It's had problems over the years, but in general it's been a good subscription choice. There's a fairly hard data lock (maybe SQLite?) but manual reentry is feasible albeit annoying.

Lately, however, ToDo has been more ragged. A recent server side change induced a date bug (time zone?) that in turn showed me I was using a macOS app last updated in 2016. It appears to have been abandoned on the Mac App Store. When I went to Twitter I found Appigo's account was closed years ago for violating TOS. Eventually I found I could download a current version of their other App Store app from their web site.

At the moment the app is more or less working again, though parts of the macOS app UI are kind of weird. I figure there was some violent ownership transition with lost dev passwords in Appigo's history (maybe they got ransomwared?).

I decided to go shopping again. I'm looking at:

• Apple Reminders: hard data lock and I have to upgrade from Mojave to get to latest version (not happening).
• Google Todo: this is one hell of a weird product. WTF is their web strategy? Tied to Gmail? Tied to Calendar? At least there's data export.
• Things
• OmniFocus: poor Omni is in some disarray ...
• Microsoft To Do

Today I dug into Microsoft To Do. Of course it's a mess, but this is 2020 so we expect that. The mess starts with Microsoft reusing product names. To simplify a bit:

• There are classic Outlook Tasks. I'll call these TasksClassic. TasksClassic was excellent in many ways, including, once upon a time, great import/export options and lots of view flexibility (I like to sort by last modified!). Unfortunately it's dead, just barely hanging on in the current desktop app with some degree of synchronization with the new product.
• There's the new Wanderlist-based product variable called Microsoft To Do and ... Outlook Tasks (name reuse!). I'll call these TasksW for Wanderlist.

If you open the Help screen page for macOS TasksW (To Do) it takes one to a page on Outlook synchronization that's obsolete -- because the Outlook.com version of Tasks has switched from TasksClassic to TasksW. On the other hand the version of Office 365 on my Mac still has TasksClassic, and it does synchronize with TasksW as displayed on macOS and iOS Microsoft To Do.app.

Are you still with me?

This gave me a brief moment of hope that there was some data freedom here. I remember the import/export options of old Windows Outlook. Alas, the only import/export from macOS Outlook is Microsoft's PST format. There might be some way to do things with Outlook Windows or with 3rd party tools but I don't have the energy for that.

At this time I think TasksW is probably a decent enough product, but this has reminded me how screwed up Microsoft is. So I'm setting this one aside for the moment.

See also:

• There are no great task managers for the iPhone - but there's hope for 2011: My current task list started out on the Palm III and has migrated many times. I probably have uncompleted tasks from the 1990s. I think we're coming to the end of that data migration though. Note from 2010: "Google might finally provide an API for Google Tasks, allowing iOS client development."
• My ancient spreadsheet on migrating from Palm to iPhone - I think I used Outlook Tasks as part of the migration process, or maybe even Microsoft Access. Alas, too long ago. Painful though.

iOS 12.1 Files.app will open Google Drive hosted ePub directly in Books.app

iOS 12.1 Files.app will open Google Drive hosted ePub files directly in Books.app (formerly iBooks). I don’t know how new this is, but tapping on the same file in Google’s Drive.app gives an “unsupported file type” error. (You can still copy it to Books, it’s just awkward.) I’d long used Drive.app to open my ePubs, just happened to try Files.app today.

Books UI doesn’t scale well to significant number of ePubs, storing them in Google Drive or iCloud Drive works much better [1]. I treat iBooks as a temporary store, periodically I clean it out.

[1] Also iOS 12 Books.app won’t sync with Sierra iBooks, so those of us who are putting off painful updates have another reason to store in the file system. Really, though, it’s just way better than using iBooks storage. I’m a bit disappointed Apple hasn’t fully integrated iBook storage with iCloud Files, but this is nice.

iOS 12 Books will not sync to Sierra or High Sierra Books

I updated my iPad to iOS 12. Books gave me this notification:

Changes you make to your library on this device sync only to devices running iOS 12, macOS 10.14, or later.

I downloaded the iOS 12 manual to my iPad and, as promised, it doesn’t show on Sierra iBooks.

Took me only a few minutes to find the first problem with iOS 12.

It’s a gift!

A similar problem happened with Mavericks.

Appigo Todo Cloud.app - don't forget to unsubscribe when you leave

Even since iOS 11 editing in Appigo’s Todo Cloud.app has been buggy for me on iPad and iPhone alike. Just aggravating. Feels like they failed to revamp something. I wasn’t delighted with their sync technology, but I could live with that. The editing bugs finally broke me.

So I decided to exit. Somehow I remembered Todo Cloud is a subscription service. I found my way to the somewhat hidden account settings and disabled premium. Turns out that turns off auto-renewal. 

Hope it really works.

Appigo is a textbook example of how subscription solutions can disappoint. They never provided a good export strategy, so there’s a strong data lock. Then they failed to do minimal maintenance but continued to collect subscription revenue and sell the app.

So what will the replacement be? I’d like a product that

1. Did what Appigo ToDo Cloud did but actually worked
2. Had a web client as well as Mac, iPad and iPhone client
3. Supported family sharing
4. Had good data export (exit strategy).

I evaluated Things.app and OmniFocus. Things got #1 and 3. OmniFocus got #1 and 4. Neither got #2.

Hmm.

On the other hand, Reminders.app for iOS got #2 and #3 and it’s free. So it’s weirdly in contention.

For now I’m using Reminders.app for tasks and Trello for projects. I manually copying over tasks that had dates, it’s not too bad. The backlog of ‘someday’ tasks I’ll gradually slog away at.

If Things gets some data export I’ll probably buy it, but it’s expensive since it’s not a universal app. If OmniFocus gets family sharing I might buy it. Meanwhile I’ll see what I can make Reminders do.

I think this is my first significant iOS functional regression.

Update 5/22/2018

I just discovered I wrote about this in 2011 …

There are no great task managers for the iPhone - but there's hope for 2011

… Neither Things, nor Appigo’s ToDo.app (which I have used incessantly since 2008), nor OmniFocus, nor Remember the Milk.app nor Toodledo.app are a great solution. They all fall short…

Seven years later and ToDo.app is moribund (I’ve been using it for 10 years!) and both OmniFocus and Things are still flawed.

Since I first wrote this I’ve run into issues with Reminders.app — including sync bugs and even text editor bugs. On the other hand, OmniFocus is promising a web client. I’m going to transiently switch back to ToDo.app and see if WWDC providers some kind of family sharing for subscriptions. That would make OmniFocus pricing less extreme.

MarsEdit - don't enable all post sync if you have a very large number of posts

MarsEdit 4 added a new feature — the ability to sync all posts. It didn’t work for my Blogger posts, somewhere around post 3,000-4,000 Google Blogger dies. It looks like a Google bug.

It did work for kateva.org/sh though — that has over 30,000 posts (it mirrors my pinboard shares and twitter tweets). Unfortunately that slowed MarsEdit launch severely — it took up to 3-4 minutes to start. Turns out MarsEdit loads all posts into memory and it doesn’t scale to that size.

I reset the kateva.org/sh sync to ‘last 50’ and startup is ok now.

iTunes Mac to iPad photo sync is broken

I had a longish holiday break this year, so naturally I spent part of it fighting with another technology regression. 

This regression was about getting photos from my Mac to my iPad; I use the iPad as a digital photo frame randomly displaying my Aperture (running on Sierra) images tagged as “slideshow” (this drains the battery fairly quickly, so I usually have the iPad charging). Years ago this was painless, but over the last few months I’ve found that only a fraction of my tagged photos make it over to the iPad. Often the process seems to die.

This go round I tried an old Aperture tip to delete the iPod Photo Cache (Apple approved) [1]. That had one effect — image transfer died completely! Instead of telling iTunes to transfer all my Slideshow tagged images (9,019) I had to select year-specific smart album slideshows one at a time (incremental add):

That generally worked, though the ghostly counts shown above in the iTunes 12.7 UI came and went inexplicably. Sometimes a mouseover seemed to trigger an appearance. If they didn’t show up that was a clue that something had gone wrong with the sync. (Infamously iTunes does not log errors to Console. Perhaps because they’d overwhelm console?)

Some part of the OS (or Aperture?) has to regenerate the .ithmb files [2] that are shared, that seems to be a slow process. I think the build process might make use of some data Aperture saves on exiting; that data is also used by the old OS X media browser (itself related to Apple’s old iLife code) [3]. I’m pretty sure some of the .ithmb creation is by an OS thread thought, and that seems to have a low priority. Maybe part of what causes a complete failure with large numbers of files is that iTunes has an arbitrary timeout placed to work around some other iTunes bug [4].

Once I broke down my slideshow into smaller bits and added them incrementally I seemed to have some success. The count (erratically) displayed in iTunes matched what showed on the iPad album counts and the Aperture smart album counts. Then things went south. The numbers transferred started to be a fraction of the Aperture counts. When things settled out I seemed to be stuck at 7,811 on the iPad (same count in iTunes if I mouseover the iTunes content type distribution bar) out of 9,109 in Aperture [5].

I did some half-hearted testing to try to figure out if this was related to filename characters or the like, but I haven’t gotten very far. I did confirm that RAW files don’t have an obvious problem.

The great news is that I learned today that I’m not alone and it’s not an Aperture specific problem! There’s a Macintouch thread on this by riley (quotes others, emphases mine):

So here's an update on the issue with syncing large Photos databases between iOS devices and iTunes.

After posting a number of followups to my Apple bug report, the response I got back from Engineering was that this is a known limitation in iOS, and "we are working to improve this in a future iOS release."

Their suggestion was to sync by starting with just a few albums, and then repeatedly re-sync, adding a few more albums each time. "On subsequent syncs, iOS will not add additional photos unless the sync preferences in iTunes are different from the previous sync, which is why it is necessary to keep adding selected albums each sync, instead of all at once."

… There is a huge discussion of this over on the Apple forums (see “Photos app on iPhone not syncing in iOS 11 ").

Various people have been told various things. Some have been told it is an iOS issue, some an iTunes issue. Personally, I think there are issues in both that are interacting, which are preventing reliable syncs with large photo albums over USB between iTunes and iOS devices.

Two years ago, our then ~40k photo library in Photos would sync the selected items (maybe around 30k photos in various albums) without problem. Perhaps around the time of iOS 10, the syncs needed to be restarted to complete, but eventually it would work. Since iOS 11, some sync, but many do not, and the sync just cancels with a variety of error messages at various points. Sometimes it is a device error, sometimes nothing. Sometimes many of the photos will be on there, but the albums they should be associated with are empty…

… The last update I heard from the person who was helping me was that it was an iTunes issue (at minimum, perhaps iOS too). She said that the Apple engineering team is aware of the issue and working on finding the problem so it can be fixed.

… I also see that since the latest tvOS update and the latest iOS 11 update, many albums are no longer being shared with the Apple TVs.

Hmm. This suggests to force updates one should add or remove some album with every sync. That did not work quite as expected. After I tried using the Media Finder code [3] and the ‘add new album’ trick iTunes said it was synchronizing another 1000+ files (that’s good!) but then the counts in iTunes and iPad dropped from 7,811 to 7303. So that’s just great.[6]

Since I now know this is not just an (abandoned) Aperture problem I added an iTunes feedback comment about the problem. Rumor is Apple processes these, presumably doing some algorithmic text extraction based on volume. So add your voice if you’re interested.

- fn -

[1] It took hours to empty the trash after deleting iPod Photo Cache — despite files being on an SSD. As of 7/2020 the links are all broken, but this is what you do: "find your Aperture or iPhoto library in the Finder. Right-click and select “Show Package Contents” then find “iPod Photo Cache” and delete it."
[2] I changed extension of a copy to .jpg and it rendered as expected. They are fairly high res images, in a small sample I examined one was a 3MB jpg. 
[3] In Sierra’s Open dialog the Media Finder still shows in the left nav pane! Maybe invoking it might trigger some updates? Worth a try to open it and let it run for a while.
[4] It’s widely assumed that iTunes source code is an impenetrable mess with bits left over from pre-Apple days. 
[5] Incidentally “Stacks” are a problem. One of the many bad things Apple did in bridging iPhoto and Aperture was to treat an old iPhoto Edit as an Aperture Stack. This threw away key information — that one image was the new version of another. There are lots of downstream implications. Years later I’m recreating thousands of iPhoto edits in Aperture. One of the problem is that when you tag an image as “slideshow” then transfer to iPad both versions go. I think this can cause the iPad count to be higher than the Aperture count, but of course the problem I haves is in the other direction.
[6] I removed the extra album and then the count went UP to 7,811 again! That album did reference photos that were in other albums, so maybe part of the bug has to do with photos that are in more than one album. I do think there are multiple bugs here though.

Update 12/31/2017 I

There are some JPEG images in my older files that do not show up in either OS X Media Browser (old iLife code) or Sierra’s Screen Saver Library browser and do not transfer to the iPad via iTunes. However, if I export the original and drop it back in again then the image appears normally — this doesn’t appear to be problem with the JPEG. They also render normally in several different apps and EXIF viewers.

Rebuilding the Aperture database does not make a difference.

So this is curious, and potentially an important clue, but to go further I need to understand how Media Browser works.  In my case the bug doesn’t only impact image transfer to my iPhone; it impacts Media Browser as well. I’m now poking around at ~/Library/Application Support/iLifeAssetManagement and ~/Library/Application Support/iLifeMediaBrowser, especially ~/Library/Application Support/iLifeAssetManagement/state/albumshare/Daemon.sqlite.

Update 12/31/2017 II

Finding more “invisibles” but no clear pattern to what’s omitted. Some are quite small - from the days when 100K was a good image size. It’s not size alone though; of a pair of similar old images a 64kb one made it across and a 70k one didn’t.

Update 1/1/2018

Same sync, same everything — but now iPad acts as though it’s been syncing to a different computer. All images removed! It took several tries to get one album to sync, but after it started working I could do about  @1,200 at a time. Now have about 7,700 on iPad out of 9,100 on Mac. I won’t sync iPad to iTunes again until there’s word of a bug fix. Just stick with what’s on there now.

Update 6/9/2018

I thought this was fixed — but it’s broken again. On latest versions of iTunes and iOS 11.3.1, macOS Sierra.

Update 6/9/2018b

I was able to sync 7777 of 7771 (different apps count photos differently, not least due to Aperture’s brain-dead “Stacks” handling of iPhoto imports) photos after installing iOS 4. I don’t know if the bug is fixed or if it’s just that process of installing an iOS update cleared something out. In the latter case it might return.

Update 7/4/2020

It's been a year or two since I last tried syncing. At this time I'm still on Aperture and Mojave. The sync was worse than ever. I wiped my old iPad and restored from iCloud backup and sync actually seemed to work! It saw only 7992 of the 9413 images in my Aperture slide show, but it did import those.

I tried the trick of deleting the iPad Cache in Aperture [1] but on resync it was still 7992/9413. Consistent at least. 

I think there's a bug in Aperture with Sierra and later with generating the images in the iPod Cache. Maybe something iTunes or OS X was supposed to do.

 I suspect the iPad flash memory is also in bad shape. 

The lack of logging or error handling is appalling.

Simplenote for Mac search has been fixed

Two years after Simplenote.app for Mac was released with broken search it is now fixed in version 1.2. A product I rely on is not only alive, it’s even improved.

This was a significant rewrite and there is still lots of basic work to do. Tags don’t autocomplete (I think they used to in the prior release and they do on iOS). You still can’t apply tag changes to a group of notes. There’s no export of notes as text files from App. There’s no markdown support (not a priority for me but iOS version has it) and of course no support for images. 

I haven’t used it long enough to spot any sync errors — recent versions of Simplenote were very reliable.

I hope this is a sign that Simplenote has a future. Now if they would only figure out a way for me to pay for it…

Podcasts.app - sync Stations between iTunes and iPhone via iCloud, not via iTunes sync

Apple removed playlists from iOS 11 podcasts.app. I loved playlists. They were a beautiful user friendly query building tool.

Instead we have Stations, which are more limited. I created some in iTunes and some on my iPhone. They didn’t sync even though I transfer podcasts from iTunes to iPhone via WiFi/Lightning cable.

It turns out Stations do sync — but only via iCloud even though files may travel via local sync.

Once I enabled podcast sync in both iTunes and iOS my stations appeared in both. 

This is good, because one advantage of stations is that podcasts play consecutively. At least I think that’s why mine started playing consecutively instead of stopping after each podcast as they did earlier in iOS 11.

There are lots of weirdnesses in iTunes when it tries to combine sync via iCloud/Store with local file based sync from iTunes. It will be interesting to see if Podcasts does better at this than Movies does.

Apple is clearly heading towards the world where we have all have 2TB iCloud Drives, a 256GB local machine store for frequently used data, and special folders in iCloud for personally owned files that sync separately from Store files (ex: personal images, ringtones, podcasts, PDF, etc). There’s no resisting this one.

Update: It is a mess. Of course. Stations need a setting to only show downloaded episodes. One problem with Stations is that while they can filter out Played stations, they only know a station is Played as long as the file is around. If the file is removed and the show is in the Cloud then it is treated as unPlayed.

Progress of sorts - integrating my simplenote memory extensions with gDrive

Nine years ago I wrote What my blogs are for: memory management and the Google-Gordon geek-mind fusion. It’s hard to believe now, but in those days we did not yet fear Google. Back then, we liked Google. Google was good.

That was a long time ago. Google has since transcended mere human morality (Galactus is a much better name than Alphabet btw). Unfortunately I haven’t transcended my need for artificial extensions to my working memory. To the contrary, that need has grown. Now in addition to my old technical expertise a change in employment means I manage more organizational and political knowledge. Oh, yeah, and I’m back to being a practicing doctor. So now I need to know adult male ambulatory internal medicine. Also, I am old and my brain is crud.

So, yeah, I need those memory extensions. I still use Google’s tools, though the Custom Search Engine I rely on to integrate my blogs with my link share stream and my ancient (hidden) web pages is all but forgotten. Since 2008 though I’ve moved away from the Google dominated web back to files and file systems as a memory store. Carrying a connected computer in my pocket made 1980s style files useful again. I rely in particular on plain text Simplenote to carry around fragments of memory. For regulatory/security reasons I have one Simplenote account for work memory and one for non-work memory — with some notes shared between both using Simplenote’s (simple!) sharing mechanism.

Spotlight indexes simplenote files on my iPhone — sadly only for one account. On my Mac nvAlt syncs to my personal simplenote account — so the text files are indexed by Spotlight there too. Today, guided by a @simplenoteapp tweet I installed Simplenote Electron (which Automattic is abandoning for a future fully native Mac upgrade [1]) and tied that one to my work account. Then I used the improved text export to create a local read-only copy of that memory store — which Spotlight Mac now indexes. So the stores are in one search space at last.

The last twist is that Simplenote local text stores are in a Google Drive folder, so synchronized to gDrive with appropriately stored folders such that appropriate text stores are searchable on both devices. Sadly gDrive does not support “Available offline” at the folder level — so when I don’t have data access I also lose some memory access. (I’m hoping Google will fix this.)

The gDrive integration is important because there are limits to plain text - particularly for medical topics. I have PDFs, Word Docs, Google Sheets and other detritus of decades of notes. I can put all of that in gDrive alongside the text notes and let search figure it out. (This is what Evernote is supposed to be good for but they don’t have a clear exit strategy. It’s trivial to walk away from Simplenote. I need that exit strategy [2].)

It’s all one heck of a kludge, but so is my memory. So kind of fits.

See also:

• What my blogs are for: memory management and the Google-Gordon geek-mind fusion 7/2008
• Snippets Explained (Version 2) 2/1999
• Beyond Simplenote: I still want a graph layer (concept map) overlay for my memory augmentation notes collection 5/2016. Sigh. This would be wonderful.
• Project Memfail: Tackling my search space problem 9/2013. My work space problems got sorted out when the company died. I’m much more cautious now about creating new memory pockets.
• Thinking tools 2014 - holding steady but future unclear 5/2014. RIP App.net.
• Full text search and digital prostheses: new email, new mind 7/2008
• Strange loops: Google custom and customized search - and a memory blog 3/2008. Non-evil Google.

- fn -

[1] It has the same search bug as Simplenote/Mac!

[2] For example - if they don’t fix that damned search bug [1].

 

Photos.app flailing during sync with iCloud Photo Library? Maybe it's a permissions problem.

My daughter uses Photos.app and a 50 GB iCloud Photo Library to manage her videos and images [1]. She edits on an older Air with a small SSD, in that environment Photos.app caches scaled res images and only downloads full res when editing.

In addition I run an instance of Photos.app for her that stores full res images. The Photos.app Library is stored on an external SSD that hangs off an Elgato T2 Hub attached to my beloved Air. The hub has been very reliable under El Cap and Sierra.

I have a user account for her on my drive, and in that account the external library is the Photos.app System Library. My Time Machine [3] and Carbon Copy [4] backups include that Library.

All was well under El Capitan. A few months ago I upgraded to Sierra [2]. Yesterday I decided to update her Photos.app library — only to discover I was a few months behind [5]. Her user account hadn’t been updated to Sierra; when I opened Photos.app her Library had to be updated.

Things did not go well. Photos.app said it was uploading @8,000 images (really it shouldn’t have uploaded anything, but Photos.app sucks), then @2,000, then @11, then … You get the idea. It did that when I went to bed, and it was doing it in the morning.

After a bit of playing around I discovered that a Sierra bug meant that she no longer had write permissions to the external SSD, even though macOS said she did. I switched to an admin account and there she had no permissions, so I added her. After that she could write to the SSD. Photos.app “stuck upload” was because it had no write permissions at all.

I decided to create a fresh Photos.app Library for her. To do that I turned off WiFi and did option-Photos.app startup to create a new Library. I copied the old Library to an external drive and deleted it. I then opened the new Library, made it the System Library (interestingly it showed images from a cache!), turned on WiFi and enabled iCloud Photo Library. The images then downloaded from iCloud (source of truth) and restored my local backup copy.

- fn -

[1] She is chronically running against the limit — which isn’t all bad. It enforces some editing. I might switch to sharing a 200GB plan, but I’m not sure how that will work with our current family use of a single iTunes password. Future experiment needed.

[2] I like to wait at least 8 months before accepting Apple’s dangerously buggy macOS updates.

[3] Our two Airs do Time Machine backups up to a Synology NAS. After some initial issues that has been utterly trouble-free. The NAS has two RAID 1 drives, if one fails the other survives. This is another reason I wait for macOS bugs to get fixed; I also need things like VMs and NAS to be updated.

[4] CCC backups to a 4TB low heat drive in a Voyager cradle with Firewire 800 connection to the Elgato hub. I rotate 4 drives. Rotation is every 2-3 weeks, 1 drive is across town, the other in my Van. A Yellowstone eruption would take them all out unless the van outran the pyroclastic flow. It is a shame that offsite internet backup has failed.

[5] Only automated backup ever works — and no form of backup is reliable.

Apple drives me into Google's arms - using Google Photos with iPhone and Aperture

Apple has broken me. I’ve left iCloud Photo Stream shares for Google Photos.

First I lost the ability to share from Aperture to Facebook. I think that was probably a Facebook change, but of course Aperture isn’t getting updates any more.

That was annoying.

Losing Apple Photo Stream was much worse. Photo stream wasn’t great, but it was simple for my daughter, sister, and other users to subscribe to. For a time I could use iCloud Photo Library on Photos.app alongside iCloud Photo Streams on Aperture [1].

Then Aperture retched and I lost my shared photo streams (but not, happily, the originals). I played around with restoring iLifeAssetManagement from backup but, despite early promise, I couldn’t defeat Apple’s black box sync infrastructure [2].

That’s it. I’m toast. I surrender. Google’s inexplicable aversion to album creation on upload is the lesser evil now.

I’ve installed Google Photos on my iPhone and enabled backup and sync. I’ll use that to cull and play with photos before I transfer them to Aperture.

I’ve freed up 14GB from my Air’s SSD by deleting iLifeAssetManagement and I’ve installed Google Photos Uploader.app. I pointed that to a folder on an external drive, when I want to share from Aperture I export there for upload. I do my post-upload organization and sharing through the web UI.

Since Google nicely migrated images when it closed Picasa Web Albums my new shares are reunited with my old Picasa web albums. I’ve come home again. Though I’m still puzzled by Google’s weird album aversion.

It’s far from ideal, but Apple has burned me yet again. They seem to despise my data.

[1] Though I gave up on iCloud Photo Library when I realized it was more or less incompatible with importing images from iPhone photo roll to Aperture.

[2] Apple is famous for sync that disallows any kind of troubleshooting.

iLifeAssetManagement

Partial restoration of lost Apple iCloud photo stream shared albums (updated: didn't work)

Something went wrong. It always does.

I had thousands of images distributed across over 60 shared photo streams. One day I rebuilt Aperture’s database and all the iCloud images were in one recovery folder. I deleted them and then most of my iCloud shared albums vanished.

This is a quick summary of how I recovered most of them from backups. I don’t know how this truly works, but it seems that this folder in my user account was a source of truth for iCloud photo streams:

/Users/[my user name]/Library/Application Support/iLifeAssetManagement

I copied what was there to an external drive then deleted it, logged out (necessary to close open databases) then logged in. With Wifi on when I launched Aperture it showed no images at first then downloaded what was in iCloud. So there was some kind of sync.

Next I did the same thing (closed Aperture, deleted, etc) but this time copied a backup of iLifeAssetManagement from prior to the bad event. I then turned off wifi.

On relaunch Aperture showed about 6100 images in “Shared:iCloud”. It rebuilt thumbnails for them. Then I turned on Wifi. Next I saw the count rise briefly as albums I’d shared previously came down from iCloud. Alas, the count started falling again, stabilizing at 5600.

I had most of my streams back — though one stream was much smaller than it used to be. Still, about 80% recovery and I didn’t lose a few I’d done post-disaster.

Better than nothing.

Sync without controls is truly hell (and Apple never provides enough control).

Update: Aperture shows 56 single owner photo streams (one is empty) and 5 shared. iOS Photos.app shows 20. At least one of the iCloud albums not seen in iOS photos.app cannot be found at its public link. The iCloud library and the Aperture iCloud library are not in sync. So I’d call this a failure.

Migrate Contacts from Outlook/Exchange server to OS X Contacts

I last wrote about migrating contacts from Outlook/Exchange server to OS X in 2011.

Back then one of the issues with migration then was that Exchange uses X400 format email addresses. One way to make Outlook convert to internet is to drag contacts into an email — that produces .vcf files (undocumented) with internet email addresses. In 2011 I wrote that dragging into a local PST did the same thing. Not sure that still works.

Here’s what I did recently …

1. Drag and drop from Outlook/Exchange into email (undocumented export)
2. From email save to desktop.
3. Drag and drop the .vcf files into OS X Contacts (undocumented import.

From OS X contacts they can go to iCloud as usual.

Incidentally, i couldn’t see any way to readily import a set of single contact .vcf files into Outlook 2016. It only wanted to import them one a time. Drag and drop of the files into Outlook resulted simply in Outlook opening each one until Windows 10 die.

Escape from Outlook Notes - export as text files, import into NvAlt or other

Back in 2010 I wrote Gordon’s Tech: Escape from Outlook Notes - ResophNotes, Simplenote for iPhone and Notational Velocity.

Seven years later ResophNotes, Notation Velocity (I now use nvAlt), and Simplenote are still around — despite lack of a revenue stream for any of ‘em. Not only are they still around, but it’s still possible to keep notes in plain text or RTF — which is as future proof as computing gets.

They are still around — but not in great health. ResophNotes was last updated in 2012 or so and it is donation ware (always was). Simplenote was purchased by Automattic (WordPress) and is now open source and apparently run as some kind of charity operation [1]. nvAlt is ancient but Brett Terpstra recently updated it to run on Sierra (a notoriously buggy version of macOS).

These apps are old and kind of worn — but so am I. So we’re a good fit. 

Recently I had another set of Microsoft Outlook Notes files to move to Simplenote. Talk about old and kind of worn! Outlook Notes is old, odd, and useful. It’s a winner in a category of one. Functionally it’s a lot like Simplenote — though you can’t print from Outlook [2].

The problem with Notes isn’t that it’s old and odd, it’s that everyone has given up on it. Microsoft tries to make it invisible. Apple dropped support for Notes sync via iTunes/iCloud — though I think Exchange sync may still work. Google ignores them too.

Which is why I needed to again move a data set of out Outlook Notes. I think export to Outlook CSV them import to ResophNotes is still the best bet. From there to Simplenote and from Simplenote to nvAlt, etc.

There’s another way to go though. You can use VB to script export from Outlook to c:\notes:

Sub NotesToText()
    
Set myNote = Application.GetNamespace("MAPI").PickFolder
  
For cnt = 1 To myNote.Items.Count
        
noteName = Replace(Replace(Replace(myNote.Items(cnt).Subject, "/", "-"), "\", "-"), ":", "-")
        
myNote.Items(cnt).SaveAs "c:\notes\" & noteName & ".txt", OlSaveAsType.olTXT
   Next
End Sub

The key thing is this script creates file names with the note title. It’s not a perfect result because the top of each file looks like this:

Modified: Thu 1/12/2017 2:36 PM

accidents and injuries

In this case ‘accidents and injuries’ becomes both the file name and the third line of the note. The “Modified: …” bit is just annoying. I suppose it could be removed using regex and a text editor that can iterate over a set of files … or script the removal.

I imported the plain text notes into nvAlt where they got the title from the file name so it looks something like this:

accidents and injuries

Modified: Thu 1/12/2017 2:36 PM

accidents and injuries

A bit of redundancy in there, and, of course, the Modified string is still around.

Overall this doesn’t work quite as well as the ResophNotes method, but it’s helpful to have options.

- fn -

[1] Automattic recently released a redo of the macOS Simplenote client. I haven’t tried it, but I hope it fixes the perennially broken search of the current client. In any case, Simplenote is not dead yet.

[2] Outlook 2013 broke Notes by essentially removing the list view — I think this might have been fixed in Office 365.

Update 3/13/2017

Speaking of ResophNotes, the current version has an impressive set of import options. Outlook CSV, Toodledo Notes CSV, text files, single file with note separator …

I donated years ago, time to send another donation.

How to backup your iCloud Notes.app data on a Mac

Apple has a support doc on backing up iCloud data with a less than ideal recommendation for macOS Notes.app backup …

Archive or make copies of your iCloud data - Apple Support

To copy notes, open the Notes app at iCloud.com. Copy the text of each note and paste it into a document on your computer, such as a Pages or TextEdit document. Save the document to your computer. To export your notes as PDF, open the Notes app in OS X Mountain Lion or later. Select the note, then click File > Export as PDF and choose a location.

Maybe Automator could make this scale, but I’ve never had much luck with Automator.

There’s a better approach that, oddly, Apple used to recommend …

How do you backup Notes? | Official Apple Support Communities

Open Notes.
Select View > Show Folders.
Create a new folder called Notes Backup in the On My Mac section of your folders list.
Select one or more notes from your All iCloud folder. Holding the Option key down, drag the notes into the Notes Backup folder. A green plus icon should appear as you're dragging the Notes to the new folder. This creates a copy of your iCloud Notes on your computer.
If you have an iCloud section in your folders list, but not an On My Mac section, you will need to create one.
Quit Notes.
In System Preferences, select iCloud. Deselect Notes.
Open Notes. Select File > New Folder. Name your folder Notes Backup. Create a new note in that folder as a placeholder. Quit Notes.
In System Preferences, select iCloud. Select Notes.
Open Notes. You should now see a section for iCloud and a section for On My Mac. Follow the instructions above for making a local backup of your iCloud Notes.

For this to work you need to enable the “On my Mac Account”. That may be disabled on most Macs and I suspect Apple would like to get rid of it. Which is perhaps why it’s no longer part of the backup support doc. In any case this does work on El Capitan, it does scale, and it enables restore.

If you ask Google how to backup Notes.app, the AI presents this as a “pre-answer” above all web page results. Google’s AI does better than Apple’s support documentation team.

Apple needs a better backup/restore solutions for all of its iCloud data, especially Notes.app, but for now this helps.

Tweak to my Photos.app, Aperture workflow.

WARNING: There are at least two issues with this workflow.

1. Not all images that appear in iOS UI for “All Photos” are actually available for import into Aperture. I suspect images that have synchronized from iCloud Photo Library are stored in a different physical location from the images Aperture sees.

2. Neither Aperture nor Image Capture can delete photos from iPhone if iCloud Photo Library is active. Aperture doesn’t show an error message but fails. Image Capture doesn’t show the delete control. Turn off iCloud Photo Library to reenable delete.

———- ORIGINAL POST ——————-

With El Capitan it is possible to have Photos.app and iPhone share photos via iCloud, while Aperture owns posting to iCloud photo sharing streams. You have to fight Photos.app a bit, but it works great.

So my workflow has been to use Photos.app on OS X and iOS to cull images, then periodically export from Photos.app to Aperture. Except Photos.app original image export is broken, and has been broken since 1.0. (Exports all images, even those that are “deleted”), so this is a pain.

Finally realized there’s a much better solution. Whenever I want to move images from Photos.app to Aperture I import them from the iPhone into Aperture (iPhone set to keep full res images). Then I use Aperture post-import delete. Then they delete from Photos.app. Much neater.

See also:

• Living within iCloud Photo Library’s storage budget 6/2016
• iCloud Photo Library, Photos.app and my Aperture plan (plus Aperture Exporter)

Modern bugs: server, client, and DRM. Also transient.

Another reason why we can’t have nice things.

My son's iPhone had access to some of our cloud media (TV) library, but not all of it. The UI looked like a mixture of age restricted on-device media, downloadable off-device media, on-device media missing normal display icons, and off-device media that wasn’t shown at all.

iTunes sync displayed odd alignments; it saw on-device media that did not show in iOS manage storage. His device requested App Store credentials multiple times, usually displaying the iOS 1 style dialogs seen when things are broken in iOS 9’s hacked together authentication frameworks.

An hour or so later things were more or less back to normal. Signing out of the App Store completely then re-authenticating might have helped. Or perhaps an old school iTunes sync or two. Or maybe an asteroid fly-by.

iOS flash style corruption? (There’s probably a reason Apple wants to retire HFS+ for something that’s Flash friendlier.) Problems with Apple’s DRM servers? Bugs in iOS? Bugs in iTunes?

Most likely all of the above. Sooner or later I’ll have to wipe and restore this phone, the iPhone equivalent of a visit to the dental hygienist. Reminds of me of Windows 95 really.

Modern bugs: emergent, complex, transient, common, and disruptive.

Uninstalling Synology Cloud Station Mac -- and why you would never have installed it in the first place.

Synology Cloud Station aka Cloud Station Drive stopped working reliably for me in El Capitan.

That’s when I discovered there’s no uninstaller.

Damn.

I have a rule of not installing Mac apps that can’t be easily uninstalled [1]. Clearly I screwed up. In my experience if a Mac apps doesn’t have a clean uninstall it’s not going to work out well.

Uninstall directions for Ubuntu mention a buried uninstall script in ./Cloudstation; I have Finder set to show invisible files (I like running this way) so I found the Mac version of this in /Users/jfaughnan/.CloudStation (you might be able to open this using Finder Go). I couldn’t find an uninstall script in the Mac package though. I didn’t find anything useful on Synology forums; a site that markets a (worthless) Mac uninstaller app gave generic directions.

This is what I did. I think EasyFind will do most of the work for you - just search on Synology.

1. Exited Synology Cloud Station from the blue cloud icon Finder bar thing.
2. Searched Activity Monitor for Synology and Cloud. Found two processes called “synology cloud station drive finder integration” and killed both.
3. Deleted /Applications/Synology Cloud Station.app
4. Deleted /Users/jfaughnan/.CloudStation
5. Used EasyFind.app (love it) to search for all files and folders containing “synology” including Invisible Files and Folders and Package Contents. It found a ton of crap. For example: /private/var/folders/5c/gx565vx110gcdhxn10mbddfh0000gp/T/com.synology.CloudStation.FinderSync [2][3]. I deleted that and several more. I’ve included below a set of partial screenshots form Easy Find (shame I don’t know enough unix to do this in shells and dump results). It’s a mess, but in practice I only had to delete 3-4 things to clear them all out. (If you have other Synology tools installed you’ll probably damage them too, but really you shouldn’t let Synology touch your Mac.)
6. Log out and then in again to confirm Finder iteration is gone.

Ugh. What a mistake that was.

 

Examples from EasyFind (many of these are package contents, so only need to find Package or Folder and delete the lots)

 

- fn -

[1] Lack of OS uninstall support is one of the historic defects of macOS / OS X.

[2] cmd-opt-c to copy file path is the best feature of El Capitan. Almost as good as old cmd-opt-shift-v for paste as plain text.

[3] There are lots of weird files in ‘5c’, looks like some kind of cache/dumping ground. I just deleted the Synology one. There’s also a com.microsoft.OneDrive-mac.FinderSync in there. Turns out Finder Sync is a newish OS X API that does a lot of interesting things (probably with bugs and Finder screw-ups, knowing Apple):

… The Finder Sync extension point lets you cleanly and safely modify the Finder’s user interface to express file synchronization status and control. Unlike most extension points, Finder Sync does not add features to a host app. Instead, it lets you modify the behavior of the Finder itself….

… With a Finder Sync extension you register one or more folders for the system to monitor. Your Finder Sync extension then sets badges, labels, and contextual menus for any items in the monitored folders. You can also use the extension point’s API to add a toolbar button to the Finder window or a sidebar icon for the monitored folder…

… Finder Sync supports apps that synchronize the contents of a local folder with a remote data source. It improves user experience by providing immediate visual feedback directly in the Finder. Badges display the sync state of each item, and contextual menus let users manage folder contents. Custom toolbar buttons can invoke global actions, such as opening a monitored folder or forcing a sync operation…

…  the user may have multiple copies of your extension running at once, and some may be very short lived…

Update 8/23/2016: I can’t kill the Finder integration on Emily’s Mac. Keeps returning. This thing must have malware in its code base.