macOS OneDrive, ScanSnap PDFs and the "could not be opened" error in Monterey

When I migrated from Mojave to Monterey I ran into the typical array of macOS upgrade issues -- including having to reinstall Monterey. There's a reason I dread updating macOS.

One of the issues was that OneDrive didn't seem to work with my ScanSnap PDF uploads. In this case there were two suspects - the Monterey update and a OneDrive update. (One of the reasons I upgraded after migrating off Aperture was that OneDrive was no longer supported.)

The iOS ScanSnap client seemed to work as before, and the PDF appeared in the OneDrive folder I used -- but the file could not be displayed by Preview. I got a "could not be opened ... It may be damaged or use a file format that Preview doesn’t recognize."

It took a few searches to find the answer ...

If you were previously navigating to useraccount/OneDrive/DocumentFolder and opening your files from there, that seems to have stopped working now, and you have to go to Locations/OneDrive/DocumentFolder instead.

I'd had a Favorites link to the OneDrive folder that held my scans prior to the update. When I study where that link goes now it's not to the old file system folder, it's to:

/Users/jgordon/Library/Group Containers/UBF8T346G9.OneDriveSyncClientSuite/OneDrive.noindex/OneDrive/ScanSnap

So the folder that used to be in the file system was now buried in Library but the Favorite somehow resolved to it still.

I created a shortcut to a folder of the same name as displayed in Locations/OneDrive and the path there is

/Users/jgordon/Library/CloudStorage/OneDrive-Personal/ScanSnap

Digging into OneDrive preferences it claims my location is "/Users/jgordon/Documents/One..." (yep, truncated path). This is the path OneDrive used to use, but now there's just a Favorite there.  The true path is ... yep ...

/Users/jgordon/Library/CloudStorage/OneDrive-Personal/ScanSnap 

Even though none of the files are in a location that I expect Spotlight to index it does appear to index the files stored there once I revised settings so all files were downloaded. Once I did that however the file was now readable even in the location my old Favorite resolved to: 

/Users/jgordon/Library/Group Containers/UBF8T346G9.OneDriveSyncClientSuite/OneDrive.noindex/OneDrive/ScanSnap

 So this is kind of what I think was going on to cause this particular time wasting problem

1. Apple made everyone switch to their preferred approach to managing Cloud files.
2. The folders that were once in the local file system were gone, but an old Favorite somehow resolved to a similar folder buried in a virtual file system. The file, however, was no longer resident locally, it only seemed to be available if one inspected the virtual folder with Finder. Preview could not access it because it wasn't there, and in Monterey Preview gives a misleading error message.
3. When I used the Locations OneDrive "folder" to navigate I went to a different Library CloudStorage folder where OneDrive will auto-download folders on demand. If, however, full download is active (as it was previously so I can backup but that's not the default) then even the internal system OneDrive uses has a full copy and Preview will open it.

Apple wants all Document folders to be stored in the Cloud and may eventually want all user folders in the Cloud, so part of this is probably to prevent different Cloud Providers from cross-synching folders.

I think the bug hits those very few people who had a Favorite to an old style OneDrive folder prior to upgrading OneDrive. Although these kinds of complex emergent bugs don't hit many people, there are thousands of bugs like this so sooner or later we run into them. Which is why it's now very hard for non-geeks to use a personal computer.

Fixing Eero performance issues - try removing a device

We bought a 3 unit Eero 6 mesh WiFi router. From early days we ran into unreliable video problems. Remote users would say our bandwidth was flagged as poor -- despite having a GB fiber connection and, you know, the Eero routers.

I never found any useful diagnostic or testing information, but the fix was to unplug one of the 3 Eero stations. With only 2 devices the video quality was much better. Presumably we have too many devices for the size of our home -- and the software for managing handoffs might not work as well as one would expect.

Google Apps (Workspace) email failing? How to check DKIM and update in your DNS settings if needed.

When I travel every bit of IT in our family starts to malfunction. On a recent trip this included Emily's emails, they were intermittently rejected by Gmail recipients. Once I was home I had to dig up old knowledge; I found the answers in prior blog posts (see references below). It looks like something wiped out our Dreamhost DNS DKIM records a few weeks ago [1]. 

It's hard these days to do email with anything that's not full hosted on the big three.

The refs have more detail but here's the outline of the process:

1. In Google Workspace - Apps - Gmail - Authenticate email get the DKIM text value. You will probably have to generate a new record.
2. In Dreamhost control panel manage websites click on DNS settings for domain and enter google._domainkey as host and the TXT record value. (DH UI makes this look like it appends a suffix to this but it really doesn't.)
3. Once DH says the record has propagated return to the Admin console and click "start authentication"

It can take 48 hours for this to fully propagate but a few minutes after adding the key it did work when I validated as below.

To see if DKIM is working follow the process Google outlines (Usual automated testing services don't work with the way Google Apps do DKIM -- the selector won't work):

Send an email message to someone who is using Gmail or Google Workspace. (You can't verify DKIM is on by sending yourself a test message.) 

In the message header, look for Authentication-Results. Receiving services use different formats for incoming message headers, however the DKIM results should say something like DKIM=pass or DKIM=OK.

If the message header doesn't include a line about DKIM, messages sent from your domain aren't signed with DKIM.

When I looked at view original sent form Emily's account to my personal gmail account I didn't see DKIM=pass or DKIM=OK but I did see two entries starting with:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;

X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; 

More importantly when I scrolled up a bit (this is in Gmail View Original) I saw a header that's interpreting the email headers (I think this might be a newer feature):

Message ID ....

Created at: Mon, Mar 13, 2023 at 2:59 PM (Delivered after 12 seconds)

From: Emily ....

To: John ...

Subject: test DKIM content

SPF: NEUTRAL with IP ...

DKIM: 'PASS' with domain ...

You can also paste the "original message" headers into toolbox.googleapps.com/apps/messageheader/. That gave similar results.

When I tested on a second family domain that did NOT have DNS TXT entry for DKIM it showed as DKIM 'PASS" in the email header interpretation with an odd domain string -- BUT in Google Apps it showed as NOT authenticating. I cannot explain this. 

Once I updated the DNS TXT DKIM entry for that domain and allowed a few minutes for propagation it did show in the Google Apps admin console as authenticating with DKIM and the headers showed the correct domain name.

DH's note on SPF records says they cannot be updated if we are using Google Workspace. This is new since 2018 and I think that's correct.

- fn -

[1] Eons ago every Dreamhosted domain had a free option to add Google Apps (now Google Workspace). That went away and earlier this year Google said they would discontinue the legacy free Google Apps. That was a thrash but on the edge of doom Google relented. I figured a DH script wiped the records but their first tier support said they hadn't done anything and instead referred to Google automated scripts that might change their DNS records (!).

- refs -

• Google - turn on DKIM for domains
• Google - troubleshoot DKIM issues
• Dreamhost DKIM records
• Check DKIM and SPF records
• Check DKIM records
• DKIM for Google Apps 2011 (easier then because we used a DNS service tied to Google. There's an analogous fix for Google Apps 2023 if you use their DNS. Yeah, I know they don't call it Google Apps any more.)
• Dreamhost SPF 2018 (looks like they were screwing up DKIM settings back then as well)
• How to get Google to DKIM authenticate your domain (2022)
• Dreamhost SPF records: "If you're using Google Workspace, you cannot update the SPF record. The field remains uneditable."

Switching from Comcast to Century Link - including how to return Xfinity equipment

Seven years ago we switched from CenturyLink to Comcast/Xfinity. At the time I thought CenturyLink was kaput. Today we switched back; I'm sharing some notes on what to expect.

We had good service from Xfinity, but over the past year we've been hitting their 1.2TB data cap. I'm sure I could call Xfinity, threaten to leave, and negotiate transition to their 'unlimited' tier and get a price reduction as well.  I could do that, but the process irritates me.

Happily Century Link has survived, we are in their fiber area, and their fiberoptic GB no-cap service is apparently about what Xfinity charges us. I say apparently because we live in the surprise-fee era. You don't know what you pay until you get the bill. [Update: Century Link quoted me $70 and I'm honestly stunned to discover that is exactly the bill.]

CenturyLink customer service seems as abysmal as ever but local reports tell me the fiber is far more reliable than their old tech.

So we switched. A few notes on the process:

1. I began by turning off Xfinity automatic bill payment. I called and confirmed we had month-to-month no-contract and that final bill would be prorated.
2. CenturyLink signup was simple but their process is weak. They insisted on contacting us by phone;  it's a miracle we were able to catch the call and respond. A snow storm delayed installation; they didn't tell us the install was canceled and they appeared the next day unannounced. I'm retired so I was able to make that work. The web site does supposedly allow formal rescheduling but overall that's a C- performance. The tech did a nice job on the install, it was a direct shot from the pole to where our Comcast modem was. Since we have an Eero network he just had to swap a cable and we were on the new service.
3. After 3 days of use I called 800-266-2278 to cancel Xfinity/Comcast. Cancelation didn't take too long, I had to say "representative" a couple of times to bypass the vmail menu. The Comcast rep sounded depressed but typed on and on trying to complete the exit process. They were unsure how to return the hardware by UPS but I found the Comcast support article (which is flawed, more below)
4. Xfinity prorates the final bill. It's not clear if they ever actually charged me for the one time we went over their 1.2TB data cap.
5. Century Link charged me on 12/27/22 though service didn't start until 1/6/2023 (December 27, 2022 - January 26, 2023). On Jan 4 there was a 0$ bill for equipment and installation. My next bill is due 2/4 so the amounts do even out. The default setup is autopay to my credit card. I'll see if that generates an extra fee.
6. It's not obvious on the web site how one sees a bill. The bill date is a hyperlink. Shockingly the bill appears to be exactly the $70 they claimed; that's what appears on my credit card. Xfinity charged me $91 so I'm unexpectedly saving money.

The Xfinity requirement return is done through xfinity.com/returns -- but the directions are buggy. The easiest option is to bring the hardware to a comcast/xfinity store but I opted to drop off at UPS. Follow the prompts to create a mailing label but be ready to conclude without, you know, the mailing label. To print the label go to customer.xfinity.com/devices/returns/status and you'll see how to print. Box up the xfinity hardware in anything, put label inside and outside box and drop off at a UPS store.

The CenturyLink model is a C4000XG; that site has an online user guide. I couldn't find a manual for it on the CenturyLink site but I did find one online. https://192.168.0.1 worked to get to the admin screen, you need the CenturyLink sticker with the Admin password. You will certainly want to change the passwords to something tappable. The device web server doesn't render properly in (old) Mojave Safari so I used Chrome.

I'll update this post if I find any other surprises in the process.

Update 2/5/2023

I dropped the xfinity router off at the UPS store with the return label and a week later I got an xfinity email saying they had the router.

After we discontinued service my Xfinity login stopped working. There was no error message, it would let me enter my password but then request my password again. I wondered how I'd pay my last bill but a paper statement came in the mail with a credit on it. We pay in advance so there's credit for the unused part of the month. It does appear they never dinged me for my data overage. The statement said that in 30 days they would send me a check for whatever is left in the account, but in fact they credited part of my last payment back to the credit card I used.

How to leave Google Apps / G Suite / Google Workspace

UPDATE: As of early May 2022 Google has relented and will allow continued personal use of legacy G Suite domains. You need to login to your domain and then use this URL. (The option is described, a bit obscurely, in a support page).

<background>

It's hard to remember now, but there was a time that geeks had some affection for both Google and Apple (but, TBH, never Microsoft). Those were the glory days my friend.

This year's bitter resentment is brought to you by Google ending free Google Apps services. Back in the glory days Dreamhost bundled these with domains, I picked up 7-9 of them. Two of these Google App domains have been heavily used by my family. They are the core of a wide range of daily things we do, including email addresses association with numerous logins, credentials, passwords, and so on. (But not with Google OAUTH identity services, that is not supported for Google Apps email addresses.)

A few months ago, in early 2022, Google told us that these services, once as permanent as gmail (*cough*, they're coming for you), would become quite expensive. For us the costs to maintain our current setup would be hundreds to thousands of dollars a year. Shortly after this announcement we were told that there *might* be a reprieve, that non-business services would continue. This false-hope was never officially withdrawn, but in May 2022 it has been replaced by a bizarre offer to maybe continue but, like, without email or domain?

Google's very limited online guidance does not review how to exit Google Suite. In email communications they mention a 'suspended state' but do not describe what that means.

So now I have to spend several lovely days in May sitting at my computer trying to salvage our digital identities. We will clearly have to pay for at least one of our domains - principles be damned. Charges begin Aug 1, 2022. </background>

The following is a rough guide to what I will do. Much of this requires knowledge from decades ago that I'm having to refresh.

Considerations and discoveries

1. It's difficult to move IMAP emails between services. IMAP emails can be copied to a local store. In mail. app I've had success dragging and dropping emails from one IMAP inbox to another, but I believe this is fragile and unreliable. You can also copy, see this iCloud example.
2. Local store email is barely supported any more. Mail.app, for example, 
3. My domains are managed by Dreamhost which does provide some classic web services though fewer than it once did.
4. Domain based email forwarding is fragile -- many services including google will reject it. See DKIM notes below.
5. Modern email is both essential and a river of spam and Google has good spam filtering (though it was better once)
6. The knowledge of how to manage DNS settings is more esoteric now than it once was, and Google Search no longer works.
7. My Dreamhost DNS and mail forwarding has lots of old detritus. That's on me!

References related to closing Google Workspace accounts

1. Microsoft on switching to Office 365 - cancel subscription
2. Fastmail also has switching options, but price not much less that Google Workspace
3. Google has not provided any migration guidance.
4. You close your account by canceling the subscription: https://admin.google.com/ac/billing/subscriptions/ then deleting the account (see below).

References for migrating to Dreamhost email services

1. Dreamhost email client configuration
2. The Dreamhost custom MX config panel has 'uses Gmail' management links that take you to Google admin (so not terribly useful but at least can tell what to change.
3. Dreamhost used to support both a mailbox and a forwarding action but you can't do that any more (still works for old settings). Dreamhost uses Roundcube Webmail but has not enabled forwarding in that app. You can use forwarding directly from a domain but I think Google treats emails forwarded this way as spam. (At one time we were supposed to have had quite large storage caps with Dreamhost, but I think email overwhelmed them. Similar to the days our Gmail storage was to be unlimited.)
4. A comment on this post mentioned imap sync for moving email: "For transferring IMAP email, imapsync works well. There's a free version you can download and run on your computer (or on your hosting provider if you have ssh access). It's well documented and relatively easy to get your head around, and is fast and reliable. I’ve not got any affiliation, but someone pointed me to it a couple of years ago, and I’ve since used imapsync to migrate email hosts for a small organisation. Highly recommended."

References for migrating to Apple iCloud+ email

Apple supports custom domains with iCloud+ email including family sharing.

1. You can assign up to 5 domains to a family group and for each domain each member can have up to 3 email addresses.
2. Apple will instruct on how to do DNS settings (there's a bug in the quotes apparently) - there's also a tech note on DNS settings.
3. Useful twitter stream on migration to iCloud
4. Detailed twitter thread on migration - Google takeout mbox, import into Mail, then drag from local to iCloud.

My steps to closing an essentially unused account where I didn't worry about forwarding

1. Go to Google admin console for account.
2. Review how many users exist. (typically one)
3. For that user review email to see if there's anything important, sites, docs, etc. Don't forget google  voice!
4. From Google Admin account cancel your subscription. Now pay close attention so you don't miss the next step - delete your account (https://admin.google.com/ac/companyprofile/accountmanagement)

When you choose to delete account you see:

Now return to Dreamhost

1. Go to DNS for domain and delete the Google CNAME records
2. Go Custom MX controls and Choose "make me regular email". It may take hours for this to work.
3. At this point Dreamhost enables webmail. But I wonder if this actually blocks email forwarding even if you set that up! (The lack of warning doesn't give me a happy feeling about Dreamhost TBH.) So disable webmail. Dreamhost also has a control panel for email forwarding that I think is a disabled feature.) - NEED TO TEST MORE HERE
4. Go to Manage Email and set up a forwarding account as needed. This can take a while. Apple picked up the DNS changes within about 15-30 minutes, but Google took 1-2h. (I wonder if DNS propagation in general works as well as it once did.)
5. Enable DKIM if not already enabled.

Impressions of the Eero 6

In retrospect my AirPort Extreme was probably flaky for a while. It's hard to diagnose router failure issues, but my son complained his iPad Zwift app was disconnecting. I was getting corrupted Synology Time Machine backups after years of good results. Then the router started to power down spontaneously. It was 6 years old and the power supply had failed.

We bought an Amazon Eero 6.

Before I talk about the various issues and surprises, let me see this is one of the more fun purchases I've made in a while. It's a huge pain to move all our various devices over to the new primary and "Guest" (where untrustworthy hardware connects) networks, but it's delightful to see 5 WiFi bars everywhere. Including the MyQ garage opener -- which is at least 50 feet from our home. (That's how we open the garage door when it's less than 15F and the external battery powered device doesn't work.) I have one device by the Comcast router, one on the middle floor towards the garage, and one basically hidden beneath the ground floor serving the basement and side patio.

Why did I buy the Eero?

1. Microsoft doesn't make a router, so the least evil tech giant was unavailable.
2. Apple doesn't make a router, so my very Evil master was unavailable.
3. That left Google, Amazon and the small ones (Linksys) who may not last very long and thus can't keep the patches coming.
4. I read that Eero does a good job of updating its devices.
5. Google is maximal Evil. I'm in the midst of a long painful divorce from Google Apps and Google Drive and I really don't want anything more from Google.
6. Amazon could get me a Eero 6 threesome within 8 hours of ordering it. It came at 6am. I felt the fangs of my Prime Parasite dig deeper into my core.

What were the surprises?

• Some older equipment will not connect to an Eero 6. The problem seems to be that the Eero has one SSID with two frequencies; my old AirPort had separate 2.4 and 5 GHz SSIDs. Devices that can only handle 2.4GHz may not work. We ran into two problems [see update below]:

1. SONY Playstation 4 (SONY's link no longer works of course): Go to Settings > Network > Setup Internet Connection > Use Wi-Fi > Easy. Select the SSID (wifi network name) but don't choose it! Now press the Options button on the controller. Select the 2.4GHz band. (When I tried 5GHz I got an obscure error message.)
2. Samsung television: It simply won't connect. Samsung has some tips I've not explored, I could try updating the firmware. Google found some Reddit discussions but that site crashed at the moment. (This TV is behaving a bit oddly, so there may be other issues.)

• The Eero 6 has two ethernet ports on the base unit. It's hard to tell from Eero's dreadful web site but I believe the Eero 6 Pro has ports on peripheral nodes too. I needed one for the Synology NAS and one to the Comcast modem. Given that not all devices work with the Eero it would be nice to have parts on each station -- I could then hardware devices that don't work with Eero wifi.
• The Eeros are smaller than I expected.
• The Eero app crashes every time I try to assign a device to a profile. I suspect it can't handle the ethernet address swapping of modern iPhones. It's amazing that's not fixed.
• If you want to use the parental control features on iOS devices you need to turn off the ethernet swapping. This will make it easier for vendors to track your use however.
• Amazon tries to upsell subscriptions to services including a security package. This is really annoying and it keeps doing it even after initial decline. Be careful not to accidentally subscribe to anything. You don't get filtering or website controls without the extra monthly fee, other routers provide those for free (but they usually don't work well or at all with modern connectivity).

The easiest way to swap routers is to keep the network names and passwords the same. I couldn't do that for reasons, so I get the tedium of reconnecting very old devices to WiFi.

Update 2/27/2022: Pause 5 GHz to allow older devices to connect

I read an Amazon review saying there was a way to pause 5G to allow older devices to connect! I haven't tried on the TV yet. It's an obscure feature:

Settings -> Troubleshooting -> My Device won't connect -> My device is 2.4 GHz only -> Temporarily pause 5 GHz.

There are some other features buried in troubleshooting, including Health Check.

Synology NAS and security risks of enabling notifications

I use a Synology NAS to backup our two MacBooks. I’ve been ignoring it for 4 years, but a recent hardware failure made me look into it.

I found a number of packages installed and/or running that I’d not updated and mostly didn’t need. So I removed all those and I created a reminder to check the NAS quarterly. I also realized I hadn’t gotten monthly status reports for a long time — for years really (if ever)

To enable Synology email status reports you have to configure Notifications. Old-school SMTP is rarely available now, so I experimented with the Gmail option. I got this:

Oookaaay … that’s an interesting range of permissions. Synology is a Chinese corporation, so this effectively gives Xi the ability to harvest my email. Instead I created a synology user on one of my domain based Google Suites and enabled access there then forwarded to my email.

Interestingly my old settings suggested I had gone down the Gmail road at one point. I wonder what I was thinking, in my 2015 post I commented “Synology is a very Chinese product — including off-key English syntax. I wouldn’t install it in a US government facility.” Maybe I started the setup and then stopped?

Enabling SPF on Dreamhost accounts

Dreamhost writes about SPF:

SPF overview – DreamHost

… Mail servers that receive an email for delivery can check SPF by comparing the sending server’s IP address against the email's envelope sender's SPF DNS record. If the email was sent from a server that is not included in that SPF record, the email is more likely to be spoofed or untrustworthy. The receiving mail server may handle the email differently because of the SPF failure, such as marking the email as spam or rejecting the email…

and

What SPF records do I use? – DreamHost

If you’re hosting your email at DreamHost, no changes need to be made to your SPF records. DreamHost’s SPF records are generated automatically and should work without any issues or additional changes.

IP's in DreamHost’s SPF records include mail servers and the relay machine IP addresses.

Except this isn’t true. When I was investigating my family’s Google email Hell I found that emails sent from DH Webmail didn’t have SFP records in the header. I tested using mxtoolbox.com, Kitterman, and by inspecting emails in Gmail using the ‘view original’ option. I also directly inspected my DNS settings. No SFP.

When I asked DH tech support admitted their documentation was wrong and responded (emphases mine):

If it were, you'd see the SPF setup on the 'Manage Domains' > 'DNS' pageby default for every domain hosting mail with us along with the DKIM
 that's already there, but SPF is NOT set by default (at this time).

… we should start setting it by default, so that info above may be accurate in the future. We’re doing a lot with emails right now, which is still mostly in the discussion phase, so this is likely something that’ll come up as well what with various hosts starting to strengthen their incoming filters to help stop spam.”

For the domain in question, I’ve added the record for you through the 'Manage Domains' > 'DNS' page as a TXT record:

… We have now added the TXT record for .faughnan.com with value v=spf1 include:netblocks.dreamhost.com. Our DNS servers will start serving this
record within a couple of minutes.

My DNS records now show an SFP TXT record:

v=spf1 include:netblocks.dreamhost.com

Unfortunately adding the SPF didn’t help with Google treating my DH redirects as spam. I have a hunch those come from DH’s own SMTP service (homiemail) and that service might have a reputational issue with Google, but I don’t really know what’s going on. I don’t think anyone does any more.

What about DKIM?

DH claims DKIM is set automatically and I can confirm that works, at least for a domain that was relatively recently added to DH. On the other hand when I examine that domain’s DNS settings I see several records not in my older domain. I wonder if DH has never updated DNS settings for older domains, such that they are now obsolete. OTOH, even for a new domain there were no SPF records.

And then there’s Google domain verification (postmaster services). I turned those on my adding their key to a TXT field, but a few days later it wasn’t there any more. I assume DH removed it. I’m kind of losing confidence in Dreamhost.

Why you should change your router DNS to Cloudflare and Quad9

I wrote this one for Facebook friends - reposting here. The TidBITS review is excellent.

—

This is roughly how things work ...

1. Someone in your home visits a certain web site.
2. Your home internet provider (ex: Comcast) associates that site with the identity of the Comcast account holder. They sell that information to the universe. Facebook buys it.
3. Facebook shows the Comcast account holder ads based on the visited site.

This happens because your internet provider is your default internet directory service (DNS provider) - Comcast knows all the sites you visit and (as best we can tell) they sell that information.

It's basically an internet design flaw.

If you're a bit geeky you can change the DNS settings in your home router to someone more trustworthy than Comcast (which would be anyone else, really). In our home that's an Apple Airport. I have our settings below but your settings will vary.

We use:

Cloudflare: 1.0.0.1 (their 1.1.1.1 address doesn't work with some ISPs)
Quad9: 9.9.9.9 (as backup)

For more information see an excellent TidBITS review. If you truly want privacy, particularly on iOS where DNS settings are a pain, you need to use a trustworthy VPN (see below).

—

PS. It’s easy in macOS Location settings to experiment with different settings. This is particularly important for a laptop that moves between locations. On my standard macOS Location I use Cloudflare, then Quad9, then Google. Sadly iOS DNS settings are a mess (per TIDBits):

In iOS, DNS server settings tend not to work the way most people would want them, which is as in macOS: setting the details once and having them work on every network to which you connect. The settings have to be set for each network. Worse, we’ve found in our testing that after changing DNS values, the settings revert to Automatic and the server IP addresses we entered are tossed. There’s also no way to set DNS servers for cellular connections.

In iOS you need to use trusted (not free!) VPN provider. I use TunnelBear’s free GB option, I wish I could buy blocks of data from them rather than yet another subscription. If you use a limited data capacity VPN on a Mac you should use TripMode to reduce background data use.

Update 4/22/2018: A Gizmodo article reminded me why this DNS control is more important now ….

“The FCC under Chairman Pai changed the rules in the United States for ISPs allowing ISPs to start selling your browsing history to target advertising against you,” Prince said…

They mention two popular VPNs - ProtonVPN and Private Internet Access. Curiously PIA claims TunnelBear does not protect me — which makes me a bit suspicious of PIA. 

How to maybe stop Sierra from preferentially auto-connecting to XFINITY WiFi.

My Mac insists on connecting to my neighbors Comcast/XFINITY WiFi rather than my home wifi. The usual recommendations didn’t work. (Note, if you don’t run as admin, and want to play with WiFi settings, you need to first click the unlock icon on the Network Preference Pane before you click “Advanced”. Bad UI.)

Despite deleting XFINITY from preferred networks I could still immediately join my neighbors network. Sierra didn’t seem to honor the order of WiFi networks I’d set. I searched Keychain but couldn’t find Comcast or XFINITY left anywhere.

The new trick to look to see if you have a Profiles PreferencePane. There, if you’re like me, you’ll find a “User Profile” called XFINITY WiFi. (I dimly recall installing per XFINITY directions, but now they advise us to use an app — I think the app may allow one to prioritize a local WiFi).

My XFINITY WiFi Profile description said “Automatically connect to XFINITY WiFi hotspots”. It had an Enterprise Profile ID and I suspect MacOS is prioritizing it. So I deleted it.

And … I still connect to the XFINITY WiFI without specifying a password. It appears to need no credentials (or the Profile is still lurking around). When I toggle WiFi on/off I don’t auto-connect to XFINITY though, so maybe removing the cert deprioritized it.

I think this is more a MacOS bug that a Comcast bug.

Things I learned connecting my upgraded Comcast modem (Arris TG1682G)

Comcast sent me an email offering a modem upgrade that would enable higher speeds. I figured I’d have to do it sooner or later so I accepted the offer. Higher speeds would fit with Comcast’s “pay to play” network non-neutrality strategy [1].

I got things working, but it was a bit of a cluster. Comcast tried to make this self-serve, but they didn’t quite hit the mark.

I’ll skip the boring details of things that didn’t work and chat advice that was misguided and just list the useful bits I wish I’d known:

1. I replaced a simple, compact wall mounted ARRIS CM820A/CT (no wifi, coax in, ethernet out) with a much larger standing ARRIS TG1682G (download manual from link. Has 4 ethernet, VOIP phone, wifi, xfinity hotspot).
2. You can disable the WiFi and the xfinity hot spot and you can run it as bridge or router. See below.
3. XFINITY support can remotely configure changes, but they can take hours to be implemented. I’m guessing there’s some sort of slow queue.
4. The TG1682G ships with two high quality coax connector cables. My original Comcast installer had trouble with excessive signal strength on the original device and installed an inline damper (resistor?). I reused that cable.
5. You have to activate it before you use it. Even after all lights are green and it seems to be on the Net it’s really not. It’s locked out by Comcast until you connect to the device (wifi or ethernet) and navigate to xfinity.com (if that fails, try http://register.xfinitiy.com). You should see an activation prompt. The ’text code’ method didn’t work (message arrived much later) and the login method failed too. I entered my account number and address. You have to accept a new web services agreement. I don’t want to know what I agreed to. 
6. Once it’s activated go to http://10.0.0.1/ to configure the device using credentials of un=admin and pw=password (yes, “password”). You will be prompted to change your password. I suspect 99% of users don’t because the standard setup skips this step.
7. From the web interface you can disable wifi (I do wifi via Airport Extreme), adjust firewall, and go to bridge mode. For now I let it in router mode (so I have two firewalls - Airport Extreme and Arris). I don’t use P2P so I upped the firewall to standard security
8. To disable the XFINITY “hotspot” service (used to provide roaming wifi services to their customers) you can use the xfinity app on your iPhone or go to https://customer.xfinity.com/#/settings/internet.

I pay $80/month for Performance 25 Internet with Blast Pro; allegedly 200 down and 10 up. To test I connected by Gb ethernet directly to the router and  http://speedtestbeta.xfinity.com gave me 238 down and 12 up. Which is what I pay for, but not faster. Maybe the higher speeds are coming?

 

- fn -

 

[1] Shortly after the start of Crisis-T Comcast capped our network capacity at 1TB/month. The timing might have been coincidence, but maybe they were waiting for the end of net neutrality.  Carriers do bandwidth caps so they can extort funds from media partners (ex: Netflix) who pay to be excluded from the cap. Higher speeds make economics sense then, because they enable more media consumption.

 

Update 3/12/2017

 

A few weeks after I’d installed the Arris a comcast technician came by my home with a new inline filter. According to this gentleman the new Arris modems chat with each other and this was somehow unwanted.

 

Knowing Comcast this all sounds ominous. On the other hand, I don’t see how a dumb filter would block a typical digital signal even over analog cable wiring.

 

After installing this additional filter I had no net service. I had to power cycle the modem and wait about twenty minutes to restore service.

Uninstalling Synology Cloud Station Mac -- and why you would never have installed it in the first place.

Synology Cloud Station aka Cloud Station Drive stopped working reliably for me in El Capitan.

That’s when I discovered there’s no uninstaller.

Damn.

I have a rule of not installing Mac apps that can’t be easily uninstalled [1]. Clearly I screwed up. In my experience if a Mac apps doesn’t have a clean uninstall it’s not going to work out well.

Uninstall directions for Ubuntu mention a buried uninstall script in ./Cloudstation; I have Finder set to show invisible files (I like running this way) so I found the Mac version of this in /Users/jfaughnan/.CloudStation (you might be able to open this using Finder Go). I couldn’t find an uninstall script in the Mac package though. I didn’t find anything useful on Synology forums; a site that markets a (worthless) Mac uninstaller app gave generic directions.

This is what I did. I think EasyFind will do most of the work for you - just search on Synology.

1. Exited Synology Cloud Station from the blue cloud icon Finder bar thing.
2. Searched Activity Monitor for Synology and Cloud. Found two processes called “synology cloud station drive finder integration” and killed both.
3. Deleted /Applications/Synology Cloud Station.app
4. Deleted /Users/jfaughnan/.CloudStation
5. Used EasyFind.app (love it) to search for all files and folders containing “synology” including Invisible Files and Folders and Package Contents. It found a ton of crap. For example: /private/var/folders/5c/gx565vx110gcdhxn10mbddfh0000gp/T/com.synology.CloudStation.FinderSync [2][3]. I deleted that and several more. I’ve included below a set of partial screenshots form Easy Find (shame I don’t know enough unix to do this in shells and dump results). It’s a mess, but in practice I only had to delete 3-4 things to clear them all out. (If you have other Synology tools installed you’ll probably damage them too, but really you shouldn’t let Synology touch your Mac.)
6. Log out and then in again to confirm Finder iteration is gone.

Ugh. What a mistake that was.

 

Examples from EasyFind (many of these are package contents, so only need to find Package or Folder and delete the lots)

 

- fn -

[1] Lack of OS uninstall support is one of the historic defects of macOS / OS X.

[2] cmd-opt-c to copy file path is the best feature of El Capitan. Almost as good as old cmd-opt-shift-v for paste as plain text.

[3] There are lots of weird files in ‘5c’, looks like some kind of cache/dumping ground. I just deleted the Synology one. There’s also a com.microsoft.OneDrive-mac.FinderSync in there. Turns out Finder Sync is a newish OS X API that does a lot of interesting things (probably with bugs and Finder screw-ups, knowing Apple):

… The Finder Sync extension point lets you cleanly and safely modify the Finder’s user interface to express file synchronization status and control. Unlike most extension points, Finder Sync does not add features to a host app. Instead, it lets you modify the behavior of the Finder itself….

… With a Finder Sync extension you register one or more folders for the system to monitor. Your Finder Sync extension then sets badges, labels, and contextual menus for any items in the monitored folders. You can also use the extension point’s API to add a toolbar button to the Finder window or a sidebar icon for the monitored folder…

… Finder Sync supports apps that synchronize the contents of a local folder with a remote data source. It improves user experience by providing immediate visual feedback directly in the Finder. Badges display the sync state of each item, and contextual menus let users manage folder contents. Custom toolbar buttons can invoke global actions, such as opening a monitored folder or forcing a sync operation…

…  the user may have multiple copies of your extension running at once, and some may be very short lived…

Update 8/23/2016: I can’t kill the Finder integration on Emily’s Mac. Keeps returning. This thing must have malware in its code base.

Comcast - after the promo rolloff

Our Comcast promo rolloff happened today. Our 1st year rate with taxes, modem rental and hidden fees was $56/month. The post-promo rate was $93. I called to see what they’d deal — not much as it turned out. I have a bit of a speed boost and a few dollars off (how much is hard to tell, because the phone quotes don’t include the hidden fees).

They reset my service after the call conclusion — too bad I had an online transaction pending :-(.

I few tips ….

• when get voice mail prompt requesting purpose of call say: “lower my bill”
• you want to speak to “customer retention". I think I only got as far as “customer solutions”
• You can hit 1 instead of saying ‘yes’. Works better.
• I think you want to have an alternative service quote in hand before calling.
• Be ready to call more than once. The first analyst I spoke with passed me on to customer solutions and then my line went dead. Well, this is Comcast we’re talking about.

See also

• New Services Renegotiate Your Connectivity Bills
• Amazon.com: Comcast XFINITY: get pricing info.
• Gordon’s Tech: Switching from CenturyLink to Comcast: last rats off the sinking ship 8/2016

Synology Time Machine backups: How to increase a user quota

After my Synology NAS updated itself to version 6 one of my Time Machine backups stopped working. It might have been coincidental. Time Machine claimed I only had 350GB free and it needed 1TB, but Synology claimed I had enough free space.

Whatever.

The fix was to increase the quota size for the user who owned the Time Machine disk image belonging to my MacBook Air.

Except I couldn’t do it. I could edit the user easily, but the quota information couldn’t be edited. Clicking on the row did nothing. 

Click-click.

Google helped. It’s a UI issue. There’s nothing in Synology’s UI to tell you to click specifically on the quota number. If you do that you can edit it.

It’s probably a good idea to turn Time Machine backup off while you’re doing this. In any case it’s fixed my problem.

How to update Synology Cloud Station Server Clients (Cloud Station Drive) after Synology NAS 6.0 update

Looking back, 2009 was kind of a bad year. Somewhere around then we were in the tail end of the Great Recession, Google had turned Evil, and, in retrospect, Apple’s glory days were behind it.

I miss the old Apple. It wasn’t perfect, but it shielded me from a lot of hassles. Like dealing with the complexity of my Synology NAS.

I bought the NAS because Apple’s Time Capsule is broken. Next I started using it as a post-server replacement for Apple’s perennially broken network shares. That’s all I bothered with. I didn’t want to bother my NAS, and I didn’t want it to bother me.

Then, inevitably, there was an update. It took me a while to figure out that Control Panel:System:Info showed the version number: DSM 6.0-7321. 

Oh, great. A complete version update. I #$@$#!# never install those. I’d turn off auto-update, but at this point the damage is done. I’m going to need the big bug fixes; I’ll turn it off in a month or two.

Meanwhile both Time Machine and my Client-Server NAS file sync are broken. I’ll fix Time Machine next, this is about fixing the file sync.

Notice I’m not naming the file sync? That’s because Synology, a Chinese company, uses English words inconsistently. They add and remove “Cloud” to everything and seem to move software names between products on a whim. I think I’ve seen File Sync, Cloud Station Drive, Cloud Station Sync, Cloud Station Server and Cloud Station Client used to refer to similar or identical things.

What I wanted was to update “Cloud Station Drive” running on my Mac, which is actually Cloud Station Server Client, to a version compatible with “Cloud Station Server” running on my Synology NAS. Notice neither of these actually have anything to do with a “Cloud”, they’re both LAN specific.

This document helped: Sync files between NAS and computer Network Attached Storage (NAS)

 Here’s what I did:

1. Quit outdated client on my Mac.
2. Start Synology Assistant to locate FLNAS (IP Address), open it.
3. Go to Package Center, All, find Cloud Station Server, click Open (alt: click the four-square-icon next to question mark to see running apps)
4. Click Overview (sometimes this is empty, quit and start over again)
5. Download Cloud Station Drive
6. Install

It seems to be working. I’ll use the DMG to update my other machines.

Next up: Fix Time Machine. It and Synology are disagreeing about how much space is free in the user-quota for my MacBook Air …

Update 8/23/2016: Synology Cloud Station Server / Cloud Drive is broken again. I think it’s worse in El Capitan than Yosemite. I’ve given up on it.

Airport Utility: How to load an external configuration file for a brand new AirPort Extreme

The AirPort Utility tries too hard to be friendly. It wants to do some fancy migration from an older device that’s running, but I’d already exported that configuration file and removed it. Everything was connected, I just wanted to import the configuration file.

I couldn’t!

The trick, of course, is the option key. Cancel out and look for “Other Wi-Fi Devices”. Option-click the new one. You get a ‘power user’ interface (same one with option clicking any device in AirPort Utility). Now you can import the .baseconfig file you exported previously.

It came up perfectly, except it had some odd default password initially. I changed it to my admin password and updated.

My 2014 AirPort Extreme just died.

I liked the ME918LL/A AirPort Extreme for 17 months. It covered most of our house with a single device and it was pretty reliable. Around 17 months it started to spontaneously power down. I suspect an issue with the power supply, Apple has a long history of wonky power supply problems.

I wouldn't be surprised if there's eventually a recall program. I purchased it with an AMEX card, so I’ll now see how good their extended warranty program is. (It used to be quite good, but AMEX has outsourced the program. If it fails me on this one I’ll deprecate my AMEX card.)

In the meanwhile, sadly, I’ve ordered another Airport Extreme. (Sigh.) I really couldn’t find a better option — Eero is quite expensive, unproven, and it comes from a startup that’s got a 20% chance of surviving. Google's hardware is outsourced and of unproven quality.

Apple’s Airport Express has a great reputation of reliability. If you can make do with the Express I’d recommend it over the Extreme. If you live in a country that mandates 2 year warranties then the Extreme is a good buy. If you live in the US either use an extended warranty credit cards or pay extra for Apple Care. Once you add the Apple Care costs the Extreme is more expensive than Google’s router.

Yes, WiFi services suck. They really shouldn’t. I need an electrical engineer to explain to me why so many of my WiFi solutions last 1-2 years. (I have a 6+ yo Airport Express still running …)

PS. Interesting that Amazon no longer sells the Airport Extreme or the Express. I had to buy direct from Apple.

Comcast's xfinity wifi and XFINITY.mobileconfig

The coffee shop’s WiFi was flailing. Periodically my MacBook popped up an xfinity wifi option. I vaguely remembered reading of this when I signed up with Comcast (the Devil we know), so in a fit of recklessness I connected. 

It required my comcast credentials, which I don’t use for anything else. I balked when the install asked for admin privileges but it turned out I didn’t need the install — my connection worked anyway.

So what the heck was going on? And what was a I recklessly installing? Why did I get a connection anyway? (Note I had no proof I was truly dealing with a Comcast site. The less crazy thing to do is to go to Comcast’s web site from a secure network and do any installs from there.)

The install, it turns out, creates a configuration file for Mac OS X Profiles called XFINITY.mobileconfig. It’s a binary file that contains your Comcast credentials in plaintext. (Yep. Delete after use.) The admin privilege escalation is needed to update OS X preferences. (If you run as admin you won’t see this; you really shouldn’t run OS X as an admin user IMHO.)

Oh, you’ve never heard of OS X Profiles? You’ve only heard of iOS Profiles? Profiles is a hidden Preference Pane introduced with Lion and only visible when you install a Profile (rather like iOS actually). "Configuration profiles can be created with the Profile Manager feature of Lion Server. They can configure accounts, policies and restrictions on iOS and Lion clients. The APN settings are iOS only.”

System Preferences will display the profile information (note it’s “verified”, this is via Yosemite):

After installation my Preferences have a new Apple pane, i can delete from there.

So what does this profile do? I was hoping it might enable VPN support, but of course it’s not that useful.  It’s actually configuring my machine to auto-join XFINITY WIFI even if it’s not even WPA encrypted. I hope I’m wrong about that, but this is Comcast we’re talking about.

Their FAQ doesn’t explain what’s happening, but this page suggests that the profile is needed to connect to the “XFINITY” SSID networks. (I was able to connect without using the profile because I was using a “xfinitywifi" SSD.). That makes sense because the profile contains an Enterprise Profile ID. (See iOS directions here.)

Which leaves the question of what’s evil about XFINITY WiFi, because, you know, Comcast. I mean, besides the auto join non-encrypted networks.

Don’t worry, it’s evil. Comcast turns customer’s routers into WiFi hotspots by enabling a kind of “guest network” (my Comcast modem doesn’t have WiFi. Smart I am.) Comcast assures customers Homeland Security will knock politely when visiting for tea to chat about your network use by local ISIS affiliates.

Comcast also enables XFINITY WiFi for business customers, who might be well informed and fine with this. I don’t think there’s any way to tell what you’re connecting to though. Can a provider tap the data stream? This is Comcast, so I would assume so. I also assume Comcast monitors the data stream and sells whatever it learns to various businesses and criminals. Lastly, with auto-join unencrypted networks seemingly enabled, I figure Comcast is getting kickbacks from the honeypot industry.

Caveat emptor.

File sharing for the all-MacBook home

Lifehacker’s guide to home file sharing was written in 2010 for Windows users. Excluding a traditional server/file share the options back then were Dropbox, a NAS, and, peer-to-peer sync solutions. Things haven’t changed much since then.

Now that I’ve retired our iMac and gone all-MacBook, I need one of those solutions for a small number of files (MBs, not even 1 GB). Our home’s options are Dropbox, Google Drive, Microsoft’s OneDrive, a Synology NAS with or without Synology Cloud Station, Mac LAN based sync solutions (ex: ChronoSync, note MSFT bundles this with Windows), and an Airport Extreme external flash drive.

There are lots of options, but nothing is quite perfect. Dropbox, Google Drive and OneDrive all move our family data into the Cloud — and I’d like to not worry about that. Sync solutions mean new software, but perhaps only on one machine.

I’m going to stick our unused $20 SanDisk Ultra Fit 64GB flash drive in back of the Airport Extreme. I already use Carbon Copy Cloner as part of our nightly backup, I’ll just back the AE Flash Drive up to disk image on one of the my OWC Thunderbolt 2 dock drives. They in turn are backed up by both CCC (to removable drives) and Time Machine (to the Synology NAS).

That should be good enough. Keep it as simple as possible…

Update: oops. "When you use Airport Utility to change AirPort Extreme Shared Disk(s) security it *seems* to wipe out everything on the disk. Except free space shows data is still there.”  The AE has an operating system with some kind of file system support and access controls, but we have very limited access to it.

This Apple article partly explains what is supposed to happen. From Airport Utility we can create username/password “accounts”. Say “Parent” and “Kids”. When a client connects you are asked username/password, that gives access to the Folder of the same name as well as a “Shared” folder. So Emily and I connect as “Parents” and see the “Parents”  and “Shared” folder, but we don’t see a “Kids” folder unless we connect with that username password.

There’s no way for me to connect with to the AE shared disk (partitions?) and see everything.

When I insert the flash drive into my MacBook I can see how it’s organized, including the folders that were on the flash drive when it was “password” access rather than “account” access.

When I switched “Secure Shared Disks” from “With a disk password” to “With accounts” it didn’t wipe my data, it created a Users folder containing the “Parents” folder and hid the existing folders. I thought I also created a Kids user, but I don’t see that Folder. Bug?

Hmm. This is a bit weird. I could experiment with partitioning the thumb drive on my Mac, but I think I need to look more at the Synology.  The AE’s file sharing security model seems to make backup impossible.

Update 2: I’ll rewrite this when I finalize things, but it looks like the Synology NAS gives me the permission controls I need. I’ll put the shared files there, then use CCC to put them back on an image on my laptop. That image will in turn go back to the Synology NAS Time Machine backup as well as to my local CCC backups.

Update 11/21/2015: I ended up enabling Synology Cloud Station, including installing the Mac client for both Emily and I. So our relatively small (1.5GB) of shared data exists on the Synology NAS (not baked up) and on both of our machines (so multiple backups). It is a strange outcome for the old file sharing/NFS/WebDav model and it doesn’t seem the most elegant solution, but sync seems to be the current technology direction. (Dropbox would be simpler, but we wanted to keep the data local and, of course, Dropbox costs money. The Synology NAS also supports a BitTorrent sync package but the Cloud Station seemed to have more users.

Configuration was a bit odd — you do need to read the documentation. The default setup is within one’s “Homes” folder, so if you want to share with two users (workgroup) you need to create a folder outside that NAS hierarchy and choose to that for sync.

Update 8/23/2016: Synology Cloud Station / Cloud Drive (it has many names) has stopped working reliably with El Capitan. I’ve given up on it. Emily’s MacBook is largely home so I’m moving these files to her machine and making them a file share. Sometimes I won’t have access, but I’ll move some things to a Google Drive we share.

Comcast (xfinitiy) Arris modem link light blinks orange (yellow?) rather than green - check your ethernet cable.

[jump to the update on this one, it’s not what I thought it was…]

When I switched from CenturyLink to Comcast two months ago my “boost” speeds were 50 up and 10 down on a rented Arris modem. The modem lights were green and i thought the link light was blinking green.

Recently comcast claimed to have doubled my internet speed. It did go up briefly to 100+ mbps, but now it’s back down again. Around the same time I noticed my modem link light started blinking yellow/orange.

It sounds like this may be associated with a > 100 mbps connection, but I suspect it also happens when there’s something amiss with Comcast’s network. Based on what I read and a call to Comcast there’s nothing to do about it as long as you’re getting the speed you paid for (which may not be the speed comcast promised, but there you go.) The comcast rep did say my area was suffering from network issues.

Update: A better thread on Amazon (!) says: "The light is orange when connected to a 100 Mbit device, and blue when connected to a 1 Gbit device”. My modem is connected to a GB device though (Airport Extreme), and the link light should be for upstream connection, not downstream. I may try a different ethernet cable...

Update b: I’m surprised, it’s not at all what I thought. The link light isn’t for upstream connections, it indicates downstream (internal) connection mode — 10/100 is yellow/amber/orange, 1000 (gb) is green. I’d forgotten that, in the midst of resolving an issue with a dying time capsule, I swapped out the ethernet cable connecting my Comcast modem to my Airport extreme. The new cable was a better length, and I thought it was excellent quality. Turns out it wasn’t so excellent! I swapped my original cable back in and the light immediately went green.

I then repeated the Comcast speed test, this time with my Macbook Air within a few feet of my router. For convenience I tried with 5GHz Wifi, not wired gb ethernet. Comcast more than passed the speed test — delivering 125 mbps over wifi. I don’t know if my modem reports a faster internal speed to Comcast and if that impacts provisioning. I’m used to berating my ISP, but Comcast did very well on this one. With 100+mbps (much less gbps) broadband internal networks matter.

In the dining room, a floor below and about 20’ feet away, the same speedtest over 5GHz wifi gave me about 73 mbps (my 11” Air might have been a wee bit faster than my 13” Air). Quite an impressive reduction.