Extracting core concepts with ChatGPT 4 from OCR of scanned sample examination PDF - Feb 2024

I think this is an interesting example of what works and doesn't work on the personal AI front in early 2024.

My son was given a printed practice exam in microeconomics. I wanted ChatGPT 4 to extract and summarize the core concepts. This turned out to require two steps, one of which only worked with Google.

Step One: OCR and download text file

I scanned the document in ScanSnap and produced a scan PDF. I tried getting ChatGPT to do the OCR but it abandoned that task. I then tried Gemini and it told me it didn't do OCR. Next I tried Microsoft Lens, but it seemed to only do OCR from a local image, I couldn't see how to use it with a OneDrive PDF. ChatGPT claimed that I could open a OneDrive PDF in Office 365 Word but that did not work with the web version (perhaps it works with full Word?). ChatGPT did not know of a way to do PDF OCR on Sonoma.

The only thing that worked was Google Drive. It allowed me to open the PDF in Google Docs and then export a .txt version.

Step Two: ChatGPT 4 analysis

I asked ChatGPT 4 to extract the key concepts from the .txt file. It provided a plausible set and then proceeded to answer some of the exam questions. Concepts captured were:

... equilibrium price, consumer's surplus, producer's surplus, total surplus, efficient output levels, negative externalities, deadweight loss (DWL), price ceilings, and the impact of taxes on market outcomes ...

I don't think it added much to the textbook chapter topic loss but it did provide a plausible set of topics to emphasize in my son's studying. I was primarily interested in the workflow today. It will be interesting to look back on this in a year and see what's different.

Google Account without Google email that routes email to a Google Workspace address

My son has an email address of the form tim@familyname.com where familyname.com is a pseudonym for a domain I own with legacy free Google Apps (now Workspace) accounts.

He also has a Google Account with the Google Account ID of tim@familyname.com. I think he created it somehow with YouTube. (He has a cognitive disability which doesn't stop him from doing some weird shit I can't figure out.)

His second Google Account does not have the email address of tim@familyname.com because that belongs to the Google Workspace account. Instead of it has an email of the form: tim%familyname.com@gtempaccount.com. If I send email to that gtempaccount.com address it redirects to tim@familyname.com.

This feels like a security risk waiting to be exploited. It certainly is weird. It reminds me that Google Workspace accounts don't have the same meaning and privileges as full Google @gmail.com accounts.

xt.local spam: Where it comes from, how to get rid of it

Recently more of the email that bypasses Gmail's spam filters has a return address with the suffix "*.xt.local". I've been able to find low quality posts about this issue from at least 2017, but in the past few months the volume has increased significantly.

One source of the emails has been spammer customers of Salesforce.

I'd previously marked one of these as spam, when I view it from inbox Google shows me: 

"You unsubscribed from <100018015.xt.local>"

If I try to reply I see:

"reply-fec215727d600275-157_HTML-100984021-100022017-13039@e.sixt.com"

If I choose to filter I get this as a filter criteria

Has the words: list:(<100018015.xt.local>)

An expert on spam filtering tells me xt.local is the name of an email list. 

I've added xt.local to Gmail's (increasingly obscure) filters as an automatic delete. I currently have it in there as a return address but if that doesn't work I'll try it as a label. 

Google's 'show original' extracts the following header information:

.... 

From: SIXT <info@e.sixt.com>

To: ********

Subject: Start your week in style! Up to 30% discount on midweek rentals

SPF: PASS with IP 13.111.115.170 Learn more

DKIM: 'PASS' with domain e.sixt.com Learn more

DMARC: 'PASS' Learn more

I'll update this post if filtering on xt.local as the sender address doesn't work.

PS. When I searched on this, even using Kagi (which is looking to be a very interesting search engine if they can stay in business), I found very few useful posts. Even five years ago there would have been dozens. Something rather important has been broken.

How to leave Google Apps / G Suite / Google Workspace

UPDATE: As of early May 2022 Google has relented and will allow continued personal use of legacy G Suite domains. You need to login to your domain and then use this URL. (The option is described, a bit obscurely, in a support page).

<background>

It's hard to remember now, but there was a time that geeks had some affection for both Google and Apple (but, TBH, never Microsoft). Those were the glory days my friend.

This year's bitter resentment is brought to you by Google ending free Google Apps services. Back in the glory days Dreamhost bundled these with domains, I picked up 7-9 of them. Two of these Google App domains have been heavily used by my family. They are the core of a wide range of daily things we do, including email addresses association with numerous logins, credentials, passwords, and so on. (But not with Google OAUTH identity services, that is not supported for Google Apps email addresses.)

A few months ago, in early 2022, Google told us that these services, once as permanent as gmail (*cough*, they're coming for you), would become quite expensive. For us the costs to maintain our current setup would be hundreds to thousands of dollars a year. Shortly after this announcement we were told that there *might* be a reprieve, that non-business services would continue. This false-hope was never officially withdrawn, but in May 2022 it has been replaced by a bizarre offer to maybe continue but, like, without email or domain?

Google's very limited online guidance does not review how to exit Google Suite. In email communications they mention a 'suspended state' but do not describe what that means.

So now I have to spend several lovely days in May sitting at my computer trying to salvage our digital identities. We will clearly have to pay for at least one of our domains - principles be damned. Charges begin Aug 1, 2022. </background>

The following is a rough guide to what I will do. Much of this requires knowledge from decades ago that I'm having to refresh.

Considerations and discoveries

1. It's difficult to move IMAP emails between services. IMAP emails can be copied to a local store. In mail. app I've had success dragging and dropping emails from one IMAP inbox to another, but I believe this is fragile and unreliable. You can also copy, see this iCloud example.
2. Local store email is barely supported any more. Mail.app, for example, 
3. My domains are managed by Dreamhost which does provide some classic web services though fewer than it once did.
4. Domain based email forwarding is fragile -- many services including google will reject it. See DKIM notes below.
5. Modern email is both essential and a river of spam and Google has good spam filtering (though it was better once)
6. The knowledge of how to manage DNS settings is more esoteric now than it once was, and Google Search no longer works.
7. My Dreamhost DNS and mail forwarding has lots of old detritus. That's on me!

References related to closing Google Workspace accounts

1. Microsoft on switching to Office 365 - cancel subscription
2. Fastmail also has switching options, but price not much less that Google Workspace
3. Google has not provided any migration guidance.
4. You close your account by canceling the subscription: https://admin.google.com/ac/billing/subscriptions/ then deleting the account (see below).

References for migrating to Dreamhost email services

1. Dreamhost email client configuration
2. The Dreamhost custom MX config panel has 'uses Gmail' management links that take you to Google admin (so not terribly useful but at least can tell what to change.
3. Dreamhost used to support both a mailbox and a forwarding action but you can't do that any more (still works for old settings). Dreamhost uses Roundcube Webmail but has not enabled forwarding in that app. You can use forwarding directly from a domain but I think Google treats emails forwarded this way as spam. (At one time we were supposed to have had quite large storage caps with Dreamhost, but I think email overwhelmed them. Similar to the days our Gmail storage was to be unlimited.)
4. A comment on this post mentioned imap sync for moving email: "For transferring IMAP email, imapsync works well. There's a free version you can download and run on your computer (or on your hosting provider if you have ssh access). It's well documented and relatively easy to get your head around, and is fast and reliable. I’ve not got any affiliation, but someone pointed me to it a couple of years ago, and I’ve since used imapsync to migrate email hosts for a small organisation. Highly recommended."

References for migrating to Apple iCloud+ email

Apple supports custom domains with iCloud+ email including family sharing.

1. You can assign up to 5 domains to a family group and for each domain each member can have up to 3 email addresses.
2. Apple will instruct on how to do DNS settings (there's a bug in the quotes apparently) - there's also a tech note on DNS settings.
3. Useful twitter stream on migration to iCloud
4. Detailed twitter thread on migration - Google takeout mbox, import into Mail, then drag from local to iCloud.

My steps to closing an essentially unused account where I didn't worry about forwarding

1. Go to Google admin console for account.
2. Review how many users exist. (typically one)
3. For that user review email to see if there's anything important, sites, docs, etc. Don't forget google  voice!
4. From Google Admin account cancel your subscription. Now pay close attention so you don't miss the next step - delete your account (https://admin.google.com/ac/companyprofile/accountmanagement)

When you choose to delete account you see:

Now return to Dreamhost

1. Go to DNS for domain and delete the Google CNAME records
2. Go Custom MX controls and Choose "make me regular email". It may take hours for this to work.
3. At this point Dreamhost enables webmail. But I wonder if this actually blocks email forwarding even if you set that up! (The lack of warning doesn't give me a happy feeling about Dreamhost TBH.) So disable webmail. Dreamhost also has a control panel for email forwarding that I think is a disabled feature.) - NEED TO TEST MORE HERE
4. Go to Manage Email and set up a forwarding account as needed. This can take a while. Apple picked up the DNS changes within about 15-30 minutes, but Google took 1-2h. (I wonder if DNS propagation in general works as well as it once did.)
5. Enable DKIM if not already enabled.

Google's mysterious new blogging platform

Google Blogger has been largely forgotten, but over the past 1-2 years it's been receiving regular updates.

Mostly these have been improvements with a few odd regressions. Some of the regressions have been fixed.

It's kind of curious. Google still uses Blogger for some of their blogs on googleblog.com (ex: Scholar), but they also have a new platform - https://blog.google (Keynote, Data Centers). On the Keynote blog page the RSS feed is hidden (but exists), on Data Centers and Photos blog there's a familiar feed icon top right. Data Centers articles date to 2012, but the .google domain was only registered in 2014. So they've migrated some old content, probably from Blogger.

I looked a the source from a Data Center post and it's surprisingly old school readable. There are commented out tags for handling IE 7 (!) and metadata for Open Graph and Twitter Card. Style sheets refer to "/static/blogv2/css/blog.min.css?version=4.4" />. 

I wasn't able to find any articles on "Google's new blog platform". That doesn't surprise me, Google search is fairly useless these days. Clearly they are up to something internally.

If they do make this a public blogging platform I'm sure it still won't handle paragraph spacing correctly.

ToDo apps: Microsoft's solution

I've used Appigo's ToDo app for about 12 years (with Toodledo at first). It's had problems over the years, but in general it's been a good subscription choice. There's a fairly hard data lock (maybe SQLite?) but manual reentry is feasible albeit annoying.

Lately, however, ToDo has been more ragged. A recent server side change induced a date bug (time zone?) that in turn showed me I was using a macOS app last updated in 2016. It appears to have been abandoned on the Mac App Store. When I went to Twitter I found Appigo's account was closed years ago for violating TOS. Eventually I found I could download a current version of their other App Store app from their web site.

At the moment the app is more or less working again, though parts of the macOS app UI are kind of weird. I figure there was some violent ownership transition with lost dev passwords in Appigo's history (maybe they got ransomwared?).

I decided to go shopping again. I'm looking at:

• Apple Reminders: hard data lock and I have to upgrade from Mojave to get to latest version (not happening).
• Google Todo: this is one hell of a weird product. WTF is their web strategy? Tied to Gmail? Tied to Calendar? At least there's data export.
• Things
• OmniFocus: poor Omni is in some disarray ...
• Microsoft To Do

Today I dug into Microsoft To Do. Of course it's a mess, but this is 2020 so we expect that. The mess starts with Microsoft reusing product names. To simplify a bit:

• There are classic Outlook Tasks. I'll call these TasksClassic. TasksClassic was excellent in many ways, including, once upon a time, great import/export options and lots of view flexibility (I like to sort by last modified!). Unfortunately it's dead, just barely hanging on in the current desktop app with some degree of synchronization with the new product.
• There's the new Wanderlist-based product variable called Microsoft To Do and ... Outlook Tasks (name reuse!). I'll call these TasksW for Wanderlist.

If you open the Help screen page for macOS TasksW (To Do) it takes one to a page on Outlook synchronization that's obsolete -- because the Outlook.com version of Tasks has switched from TasksClassic to TasksW. On the other hand the version of Office 365 on my Mac still has TasksClassic, and it does synchronize with TasksW as displayed on macOS and iOS Microsoft To Do.app.

Are you still with me?

This gave me a brief moment of hope that there was some data freedom here. I remember the import/export options of old Windows Outlook. Alas, the only import/export from macOS Outlook is Microsoft's PST format. There might be some way to do things with Outlook Windows or with 3rd party tools but I don't have the energy for that.

At this time I think TasksW is probably a decent enough product, but this has reminded me how screwed up Microsoft is. So I'm setting this one aside for the moment.

See also:

• There are no great task managers for the iPhone - but there's hope for 2011: My current task list started out on the Palm III and has migrated many times. I probably have uncompleted tasks from the 1990s. I think we're coming to the end of that data migration though. Note from 2010: "Google might finally provide an API for Google Tasks, allowing iOS client development."
• My ancient spreadsheet on migrating from Palm to iPhone - I think I used Outlook Tasks as part of the migration process, or maybe even Microsoft Access. Alas, too long ago. Painful though.

Blogger (draft) supports mobile

If you are using Blogger Draft you will find that it works fairly well in Safari.app and Chrome.app for iOS.

The new interface is responsive. 

I’d prefer a different font for writing on mobile but it’s very doable. Competitive with WordPress mobile app but expect some rough edges.

(Eons ago there was a mobile app for Blogger, but it was discontinued. I doubt it will return.)

Unable to use MarsEdit with Blogger - Google web OAUTH failure with DHCP mode on AirPort Extreme

I'm now unable to use MarsEdit with Blogger on Mojave or High Sierra. When Google asks me to authenticate instead of getting the usual embedded web page for entering credentials I get a blank page.

Either Google has changed something so this only works on Catalina or there's something odd about my Google accounts. I've tested with

• two non-2FA Google Apps identities that used to work
• my personal 2FA Gmail Google ID
• my wife's personal Gmail Google ID never used for blogger
• two user accounts on my Mojave machine
• 1 user account on High Sierra
• Admin and non-admin account
• Clearing caches with Onyx
• I think I also tried an older version of MarsEdit that was on my old account on the High Sierra machine

macOS Console has not been helpful, but I have yet to download and try it using Consolation.

 

MarsEdit dev (Daniel Jalkut) says it works for him on Catalina and on pre-Catalina.

Update 12/6/2019: I eventually realized the problem was my home network. I have a Comcast Xfinity router and an Apple AirPort Extreme. Both have had no configuration changes in months, but Comcast recently updated my service and the AirPort firmware went to version 7.9.1 a few months ago.

 

Inquiry on Apple Discussions.

 

Update 12/7/2019: Today I discovered my AirPort Extreme Guest Network no longer works. That’s the network all our IOT devices connect to.  Turns out suggests that the Guest network doesn’t work in Bridge mode unless the internet router is configured for VLAN support. Would be nice if Apple documented some of this stuff. Maybe this is why they exited the router business.

 

So I have to go back to double NAT for now then try reverting the firmware. (Though once I’ve authenticated a blog I don’t need the web OAUTH, so there is that option.)

 

Update 12/8/2019: So I reverted from 7.9.1 (79100.2) to 7.7.9 and it still doesn’t work! So I think I’ve ruled out a problem with the AirPort Extreme firmware update. I think I tried an older version of MarsEdit. I can’t say for sure but now I suspect it’s something Google did that broken compatibility with Double NAT (DHCP) on the AirPort Extreme.

 

I went back to 7.9.1 (got an ominous notice that firmware update failed but it seems fine) and I guess I’ll use my TunnelBear VPN when I need to authenticate with Google. I’ll test periodically to see if Google fixes things.

 

I tried out Google OAUTH playground. It’s a bit above my pay grade but it seemed to work across my Double NAT setup.

How to clear Google Voice conditional call forwarding from an AT&T phone including iPhone

Long ago I setup Google Voice as my son’s phone’s voicemail using “conditional call forwarding” on H2O Wireless mobile (very cheap, but also weak IT service). I switched him from H2O Wireless to our family AT&T account and wanted to disable the forwarding. 

Unfortunately I couldn’t get Google Voice legacy site to come up on my iPhone so I didn’t disable Conditional Call Forwarding prior to the port. I tried removing the his number from GV but that doesn’t work — it only messes up Google Voice (below).

After the port I couldn’t set a voice mail code for his account. It seemed like voice mail was setup, but when I called him I was routed to Google Voice. Alas, there’s no useful error message.

I restored his mobile number as a forwarding phone in Google Voice but there was  no option for him in Legacy Google Voice to add or remove conditional call forwarding, no “Activate Google voicemail on this phone”. (Probably because the call forward was already in place.)

I tried a #21# trick I read of, but that didn’t work. I thought I’d have to call AT&T but after several searches I found an obscure site with the answer: Dial #61#. That clears “call forward if not answered”. I also used the other deactivation codes as below (67, 61, 62).

I don’t know if that site will stay around, and I couldn’t find this information anywhere else, so here are the set of deactivate codes for several purposes:

To deactivate “Call forward if busy”, 
Simply dial #67# and tap “Call”

To deactivate “Call forward if not answered”,
Simply dial #61# and tap “Call”

To deactivate “Call forward if not reachable”, 
Simply dial #62# and tap “Call”

Android phones may have more control over Call Forwarding but iOS has only forward or don’t forward.

The site also reviews how to setup call forwarding, for example:

To setup “Call forward if busy”,
simply dial 
*67*{NumberRetrieved}*11# and tap “Call”

After using those codes I could set a PIN for his voicemail and it worked normally.

Migrating from Blogger to WordPress ... again ...

I’ve been contemplating migration to WordPress for almost a decade, but Google kept Blogger good enough to keep that headache at bay.

Alas, the days of good enough are ending. Google is removing their photo management API without recourse. They do support posts with images, but only by using their web interface. It’s a concrete and undeniable sign that Blogger is either dead or going to a bad place.

I though I’d migrate first to wordpress.com then to my Dreamhost open source wp install, but via Twitter Daniel Jalkut tells me he got better results using the open source importer directly.

I’ll do a dry run on one of my big blogs first. The URL won’t change but I’m sure feed subscriptions will have to be redone (ugh).

Update 2019/04/06 - results of the pipdig import process

I tested the Dreamhost free version of the pipdig importer from a Dreamhost wordpress (open source) blog. The results can be seen here for the moment, I’ll eventually delete them. I found:

• It doesn't remap internal links. This is a big disappointment. Links continue to direct to blogger, once that account is gone they will be invalid
• There’s no option to migrate images that I can see.
• It missed at least 4 posts from the source blog — specifically from early on. No idea why and it suggests more are missing.
• It does copy drafts over.
• It requires a LOT of access to your Google account! If you use this utility I suggest creating a new google account, give it access to your blog, then after the import destroy it. 
• The paragraph breaks are missing - line feeds vs <p>. This is an ancient Blogger problem with MarsEdit; a legacy of the original sin of English language text formatting end-of-line standards. I think Blogger is mostly to blame.
• Images were not relocated locally, they remain at their original locations.

Pipdig is better than nothing, but I’m going to try wordpress.com’s import tool next. I wonder if a better solution wouldn’t be a static site that I could archive on my personal web server, then do a web server redirect to handle the links. For now I’m still on Blogger. The porting experience reminds me of the impossibility of leaving Apple’s defunct Aperture photo management app.

(As I write this the wordpress import is processing - result should eventually show up at gordontest.tech.blog temporarily, but we’ll see if it works. It’s taking a long time.)

Update 2019/04/06b

Well, that wordpress migration didn’t go so well:

Your site has been suspended from WordPress.com for violating the Terms of Service. If you believe this was done in error, please contact us as soon as possible to have the suspension reviewed….

I sent a contact inquiry, nothing yet.

The end of Google+ will impact Blogger

Visiting Google’s official Blogger blog today I tried viewing comments on a May 2018 post (a list of things removed and a promise of future work). There are 858 comments, based on Google+. I wonder what will happen to them now that G+ is dead. (So will we get our + back in search syntax?)

At one point Google tried to integrate G+ and Blogger — particularly identity management. It didn’t go well. I suspect the divorce won’t go well either.

- fn -

[1] Suggestively most of the future work mentioned were enhancements to moving data out of Blogger.

PS. Google+ was a really dumb name.

Google Voice: "We could not complete your call - Please try again."

I’ve been using Google Voice for eons — starting before Google acquired GrandCentral. It saved me thousands on my daily mobile calls to my mother in Canada.

GV must be a real money loser for Google though — the interconnect fees to landlines are a real cost [1]. So it’s not surprising that it’s a bit of a mess — between GV legacy, GV current, Hangouts [2] and heaven knows what else.

Today I tried a GV call from my iPhone and got “We could not complete your call - Please try again.” In my case this is a bug that happened because the GV number associated with the Google ID I was using did not have an associated mobile phone (I’d moved that phone to a different GV account). I think Google Voice.app for iOS used to permit that, but it doesn’t now. To use Google Voice.app now you need to have an associated verified mobile phone with the active GV account.

So I moved my two mobile numbers to the two GV accounts I wanted them on. Now if I try to use Google Voice.app with an account that doesn’t have a verified mobile number it gives me a more appropriate error message (that’s why this was a bug — wrong error message).

I’d be happy if GV would figure a way to make money from me, it is a great service.

- fn -

[1] Funny thing and cautionary tale: In the early 90s we were sure that by the year 2000 voice calls would be so cheap they wouldn’t be worth metering. That was only sort of true. Never underestimate the power of discontinuities.

[2] GV sort-of migrated to Hangouts, but that seems to have stalled and perhaps reversed. Hangouts seems to be dying, caught up in Google’s flailing messaging strategy.

Google sent my family into email Hell. This is how we climbed out.

… we’re living in a time when algorithmic software is just good enough to eat the world and still bad enough to be endlessly frustrating. (Daniel Genser, via Twitter)

A few days ago I sent an email to Emily and a few seconds later this came back:

Mail Delivery System <MAILER-DAEMON@homiemail-mx1.g.dreamhost.com>
Tue, May 8, 9:52 PM (4 days ago)

This is the mail system at host homiemail-mx1.g.dreamhost.com.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

…

                DreamHost Email Support

   <emily@googleappdomain.com> (expanded from <emily@redirectemail.com>): host
    ASPMX.L.GOOGLE.com[173.194.202 .27] said: 550-5.7.1 [208.97.132.209      12]
    Our system has detected that this message is 550-5.7.1 likely unsolicited
    mail. To reduce the amount of spam sent to Gmail, 550-5.7.1 this message
    has been blocked. Please visit 550-5.7.1
    https://support.google.com/mai l/?p=UnsolicitedMessageError 550 5.7.1  for
    more information. q3-v6si11849599pgn.272 - gsmtp (in reply to end of DATA
    command)

Google had sent me to email Hell. Everything I sent to Emily was rejected. It was the same story with emails sent to my children. Whatever I sent, Google rejected.

I think the problem, as far as Google was concerned, was that I was using redirects. This is old net tech. If you own a domain, say “kateva.org”, your hosting service will usually let you define as many redirect “email addresses” in the domain as you might want. They aren’t real email addresses though, they’re redirects. You have to define a true email recipient for each one.

Google didn’t see the email I sent Emily as coming from my personal gmail account, it saw it as coming from the SMTP service that works with my domain (which I’ll call “kateva.org"); in this case homiemail-mx1.g.dreamhost.com. It appears either my domain, or that sending service, had bad Google-karma [1]. Google sent the rejection to “homiemail” (I think!) and homiemail sent it to me.

We were in trouble. It was likely that email sent to our family members was going to be bouncing back in a very confusing way.

I tested sending from various email addresses and using various software (web client vs. iOS Mail, etc) to see which was rejected and which accepted:

Sender	Redirect	Result
GSuite (web client)	Yes	Success
Yahoo (web client)	Yes	Success
Gmail (web client)	Yes	Success
GSuite (macOS mail, iOS mail)	Yes	Success
Gmail (macOS mail, iOS mail)	Yes	FAIL
Gmail (macOS mail, iOS mail)	NO	Success

It turned out my mail was only hitting Google’s spam threshold when I sent it from my personal Gmail (not GSuite email) using iOS or macOS Mail. Everything else worked, including using my personal Gmail by web interface.

No, I don’t understand what’s happening here. I can speculate that Google rates sending services roughly as follows: GSuite_web > Gmail_web, Yahoo_web > GSuite_client > Gmail_client. The combination of Gmail_client (macOS Mail) and the redirect dropped my email score below Google’s internal cutoff.

There’s not much to do about this. It’s not like Google is going to help. I was on my own and we were in trouble.

The first thing I did was change the redirects to send all the mail to an old school Dreamhost local webmail box. That stopped the bouncing. I could forward from that box to my gmail and reroute important emails manually to other addresses for Emily and the kids.

Next I discovered, contrary to their documentation, that Dreamhost had not enabled DKIM or SFP on my domain. I fixed that (separate post pending) to see if it would improve the reputation score for my domain but it had no effect. I suspect the reputation that mattered was that of homiemail-mx1.g.dreamhost.com.

That left me with these options …

1. Setup Dreamhost mailboxes for each person and create redirects to them.
1. Set up a forward from the DH mailbox to another of our family GSuite emails (I picked several Google Apps/GSuite things when they were free)
2. Have our family GSuite emails do POP retrieval from the DH mailbox.

Dreamhost recommended the 1.2 (second). But I had a third option…

… even though I’d not used it, the problem domain (“kateva.org”) had an old legacy GSuite. I went into that GSuite, defined an account for Emily and kids, and then switched Dreamhost MX to use the GSuite email rather than DH email [2]. I had lost some faith in DreamHost by that point and I figured that since Google rules the net I was better off inside their castle.

And that’s where we are at the moment.

One more weird thing. For 2-3 family members I received a notice that a non-apps Gmail account already existed for them. I didn’t create those accounts but maybe my kids did? (It’s complicated, but somehow when I did Google Voice for them it did … something … brain shutting down ….) Apparently Google lets one do this! They wouldn’t have received email though, DNS records were using DreamHost MX redirects. If you own a domain, and create Google Apps account that matches the existing address, it gets renamed to something like “username%domainname@gtempaccount.com” email. Wow. [3]

It’s Google’s world, we just play in it.

Oh — and don’t use redirects. Google doesn’t approve.

- fn -

[1] Dreamhost denies that they’re in any kind of trouble with Google — but really, how would anyone know. Maybe it was my domain that Google didn’t like — we get a lot of spam and in this setup all that spam would seem to be coming from my domain.

[2] This wipes out all the past redirects. GSuite supported aliases so I moved them into there. Fortunately I’d saved the redirect records before making the MX switch.

[3] Google has an article on conflicting accounts, here is how they are resolved. I think Google Voice was the primary route for creating conflicting accounts — an artifact of how Google switched Voice credential systems post acquisition. 

Google's phishing vigilance and the risk of blog comments

I got a notice from Google yesterday. tech.kateva.org had been quarantined as a phishing vector. Visitors would be warned away. I had a short time to fix the affected page that was listed below …

But there was nothing there. Same thing with a link to a downloadable spreadsheet of issues. That was empty too.

Elsewhere in the message a page was identified. It sure looked benign, the only link was to an Apple support site. It’s a (Google) blogger site and all the widgets were default Blogger. No extra HTML. No comments.

Also, despite Google’s warning, the blog was not quarantined.

A puzzle.

Oh, I left something out. The day before a published spam comment showed up in my comment tracking feed. I get a few spam comments every day or two, but this was the first published one in a while. I went to delete it … but by the time I got to Blogger’s post management menu the comment was gone.

This is what I think happened:

• This blog was set to allow comments without approval from authenticated posters for posts less than two weeks old.
• A bot created an authenticated identity and created a phishing attack comment.
• Google spotted the comment, quarantined the site, and sent the notification email.
• Blogger spam detection identified the comment author as a spammer and deleted all comments by that identity — including the one on my site (why it was gone).
• Google rechecked my site and lifted the quarantine — but couldn’t retrieve the notification email.
• The notification email was partly empty because that it was a query — that returned Null. It had one part that was written at time of email generation and that contained the link to the once contaminated page.

I changed comments on the blog to require authorization at all times — no two week window for authenticated users. Clearly authentication is no longer a sufficient barrier. I don’t want phishing attacks on my blog, and I don’t want to get quarantined.

This reminds me what a strange fish Blogger is. It works fairly well, though there’s a longstanding problem with CR/LF handling that reminds me too much of DOS 2.1. It gets very few, but still some, updates. Google has switched their blogs off Blogger, but they haven’t used their new proprietary RSS/Blog platform to replace Blogger. Blogger is neither dead nor alive, and Google RSS is similarly quantum.

Selling or retiring an iPhone -- I hope you disconnected Google Authenticator and Google Prompt first.

Wipe that iPhone to give to your child or sell? I’m sure you remembered to launch Google Authenticator and remove the device from your trusted device list before you erased all …

Using a new phone to receive 2-Step Verification codes - Google Account Help

… On your old phone, open the Google Authenticator application…

Oh, you forgot about this step? You are clearly inadequate.

There’s hope though. Assuming you have a computer, there’s another option hidden away …

Add or remove trusted computers - Google Account Help

… Under “Devices you trust,” select Revoke all...

Except that’s not a bit misleading. There are more options once you sign into you Google Account and dig through the “Sign-in & security” section, select Signing in to Google, and Choose 2-step verification.

There you can remove the “Google Prompt” iPhone that manages authentication via Google App. That flips authentication to an Authenticator app (OTP authentication with RFC 6238 and 4226). The Authenticator app might be Google’s, a 3rd party, or 1Password or another password manager. I use Google’s because I started with it and I’m lazy.

You can also “change phone” on Google Authenticator. Authenticator is working for my new device though — which was restored from a 6s backup. So I didn’t do that immediately. 

Instead I removed the Google Prompt device, since that was still going to my (since erased) 6s. Then I added it back to my new iPhone 8. Google had the 8’s name so I authenticated there. 

Then, because I’m a paranoid sort, once I had Google Prompt working, I went into Authenticator on my i8 and removed my Google account then setup authenticator again from the 2-step verification page (scan barcode).

I’m sure everyone knows to do this. Otherwise why would pundits keep telling us to enable 2FA on every service we care about?

PS. I think when you authenticate within Safari for iOS Google can’t identify the host device. So my https://myaccount.google.com/device-activity list shows both “John8” (my current iPhone) and “Apple iPhone” — even though I believe they are the same thing.

PPS. I think if you want to be very careful you really should do the “Revoke all” as I suspect the old device Safari authentication can still be an issue (except I erased it, but if you’re paranoid …)

See also

• Things to do when selling or retiring an old iPhone 9/2015. Except its really worse than this … 

Apple drives me into Google's arms - using Google Photos with iPhone and Aperture

Apple has broken me. I’ve left iCloud Photo Stream shares for Google Photos.

First I lost the ability to share from Aperture to Facebook. I think that was probably a Facebook change, but of course Aperture isn’t getting updates any more.

That was annoying.

Losing Apple Photo Stream was much worse. Photo stream wasn’t great, but it was simple for my daughter, sister, and other users to subscribe to. For a time I could use iCloud Photo Library on Photos.app alongside iCloud Photo Streams on Aperture [1].

Then Aperture retched and I lost my shared photo streams (but not, happily, the originals). I played around with restoring iLifeAssetManagement from backup but, despite early promise, I couldn’t defeat Apple’s black box sync infrastructure [2].

That’s it. I’m toast. I surrender. Google’s inexplicable aversion to album creation on upload is the lesser evil now.

I’ve installed Google Photos on my iPhone and enabled backup and sync. I’ll use that to cull and play with photos before I transfer them to Aperture.

I’ve freed up 14GB from my Air’s SSD by deleting iLifeAssetManagement and I’ve installed Google Photos Uploader.app. I pointed that to a folder on an external drive, when I want to share from Aperture I export there for upload. I do my post-upload organization and sharing through the web UI.

Since Google nicely migrated images when it closed Picasa Web Albums my new shares are reunited with my old Picasa web albums. I’ve come home again. Though I’m still puzzled by Google’s weird album aversion.

It’s far from ideal, but Apple has burned me yet again. They seem to despise my data.

[1] Though I gave up on iCloud Photo Library when I realized it was more or less incompatible with importing images from iPhone photo roll to Aperture.

[2] Apple is famous for sync that disallows any kind of troubleshooting.

iLifeAssetManagement

How to upload images to a specific album in Google Photos

How to upload images to a specific album in Google Photos:

1. Create the album. You have to choose an existing photo to create it.
2. Drag and drop the photos you want to upload onto the album you’ve created.

Despite years of customer requests you still can’t select a folder and upload it into an album.

I’ve read that if you work with full res (not reduced) images in Google Drive you can organize them in folders and turn those folders into albums in Google Photos. These count against storage costs. I have not tested this.

Something wrong with Google's Device Activity page?

Google’s device activity page used to let me remove devices that I no longer used:

https://myaccount.google.com/device-activity

Today it lists 3 iPhones for me. I think they are all actually one device, showing up with two different names — none of them current.

There used to be away to remove these devices, but today I can’t. I am able to Remove Account Access, but not the devices.

I removed access to all devices, then reentered credentials on my phone. So now I know that ‘iPhone” is my phone (phone name is John6s).

I think something is broken somewhere….

Enhanced Google Security: Security Key and Password Alert

Google is tracking a curious uptick in government-backed (Russia, China, ?) attacks on journalist gmail accounts.

They are suggesting two security measures that are new to me - a security key and password alert.

Security Key is a USB dongle (FIDO Universal 2nd Factor) Instead of running Authenticator.app on your phone. It’s less vulnerable to man-in-the-middle attacks, but “Security Key does not work on browsers other than Chrome.” You can buy one from your favorite Chinese manufacturer on Amazon.

Password Alert is a Chrome app that tries to monitor for man-in-the-middle and phishing attacks. I’ve installed it in Chrome on my Mac. You have to trust Google to use it but if you’re using Chrome you’ve already made that commitment.

The Password Alert extension was part of a series of 2015 security enhancements. I’m surprised I didn’t hear anything about it.

Security Key may be newer, I couldn’t find much about it. I think Google is going to have to start selling these. Why would I trust a Chinese vendor?

Force Mac Chrome to open for a specific user profile - as of April 2016

I’ve been trying to get Chrome to open in one of my 15 user profiles since at least 2012. I have visited an open SuperUser question on this topic for four years. Today I added a method that work — for this moment:

As of April 2016 on Yosemite I was able to locate Google Apps for all of my 15 Google Profiles (yes, 15). Some were in ~/Applications/Chrome Apps, though we have been told these are going away [1].

To find all of them however I had to do this:

Go to ~/Library/Application Support/Google/Chrome.
In Spotlight search on kind:app
Find 300kb "Applications" with names like Gmail (username).
Copy the ones you want and rename them, then put in your Applications folder of choice.

All victory is transient, but Sisyphus is ahead most of the time.