My Google Custom Search just died. Did I offend the GoogleNet? (fixed)

Two days ago my much loved Google Custom Search was working beautifully (emphasis added) ...

Why coupons? Price concealment information and memetic archeology in the pre-web world 

... I found that reference through my pinboard/wordpress microblog/memory management infrastructure now integrated into my personal google custom search...

My latest enhancement was paying off; my ("free" = ad supported) personal custom search engine was now successfully indexing a blog that archived my pinboard.in shares and annotations [1] as well as my ancient web pages (archived) and my tech.kateva.org and notes.kateva.org blogs.

My extended memory was better than ever!

Until it died. [2] As of yesterday my custom search engine is returning very few results.

My first thought was that I'd unwittingly committed a Class One transgression against the GoogleNet. Perhaps Google considers my link/annotation blog to be a link-farm-equivalent -- and had blacklisted my entire kateva.org domain. Perhaps I had broken an unwritten rule of the GoogleNet (formerly known as the Internet, home of Archie and Veronica [3]).

I'm still able to find my notes.kateva.or and even kateva.org/sh posts in Google's standard search however (if I restrict by domain). So I'm not certain I've transgressed. If the search doesn't work soon I'll try recreating that engine. If that doesn't work, or if I detect more signs of transgression, I'll have to remove my pinboard archive and beg mercy of The Google.

I've a broken iPhone I could burn. Perhaps that will appease.
--
[1] It's my tawdry substitute for my long lost and much mourned Google Reader Share page. I hate the way it looks, but it's primary use is RSS consumption and index fodder. I am looking for a better template but WordPress themes/templates are a rats nest of complexity.
[2] Echoes of losing Google Reader Share!
[3] If you know what that means you either used Google or you are a very old geek.

Update 5/31/2012: It's back.

I followed some of advice that "omr" (not a Google employee) generously gave on the Google Search product forum. Instead of creating a new CSE however, I replaced many of the entries of the old CSE with the patterns he suggested. Perhaps most importantly, I changed the setting for indexing kateva.org/sh.

I'd previously opted to index all entries and all linked pages. Considering I add about 20-60 links a day I think that was a tad ambitious. I now index only the text of this shared items/pinboard (micro) blog.

For reference, here's an edited version of omr's recommendations:

In the "Sites to search" box, enter this URL Pattern:
  *.kateva.org/*
If you wish to include some of your other sites, enter additional URL Patterns to match them.  (Enter one URL Pattern per line.)  For example, if you want to include the msptrails site, add:
  *.msptrails.org/*
For more information about URL Patterns, see
Please include only a limited number of sites.  Start with the minimum number of sites that you consider necessary to include; or, if you wish to include several, preferably no more than ten.  (If you own some older or less-active sites that you don't need to search anymore, don't include them.)
Click the new CSE's "control panel" link (which takes you to the "Basics" page of the control panel).
Leave the "Search engine keywords" box empty.
Near the bottom of the page, note the "Show automatic thumbnail" option.  The automated thumbnail-image selection is not always ideal, so perhaps you may want to turn off that option.  (Click to remove the check-mark, then click the "Save Changes" button BELOW the option.)

The mysterious iTunes -100000 error: This file cannot be downloaded - Fixed

I think I need to sacrifice a goat to the spirit of the Nameless One. How else can I escape the curse of really weird Apple bugs?

This bug isn't quite as weird as the Apple ID vs. MobileMe verified forwarding email bug, or even the mysterious Image Capture Slowdown, but it's bad enough. 

The other day an iTunes track download failed. First time I'd seen that. So I tried with iTunes. It made 3 tries and failed there too; the download list showed an error code of "-100000" and iTunes told me:

There was a problem downloading ...

The file seems to be corrupted. To redownload the file, choose "Check for Available Downloads" from the Store Menu.

I tried several tricks including changing user accounts, trying it on another machine from my home network, trying it again from my iPhone, trying it on AT&T's network, etc. Of course I restarted my machine, different version of iTunes, etc etc.

Nothing worked. 

I tried downloading to my wife's phone. Everything downloaded ok except for that one Arcade Fire track: "Neighborhood #1 (Tunnels".

So I went through Apple first tier support, then to 2nd tier. They reset my password so they could try downloading -- and it apparently worked for them. So they washed their hands of me.

Of course it doesn't work for me, no matter the device. The one track is bad.

So what's going on? Well, I'm not the only one. There's a clue in an iTune's Discussion:

...I  also tried all of the suggestions to no avail.  As an experiment this weekend I tried accessing the "corrupt" file from my daughter's  network in the Chicago area.  Logged in, downloaded, bam. Worked like charm. I have no security software on my home network, but by golly I worked from another network...

I bet this is a geography problem. The file is corrupt on a server that manages my geographic area.

One way to test this would be to travel and try it.

Another would be to connect via VPN to a different region and then retry.

It's late for me to try that tonight, but Dreamhost (California) does support SSH connections. Tomorrow I'll try that one ...

Update 5/23/12: VICTORY. I connected via SSH Tunnel to Dreamhost then downloaded again (clicked Buy). It worked instantly. This supports the theory that an instance of the tune was corrupted on the iTunes server for my geographic region (Minneapolis St Paul).

Apple ID Hell: Apple's identity mismanagement and an insanely obscure ID bug with MobileMe send as email

Apple has long had trouble with truly basic concepts of authentication, but lately they've gone off the rails.

It's not just the new and quite demanding Apple ID password requirements that will lead most customers to put them into iPhone Notes -- since it must be entered each time you buy something.

It's not just the bug in their password instructions; when they say "no more than 3" repeating characters they mean "no more than 2" repeating characters (test it yourself).

It's not just that Apple prevents one from seeing the ugly and unmemorable Apple ID that we have to type in -- while also disallowing copy/paste into the password field (take that password manager!).

It's not just that Apple's Apple ID maintenance UI wants me to change my primary email address to match my .mac ID -- which doesn't have email.

No, it's even worse than all that.  The big problems are the proliferation of Apple IDs, and a bug that hit my very old Apple ID.

By trial and error I've discovered I have no less than four distinct Apple IDs.

I have an Apple ID that matches an old MobileMe account I abandoned years ago.

I have an Apple ID that matches my current MobileMe account. When I changed its password I also changed my MobileMe account password.

I even found an old developer account that is now an Apple ID too. (Apple merged these databases). So that's #4. It has yet another email address of mine and so I verified that. 

Lastly I have the Apple ID that is associated with hundreds of dollars of purchases. That's the one I care about -- and that one has two email addresses associated with it. That's critical for password security. One of the email addresses is my personal Gmail address.

Except ... neither of them are "validated". That's bad.

Apple won't let me validate them either. It claims both are associated with other Apple IDs.

One of them is my current MobileMe email. Since I've discovered that account morphed into yet another Apple ID, that makes sense. I removed that one and added a new email that I was able to verify (I own my own domains, so it's easy to make email redirects).

Then I tried removing the Gmail address and adding it back in. Uh-Oh, that's no longer allowed. It's been orphaned.

I check each Apple ID in turn, including my mother's and the kids. The Gmail address is not associated with any of them. So is there a fifth AppleID somewhere? Is there a bug? Is there a security breech?

There's more. After I changed the email address and password for my mac.com Apple ID, Apple enrolled me in iCloud for that account! So now it has an email address too?

I'm beginning to understand why Apple's share price is relatively low. If they can't manage something this basic, they're in trouble.

Apple, if you need a clue, here's a few:

1. We need a way to delete Apple IDs. Unwanted IDs are a security risk.
2. Or we need a way to merge Apple IDs!
3. You need to fix my gmail bug Apple.  If #2 is not a bug, you need a way to reclaim that address (does someone else have it as "unverified"? Is that part of how iTunes accounts are hacked?

See also:

• Apple's messed up Apple ID system: what are they smoking?! 7/2008 (!)
• Apple's identity and account system is screwed up. Again. 10/2008
• Apple is again messing with Apple IDs – pay attention! 6/2009

Update: I've figured out the bug. It arose as a side-effect of changes to the way Apple IDs work, and it only impacts people who are still on MobileMe accounts and who have the same email address associated with two Apple accounts prior to the time Apple made that illegal. In brief:

1. The Gmail address was a validated 'send as' forwarding address in Mobile Me Mail.
2. The Gmail address was the primary email on my mac.com Apple ID.
3. Then Apple created an Apple ID for Mobile Me email. Somewhere in Apple's databases the Gmail 'send as' address became an alternate email for the new Apple ID. HOWEVER, it was already a primary email for another Apple ID. So it didn't display in the AppleID maintenance screens for my Mobile Me account.
4. Since it existed in the database, however, it couldn't be validated in my .Mac account.
5. When I removed it from my .Mac account I wasn't allowed to add it back, since it was associated with my MobileMe account -- but in a partial state there.
6. It couldn't be located using my First and Last name in the Apple ID locator page because it wasn't properly associated with the MobileMe account. However, Apple's 2nd tier phone support was able to see it there.
7. I went to the MobileMe account and added the email manually. It suddenly appeared -- as verified! I removed it, but Apple Support confirmed it didn't remove completely. So I left it "verified" with the MobileMe account. Now I know where it is.

Apple is closing MobileMe, so there's no way they'll ever fix this bug. I'm glad I seem to have fixed it however, it might have screwed up my iCloud migration. I think before I go to iCloud I'll remove that forwarding email address.

Postscript: Apple's Support Profile is supposed to show the products associated with my Apple ID. I think it used to. I don't see them any more. It says my home number is associated with a different Apple ID...

Image Capture hangs at "Scanning", process pegs at 100% - ghost iPhone and dust removal

I've used my Epson V700 with Image Capture for years. Until today it's never had a problem.

Today it has a problem!

My scans are taking a very, very long time to complete. They seem to hang after the scan is completed, but after about five minutes they complete. It's as though something is timing out. In Activity Monitor the Epson Scanner Process is pegged at 100% CPU.

I switched to another account and the scan completed in seconds. So it's not hardware. It's something about my primary account.

A restart didn't fix it and deleting Image Capture preferences didn't help.

The problem started after several scans, so it's not related to a software install. The /private/tmp folder that Image Capture uses to cache its work seems fine.

Next up I'll try Onyx and clear out all my caches.

It's a weird one.

Update: I've identified one difference. On my primary account Image Capture thinks an iPhone is attached, even though it isn't. I found one other report of this in Snow Leopard and another here.

Update 2: I got rid of the "ghost" iPhone camera by removing all USB peripherals and restarting. That didn't help however. Scanning in my primary account is still far slower than scanning in my admin account. I even tried repairing permissions; as usual that made no difference. I started an Apple Discussion thread on this.

Update 6/9/2012: There were two issues. Removing the "ghost" iPhone did make a difference. The rest of the difference was dust removal. I was used to having it turned off. Turning it on really extends scan duration. Perhaps it's on by default and removing preferences enabled it?

Finding partly played podcasts: In Our Time

iTunes tracks how many times tracks have been completely played [2], but it doesn't provide any UI for tracks that have been partly played.

This isn't a big deal for music or videos, but it's a real bother for podcasts. I have 72 episodes of "In Our Time" in my "IOT unplayed" shortlist; and I know some of them are partly played. I just don't know which ones. 

It's been a longstanding frustration, but this week, Doug's AppleScript for iTunes gave me the answer:

Project: Gather Partially Played Tracks « Doug's AppleScripts for iTunes

... Smart Playlists don’t include any criteria for detecting how far along a track has been played, and Last Skipped may not necessarily have been set if a track was simply stopped rather than skipped.

... a track’s bookmark property will contain the number of seconds the track had been played before it was stopped. Thus, if any tracks have a bookmark value greater than zero then they’ve been partially played...

I'm lousy at AppleScript, but it wasn't hard to modify the example Doug provided. I have a smart Playlist called "IOT Unplayed" and I modified the AppleScript to find tracks in that Playlist that had a bookmark value greater than zero [3]:

property nameOfPlaylist : "Partially Played"

tell application "iTunes"
	set opt to button returned of (display dialog ¬
		"Find partially played tracks in:" buttons ¬
		{"Cancel", "Podcasts", "IOT Unplayed"} default button 3)
	
	if opt is "Podcast" then
		set targetLibrary to (some playlist whose special kind is Podcasts)
	else
		if opt is "IOT Unplayed" then
			set targetLibrary to some playlist whose name is "IOT Unplayed"
		else
			set targetLibrary to library playlist 1
		end if
	end if
	
	try
		set thePlaylist to some playlist whose name is nameOfPlaylist
	on error
		set thePlaylist to (make playlist with properties {name:nameOfPlaylist})
	end try
	
	try
		delete every track of thePlaylist
	end try
	
	duplicate (every track of targetLibrary whose bookmark > 0) to thePlaylist
	reveal thePlaylist
end tell 

Of course this isn't as elegant as a Smart Playlist -- I need to run this AppleScript manually. Even so, it solves 80% of my pain. Thanks Doug!

[1] I donated $5. It was a royal PITA to do so. It reminded me of how bad our donation system is. Among other things:

• I distrust PayPal intensely based on past experience. So I had to use the awkward data entry UI.
• It took me a while to figure out I had to 'update amount' - so that cost me two data entry episodes
• I had to go back and add an email
• If I could keep track of how much I'd previously donated to the site, I might make a larger (or smaller) donation.

Sucks.

[2] I have a feeling that years ago iTunes would consider a track "played" if one simply started listening to it. I'm probably wrong about that.

[3] In my hacked AppleScript the 'library playlist 1' statement will never be reached; I just kept it in as a reminder of how to scan all tracks. AppleScript only allows 3 buttons, otherwise I'd have made this an option.

Navigating AT&T mobile's web site: a cheat sheet

I have a hard time navigating AT&T's mobile site. So I wrote this post to store some of the links I use. I'll update it periodically. Some links may embed one of our phone numbers, so they're not so useful for others.

In general I login first, then copy and paste these links to navigate.

• SMS blocking options, esp. email management
• Data Usage on key number (filter to show text/im)
• Manage/review features: how to see what's currently on your phone. It's not a very useful summary.
• Bill Summary: includes current charges
• Device Management: change wireless number ($36), reset voicemail password, suspect/reactivate, unblock SIM card
• Send a text message (free to send, is sent from own number)
• Marketing preferences: turn off spam from AT&T and "partners"
• User Information: including link to customer service summary, contract and contract end date
• Support: 611 from phone (best) or live chat agent or phone support

Google results 100% ads?!

I did a search on "open source document management".

My result page was 100% advertising.

I've never seen this before. I'm hoping it's a bug of some kind. I tried opting out of Google personalized ads to see if that helped. Next up I'll just block every advertiser on the page using Ad personalization.