Thing was, the phone was in silent mode (red dot on the ring/silent switch). It's not supposed to chime. It wasn't ringing when I tested however, so I knew the switch was working.
I shut down and restarted the phone and changed the text message sound. Of course power cycling makes sense, but the I had a reason to change the text message sound too. That comes from my OS X experience; sometimes changing settings will fix a corruption problem.
That worked. The phone no longer chimes when in silent mode.