I had a kid -- like, just-out-of-high-school kid -- call me "sweetheart", once, at a restaurant. I was in a foul enough mood already (travelling for work does that to me) that I told him upfront that he'd best get another waiter to work my table, because he was twenty years too young to talk down to me.

I still rankle at "ma'am," sometimes, but I've gotten to the realization that I'll still take a dozen ma'ams over any infantilizing diminutive.