It's painful, but you do get used to it.

Or just... wear your headphones all the time, like I do. Would be nice if people didn't do that, but, well.