There is a fairly large proportion of people who would gleefully restrict certain types of speech that they disagree with.

Though it is (probably) not their intent, that doesn’t make that speech go away, but rather moves it out of the reach of sunlight.