It destroys people's livelihoods. Regular workers in Vancouver have to pay an absurdly high percentage of their income towards their mortgage payments, because of outside investors that jack up prices and provide no economic growth to the city.
We don't seem to be on the same page. My argument (restated somewhat) is that the ownership controls keep the value distributed among the residents, instead of allowing it to be captured by an outsider (or one insider or whatever).