I think a method of ranking news based on the number and quality of comments would be better than voting up and down. This method would be somewhat analogous to page rank in that the rank of a story (webpage) is indirectly determined by the comments (links) and quality of the comments. Writing a comment is arguable less reflexive than clicking an arrow.
Also interesting news that was generating a good discussion could tend to persist and stay near the top. This would encourge thoughtful comments as you are not penalized for taking some time to think about the matter rather than hastily shooting from the hip.
The problem with this is firstly that it creates a feedback loop, where the more commented something is, the higher it climbs, the more commented it is...
Secondly, it'd be hard to extract a quality measure from comments alone - there are a number of metrics (number of unique users commenting, rate of commenting, amount of voting...) but how do you get from that to a quality measure? How do you distinguish controversy from quality? Or comment threads full or jokes and snark?
Also, commenting on anything you disagree with or dislike would result in that link getting a boost, which can't be right.
All that said, I think comments could be used, but modestly, as part of a bigger algorithm.
But a feedback loop is exactly what is desired. The hard part is to amplify the signal while suppressing the noise. Up/Down voting also creates a feedback loop that I would argue provides a very noisy signal of quality at best and an irrelevant signal at worst. The thesis under consideration is whether comments can provide a better signal corresponding to the item commented on. If this is true, it is desirable that items commented on climb higher and generate more comments,
If you consider comments to be analogous to links, the quality of a news item (or comment) can be extracted as easily as pagerank determines the quality of a webpage. Controversy can not be distinguished from quality because they are orthoganol measures. Many controversial issues (to VC or not to VC) are of high interest to readers of this site and would be considered high quality while off topic political controversies would probably not be considered high quality for this site. In general, most interesting things are controversial.
The existence of noisy comment threads (jokes and snark) is a good point. I would think that some form of Bayesian spam filtering could penalize these comments. Off topic comments could be penalized similiarly if a Bayesian filter was trained using the text of the news item.
I disagree that disagreeing or disliking is disagreable. I would call it conversation.
I don't think a bigger algorithm is the answer. One with a good impedance match would suffice.
I've never heard of anything like this, though I've thought of it quite a long while back for Reddit. The problem is that it won't solve anything. Most people will still downvote / leftvote things they don't agree with and upvote / rightvote things they do agree with. If the desire to silence a dissenting opinion could be controlled by the majority of people, free speech would not be such a rare thing, let alone need to be written into law.
I agree. That is why I purposely did not equate quality with up and down voting, but left it undefined. Perhaps a solution could be that the quality of a comment was ranked by the number and quality of sub-comments. We would have the beginning of a highly ranked comment thread, for example.
Also interesting news that was generating a good discussion could tend to persist and stay near the top. This would encourge thoughtful comments as you are not penalized for taking some time to think about the matter rather than hastily shooting from the hip.