The problem with this is firstly that it creates a feedback loop, where the more commented something is, the higher it climbs, the more commented it is...
Secondly, it'd be hard to extract a quality measure from comments alone - there are a number of metrics (number of unique users commenting, rate of commenting, amount of voting...) but how do you get from that to a quality measure? How do you distinguish controversy from quality? Or comment threads full or jokes and snark?
Also, commenting on anything you disagree with or dislike would result in that link getting a boost, which can't be right.
All that said, I think comments could be used, but modestly, as part of a bigger algorithm.
But a feedback loop is exactly what is desired. The hard part is to amplify the signal while suppressing the noise. Up/Down voting also creates a feedback loop that I would argue provides a very noisy signal of quality at best and an irrelevant signal at worst. The thesis under consideration is whether comments can provide a better signal corresponding to the item commented on. If this is true, it is desirable that items commented on climb higher and generate more comments,
If you consider comments to be analogous to links, the quality of a news item (or comment) can be extracted as easily as pagerank determines the quality of a webpage. Controversy can not be distinguished from quality because they are orthoganol measures. Many controversial issues (to VC or not to VC) are of high interest to readers of this site and would be considered high quality while off topic political controversies would probably not be considered high quality for this site. In general, most interesting things are controversial.
The existence of noisy comment threads (jokes and snark) is a good point. I would think that some form of Bayesian spam filtering could penalize these comments. Off topic comments could be penalized similiarly if a Bayesian filter was trained using the text of the news item.
I disagree that disagreeing or disliking is disagreable. I would call it conversation.
I don't think a bigger algorithm is the answer. One with a good impedance match would suffice.
Secondly, it'd be hard to extract a quality measure from comments alone - there are a number of metrics (number of unique users commenting, rate of commenting, amount of voting...) but how do you get from that to a quality measure? How do you distinguish controversy from quality? Or comment threads full or jokes and snark?
Also, commenting on anything you disagree with or dislike would result in that link getting a boost, which can't be right.
All that said, I think comments could be used, but modestly, as part of a bigger algorithm.