Most people, even here on HN, do not know what the DK effect actually claimed to show. It does not show that confident people are more likely to be incompetent. Their primary result shows a positive correlation between confidence and supposed skill. (What skill, you ask?*)
I don’t know which statistical artifact it is, but I am quite convinced that the so-called DK effect is not demonstrating something interesting about human psychology, I don’t buy that this is a real cognitive bias. I’ve read the paper several times, and the methodology seems to be lacking rigor. They tested a small handful of Cornell undergrads volunteering for extra credit, not a large sample, not the general population, and tested nobody who actually fits the description of ‘incompetent’ in a meaningful way. They primarily measured how people rank each other, not what their absolute skill was - and ranking each other requires speculating on the skills of others. There are obvious bias problems with asking a group of pampered Ivy League kids how well they think they rank.
* One of the four “skills” they measured was ability to get a joke - “appreciation of humor” - Huh? This is subjective! The jokes used aren’t given in the paper, either. Another was ‘grammar’ tests.
This article suggests DK is even simpler than autocorrelation, that it’s just regression toward the mean. https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...
I don’t know which statistical artifact it is, but I am quite convinced that the so-called DK effect is not demonstrating something interesting about human psychology, I don’t buy that this is a real cognitive bias. I’ve read the paper several times, and the methodology seems to be lacking rigor. They tested a small handful of Cornell undergrads volunteering for extra credit, not a large sample, not the general population, and tested nobody who actually fits the description of ‘incompetent’ in a meaningful way. They primarily measured how people rank each other, not what their absolute skill was - and ranking each other requires speculating on the skills of others. There are obvious bias problems with asking a group of pampered Ivy League kids how well they think they rank.
* One of the four “skills” they measured was ability to get a joke - “appreciation of humor” - Huh? This is subjective! The jokes used aren’t given in the paper, either. Another was ‘grammar’ tests.