Psychologists argue that the extent to which someone can accurately assess their own ability in some domain depends on their ability in that domain. Perhaps people who lack ability in a particular area don’t even know enough to know how deficient they really are. Named after the researchers who first developed this hypothesis, this phenomenon is often called the Dunning-Kruger effect. The typical evidence offered in favor of the Dunning-Kruger hypothesis is shown in the figure below. Subjects were first asked to predict their own IQ, and they later took an IQ test. The figure shows a scatterplot of these two scores for each subject. The regression line is shown in gray, and the dashed, 45-degree line is in black. People with low IQs (as measured by the test) tended to over-estimate their score, and people with high IQs (as measured by the test) were, on average, correct.
(a) Can you come up with another explanation for this empirical phenomenon that wouldn’t necessarily imply that high-IQ people are better at assessing their own IQs? You’ll want to think about mean reversion, but that alone won’t do the trick since the high-test-score people didn’t under-estimate their IQ. Perhaps you’ll also want to think about bias. Remember our favorite equation. (b) To check your intuitions, simulate some data on your computer that generates a similar result even though the assessments of high-ability people are just as noisy as those of low-ability people. (Hint: It will help to remember that the test-assessed IQ is not a perfect measure of true intelligence.) (c) Download “IQdata.csv” and the associated “README.txt,” which describes the variables in this data set, at press.princeton.edu/thinking-clearly. These are the data used to produce the figure. (We obtained the data from a 2020 study by Gignac and Zajenkowski.) Let’s think about how we can assess whether high-IQ people really are better at assessing their own IQs. i. First, compute the absolute value of the error for each subject (that is, how far off were their self-assessed IQs from their test-assessed IQs?). ii. Now, regress this absolute error on the test-assessed IQ and interpret the results. (d) As you can see in the figure, people tend to over-estimate their own IQs. i. Estimate the average extent of this bias in this data. ii. Subtract this estimate of the average bias from everyone’s self-assessed IQ to get a bias-corrected self assessment. iii. Using this bias-corrected self-assessment, recompute the absolute value of the errors—that is, calculate how far off on average, a person’s bias-corrected self assessment is from their test-assessed IQ. iv. Finally, regress this new measure of error on the testassessed IQ and interpret the results. (e) Provide your final assessment of the Dunning-Kruger hypothesis on the basis of this data. Are high-intelligence people better at assessing their own intelligence?
