Back in the 1990s, academics decided that “stereotype threat” depresses the performance of groups for whom society has lower expectations. For example, in experimental settings in which girls are reminded that boys tend to do better on math tests, those girls will then perform worse on a math test than girls who receive no reminder. The upshot is that education aimed at countering stereotypes could boost girls’ performance.
You may be thinking, “Gee, stereotype threat sounds like a simplistic and awfully convenient result that just happens to validate the left-wing conception of gender differences as socialized. Isn’t there some sort of ‘replication crisis’ going on right now that has overturned a lot of these types of findings?” If you’re thinking that, good for you. You have healthy instincts.
In 2014, Dutch academics Paulette Flore and Jelte Wicherts performed a meta-analysis of the stereotype threat literature as it applies to gender differences in math. They found a significant negative effect overall, but they also found evidence of “publication bias” — null results less likely to be published than significant results — and wondered whether stereotype effect has been overstated. “We propose a large replication study to provide a less biased effect size estimate,” the authors stated.
Now their large replication study has been released, and the conclusion should not surprise followers of the replication crisis: There was no evidence of stereotype threat. Boys did perform better than girls on the math test, as expected. However, the gender gap was not significantly different when girls were told that gender differences exist on the test compared to when they were told there were no gender differences.
This experiment has several advantages over the prior studies. First, it was one of the largest to date, with 2,064 students involved. Second, it was “pre-registered,” meaning the authors announced their methods before conducting the experiment. Pre-registration discourages researchers from torturing their data every which way before finding something they want to report. It also helps prevent publication bias, since pre-registered studies are expected to be written up regardless of the results.
This replication failure applies only to gender differences in math, not to the many other instances of stereotype threat that academics claim to have found. It should give everyone pause, however, to learn that such an oft-cited and long-accepted part of the literature is likely wrong. It’s another reminder that “I’ll wait for the replication” is a healthy reaction to any new study from the world of social psychology.