By now you very likely know about Kevin Drum’s fascinating article arguing that changing levels of childhood exposure to lead explain the massive rise and subsequent decline in U.S. crime rates over the past half-century.
Kevin Drum is a smart and sensible guy. There is very strong biological evidence that pumping a bunch of lead into kids’ bloodstreams will cause brain problems. And it seems commonsensical to me that more kids with more lead in their blood, and therefore more brain problems, will cause, all else equal, more crime. So far, so good.
The argument in the article is that we have an analytical understanding of the link between lead and crime sufficient to justify a $400 billion expenditure to reduce environmental lead.
Drum first presents a pair of curves for lead and crime that are visually striking. But Drum asserts (with good reason) that these are not definitive. He then cites his key econometric source for establishing and quantifying a causal link between lead exposure and crime:
What’s more, a single correlation between two curves isn’t all that impressive, econometrically speaking. Sales of vinyl LPs rose in the postwar period too, and then declined in the ’80s and ’90s. Lots of things follow a pattern like that. So no matter how good the fit, if you only have a single correlation it might just be a coincidence. You need to do something more to establish causality.
As it turns out, however, a few hundred miles north someone was doing just that. In the late ’90s, Jessica Wolpaw Reyes was a graduate student at Harvard casting around for a dissertation topic that eventually became a study she published in 2007 as a public health policy professor at Amherst. [Bold added]
So what do you see when you go to the Reyes paper?
She puts forward a regression model that she says is “the best way to determine the relationship between lead exposure in childhood and criminality in adulthood.” This model purports to show a causal relationship between lead exposure and violent crime. But here’s a funny thing about her regression: It also shows no statistically significant relationship between lead exposure and property crime, and no statistically significant relationship between lead exposure and murder. This is extraordinarily counter-intuitive. Loss of IQ (and hence earnings power and foresight) plus loss of impulse control makes me more likely to assault people, but not more likely to kill them or steal from them? Anything’s possible, but that sure undercuts the intuition behind the case. Either the impact of lead on crime is incredibly specific to non-property crimes other than murder, or we have a problem with the measurement method.
I think the problem is with the measurement method.
Basically, the regression model looks at those states that were earlier in phasing out leaded gasoline, and then sees if they subsequently experienced falling violent crime rates earlier than those states that were slower to phase out leaded gasoline. The obvious problem is that it’s possible that the early-phase-out states might have differed systematically from late-phase-out states, and these systematic differences could also have led them to have systematic differences in subsequent changes in crime rates. An illustrative example is that richer states might phase out earliest, and these might be states with a lower propensity to crime increases 20 years later because of their wealth, not because they phased out lead earliest.
There are two complementary approaches that Reyes takes to try to control for these potential systematic differences.
The first is that Reyes claims that this isn’t really such a problem, because the way regulations mandating unleaded gasoline were implemented meant that the sequence of which states went early versus late was “largely random.” Analytically speaking, this would be ideal. How does she know this? Here’s the crucial passage in the paper:
The variation resulted not from state government policy or state-specific EPA policy but rather from a variety of features of the petroleum industry. The network of petroleum pipelines delivered gasoline with different lead contents to different regions of the country. Even within a region, the lead content of different grades of gasoline (regular, midgrade, premium, superpremium) differed significantly (by as much as 50%). Demand for the different grades of gasoline also varied with consumer preference and with the age of the stock of cars (which also varied with climate). Even the number of gasoline pumps available at gas stations affected the path of the introduction of unleaded gasoline, and particularly the phase-out of high-lead premium gasoline between 1979 and 1980. Thus, grams of lead per gallon appears to have experienced substantial and largely random reductions in the period 1975 to 1985, reductions that varied significantly from state to state and that were indirectly induced by EPA policy. [Bold Added]
But which states have what consumer preferences for mix of car types — think states with lots of American pick-ups versus states with lots of Toyota sedans — is very likely correlated with differences in political economy that in turn will affect changes in crime rates over decades in material ways. Climate — for example, who chooses to live in the Sunbelt versus the Upper Midwest — is similarly a confounding factor. Number of gasoline pumps at stations is highly related to land costs, road networks, land-use regulation, political strength of dealer lobbies, and other factors highly related to political economy. Age and stock of cars, ditto. And so on with everything else she lists (and many she doesn’t). These are all confounders that we do not have any reason to believe are largely randomly distributed between the early-phase-out versus late-phase-out states. In fact, it would be counter-intuitive that there are not systematic differences in political economy between the kinds of places that reacted quickly versus slowly to such environmental regulations.
What Reyes has really argued in this passage is that the relationship between timing of phase-out and other possible unobserved causes of future changes in crime rates is complicated, not that it is random.
The second method she uses to try to hold all other factors constant is to include terms in her regression equation. For reasons I have gone into in detail in Uncontrolled, putting a term for “Amount of beer consumed per capita,” or even a catch-all like “state is Montana” in a regression equation doesn’t really mean we have “held the effect of that factor constant” across states, even though people will often say this causally.
But, for the sake of argument, let’s assume away this and all other of my objections so far. That is, for the moment, let’s assume that (1) we give Reyes that it is okay to find a causal link for lead and violent crime that does not appear for either property crime or murder when using the same measurement method (which seems extremely counter-intuitive), and (2) give her that she has largely random assignment of states to early and late phase out (which is not demonstrated, and is also counter-intuitive), and (3) further give her that she has controlled for other systematic fixed differences between states by including “state-fixed effects” (which is not a reliable methodology) — only then would we would get to her “main result” of a statistically significant relationship between lead and violent crime.
But even under all these unrealistic assumptions, what this model hasn’t accounted for is that the evolution of political economy over time during these decades could systematically vary between late- and early-phase-out states. This could easily be the case, if the evolution of, say, the political economy of Sunbelt states versus Rustbelt states evolved systematically differently over a time frame of many decades, and Sunbelt states tended to phase out leaded gasoline either earlier or later than Rustbelt states. In that case, you would have to consider each state/year combination as a control rather than just each state. Think this is an obstructionist objection? Reyes herself considers this possibility significant enough that she does this analysis. The result? No statistically significant relationship between lead and violent crime.
Reyes seems like a diligent and thorough analyst, and she has an admirably clear prose style. But this regression model is reading tea leaves. The problem is not with her, but with the econometric method she is using to try to tease out causality. It is like using a child’s magnifying glass to try to investigate the structure of a skin cell.
Drum has made clear that his purpose in doing this article was to get further research and attention on the topic. He has succeeded, and I think it a worthy goal. Before reading his article, I had the intuition that lead exposure should have some effect on crime. Reading the article strengthened this belief. I think it should strengthen this belief in any rational person who has not previously seen this evidence. But that is way short of making a convincing case for spending $400 billion of taxpayer money.