It’s Not Researchers’ Fault Liberals’ Favorite Pre-K Study Is Antiquated, But It Still Is

The Carolina Abecedarian Project is one of the few early intervention programs to show cognitive and behavioral gains for participants several decades running (it’s often cited along with the Perry preschool program). The latest results, published in Science last month, show superior adult health outcomes in the treatment group.

But can a small program started way back in 1972 really hold lessons for today’s America? Echoing the sentiment of most preschool advocates, Aaron Carroll at the Incidental Economist answers this way: “Anytime you do a follow-up of 30+ years, by definition the intervention will be old by the time you get results. There’s no other way to do it. It’s such a silly attack.”

It’s not silly at all — research problems shouldn’t be dismissed just because they are unavoidable! Critics are not demanding to see the results of a 30-year study that somehow began yesterday. Their point is that the long-term nature of preschool analysis inevitably complicates its interpretation and calls into question its modern applicability.

Abecedarian was begun during the Nixon administration. Few people would argue that the life of the typical poor family in the U.S. is the same today as it was then. To take one example, nearly all participants in the original Abecedarian project were black, but if we implemented a new Abecedarian today, more Hispanics than blacks would be likely to participate. The socioeconomic challenges facing low-income immigrant families in 2014 are probably quite different than those facing a segregated group of slave descendants in 1972.

For pedagogical clarity, imagine a 75-year follow-up to a small program that served kids born before World War II. Most of us would question how relevant that program is to public policy today. I doubt anyone would respond to that objection with, “Oh, how silly. The program has to be 75 years old if you’re going to do a 75-year follow-up!”

It’s frustrating that ongoing studies become less relevant as time goes by, but they still become less relevant as time goes by. In my view, uncertainty in long-term interventions should be treated much like uncertainty in finance. A guaranteed $100 payment is worth more than an expected $100 payment that might end up being $90 or $110. Similarly, expected gains from preschool evaluations should be devalued as the uncertainty around them increases.

On a side note, Abecedarian’s age is not its only interpretive challenge. Back in the early 1990s, psychologist Herman Spitz pointed out that the treatment group exhibited cognitive advantages as early as six months of age. To supporters, this indicates that Abecedarian works its magic very early in life, and the subsequent pre-K intervention years are necessary to sustain the advantage.

But to skeptics, an early cognitive advantage is evidence that the treatment and control groups were never randomized properly in the first place, invalidating subsequent comparisons. Problems with randomization are not unusual in tiny demonstration projects, and there were some inconsistencies in participation rates between Abecedarian’s treatment and control groups.