Census Bureau: Sorry, We’re Going to Have a Hard Time Measuring Whether Obamacare’s Working

The U.S. Census is changing the survey it uses to assess how many Americans have health insurance, beginning this year, meaning that it will be extremely difficult to use their data in assessing the effect the Affordable Care Act has on insurance coverage. The Census’s new survey, tested out last year, finds lower rates of uninsured than the earlier method, apparently. The Times reports:

An internal Census Bureau document said that the new questionnaire included a “total revision to health insurance questions” and, in a test last year, produced lower estimates of the uninsured. Thus, officials said, it will be difficult to say how much of any change is attributable to the Affordable Care Act and how much to the use of a new survey instrument. 

“We are expecting much lower numbers just because of the questions and how they are asked,” said Brett J. O’Hara, chief of the health statistics branch at the Census Bureau.

With the new questions, “it is likely that the Census Bureau will decide that there is a break in series for the health insurance estimates,” says another agency document describing the changes. This “break in trend” will complicate efforts to trace the impact of the Affordable Care Act, it said.

The Census Bureau’s Current Population Survey asks questions about health-insurance status and, unlike the decennial census, is gathered constantly, from a small but acceptable sample of the U.S. population. It’s one of the surveys the Bureau of Labor Statistics uses to calculate the monthly jobs numbers, but health-insurance questions are “supplemental” queries, asked only in the spring. The old survey, essentially, just asked whether people had coverage at any point in the year; the new survey will ask them if they had it, when, and for how long. There are private data on insurance rates, there will be plenty more private research on it, and the CDC does a survey of its own, but the Census’s change is a big problem. (On the other hand, Josh Barro points out on Twitter that 16 House Republicans have co-sponsored a bill to prevent the Census Bureau from carrying out the Current Population Survey at all, so it could be worse.)

The redesigning of the health-insurance survey has been in the works for years — since well before Obamacare was passed — but the timing of this full-scale rollout is absurd, and it’s unclear why it can’t just wait. Plenty of people will detect political meddling in an ostensibly nonpartisan federal bureaucracy here, but this looks more like a tale of the total indifference federal bureaucrats can have for the needs of policymakers and the American people.

“It is coincidental and unfortunate timing” for the change, a paper from the Census Bureau says. “Ideally the redesign would have had at least a few years to gather base line and trend data.” Yes, ideally.

UPDATE: Sarah Kliff of Vox reports that the White House argues this isn’t going to be a problem, since 2013 and 2014 data — before and after the implementation of Obamacare’s biggest policy changes — have been and will be collected with the old method.

The White House and Kliff are partly right: The data the Census will use to determine rates of health insurance for 2013 (roughly) were gathered this March and April, with the new method. Next spring, the Census will ask people whether they have insurance, and whether they had it over the preceding year. So that should provide comparable data, one year before Obamacare and one year after (if people this spring accurately answer whether they had health insurance in 2013, which the Census thinks they can, based on years of testing).

It cannot be compared to the 2012 data, gathered in the spring of 2013, which asked a simpler, though less interesting, question (whether people had insurance at any point). People won’t be totally bereft of comparable pre- and post-Obamacare Census data, but its usefulness is watered down by the fact that the Census picked this moment to move to a more informative survey question. The New York Times wasn’t making up these issues, that’s what Census officials told them. The fact that we have just one year’s worth of data before Obamacare to use is a real problem. It’s possible, though, some Censors thought having one year of the more informative survey will provide a better way to assess Obamacare than having years of comparable data from the old method, and that’s why the decision was made to go ahead with the change now.