I can’t recommend highly enough this “Numbers Guy” column in the Wall Street Journal today. Carl Bialik’s piece discusses the debate about the widespread use of automated polling in this election cycle, giving both advocates and detractors room to argue their points. Survey USA, Rasmussen, and other interactive voice response (IVR) pollsters are now generating a huge chunk of the survey numbers that political pros and pundits chew over every day, though some major-media outlets still shun them:
Their accuracy record in the primaries — such as it was — was roughly equivalent to the live-interviewer surveys. Each missed the final margin by an average of about seven points in these races, according to Nate Silver, the Obama supporter who runs the election-math site fivethirtyeight.com. “I think the networks are being snobbish and probably a little bit protectionist about their own polling outfits,” he says.
SurveyUSA, which pioneered these polls, has an impressive record for accuracy. The company ranks second among more than 30 pollsters rated by Mr. Silver. Its own report card shows it ranking at or near the top in predictive power for recent national election cycles. That may seem like a newspaper naming itself the best newspaper, but SurveyUSA is transparent in its ratings methods and competitors haven’t offered alternative ratings.
SurveyUSA founder Jay Leve launched the report cards to counter criticism of his methods. Yet, after conducting such polls for 16 years, he still finds himself defending them. That’s because, by policy, the national news divisions of CBS, NBC and ABC won’t air results from automated polls, even as many of their local affiliates sponsor and air SurveyUSA polls.
The Associated Press, the New York Times and the political publication the Hotline also exclude them. (The Wall Street Journal doesn’t have such a policy, according to a spokeswoman. Fox News, which like the Journal is owned by News Corp., also airs them.)
Scott Rasmussen, president of Rasmussen Reports, says that he believes that automated polls have arrived: “We get coverage on the cable networks and on the Internet, and that’s really what our game is.”
My own view is that IVR polling is here to stay, given the relative costs, and its practitioners have improved their performance significantly over the past decade, primarily because of growing sophistication in how they weight their samples to generate reportable results. It’s important to disclose, however, that Rasmussen used to chair my think tank’s board of directors.