Though the Quinnipiac and Marist polls had Bill Thompson down by double digits during the week leading up to the 2009 mayoral election, the candidate’s own internal poll showed just an eight-point gap—and momentum in his favor.
“We knew differently,” Thompson said of the projections in the race, which he ultimately lost by just five points.
Did he really know better? Consultants and campaigns frequently dismiss public polls and the methodology behind them, but public pollsters counter that campaign polls are the ones that should be viewed skeptically. So who’s right?
Each side can point to examples to bolster its case. The Thompson campaign’s poll was more accurate than Marist’s and Quinnipiac’s, but a SurveyUSA poll trumped all three.
On the national level Gallup was widely criticized for its inaccurate predictions in the 2012 presidential race, including its projection that Mitt Romney would win. But internal polling for Romney, which had given his campaign a false sense of security, wasn’t any better. Meanwhile, Nate Silver, who writes The New York Times’ FiveThirtyEight blog, analyzed public polls to correctly predict the presidential winner in all fifty states last fall.
Campaign pollsters claim that their methodology is superior because they use registered voter files that include the names and phone numbers of people who have actually voted or registered to vote. By contrast, public pollsters typically use random dialing to get a representative sample of voters while relying on respondents to verify that they are registered voters and, in some cases, share whether they plan on voting in an upcoming election.
Nick Gourevitch, a senior vice president and director of research at Global Strategy Group, said that his firm looks closely at past voting history and turnout trends to figure out who will make up the fraction of registered voters who are most likely to vote. The firm is staying out of the 2013 Democratic mayoral primary in New York City because it has worked in the past with several of the candidates.
“This particular race has a little bit of nuance because it’s likely to be more high-profile and higher turnout than New York City primaries of years past,” Gourevitch said. “ was Bill Thompson [versus] Tony Avella, right? This is obviously going to generate more excitement. So there are some challenges and guesswork and analysis that needs to be done.”
While campaign pollsters have a strong incentive to get the best possible polling results for their candidates, Gourevitch argued that public pollsters are partly driven by trying to make news, potentially making their efforts less rigorous.
“Quinnipiac is trying to be accurate and all those things, but they also usually wrap these things into a larger story of issues, and, you know, ‘How’s Ray Kelly doing?’ and ‘What do people think about this issue or that issue?’ and then they also ask about the mayor’s race,” Gourevitch said.
Micheline Blum, the director of Baruch College Survey Research, rejected the notion that internal polls are better than public polls. She said the internal polls using voter lists include few cell phone numbers and probably leave out a lot of landline numbers, too. The strength of campaign polls is identifying reliable voters so the campaign can get them out to the polls, she added. When it comes to predicting winners, campaign polls released or leaked to the press may be cherry-picked from multiple polls or leave out key details.
“The public polls, at this point, are remarkably good and accurate and methodologically sound,” Blum said. “Your only motivation is to get it right. The nonpartisan folks are not trying to get you to believe something that’s not the case. They have no reason to. Believe me, if they thought they’d get the answer better the other way, they would.”
In fact, Quinnipiac did just that in the 2009 mayoral race. After it misjudged the 2005 mayoral race, the polling institute scrapped its random-digit-dialing methodology in favor of the voter lists used by campaigns. When its performance wasn’t any better than it had been in the 2005 race, Quinnipiac reverted to its standard model.
“This time we’re back to our normal RDD, so I have more confidence in that,” said Doug Schwartz, Quinnipiac’s polling director. “The voter registration lists, if you’re just relying on those and the matches that you get, you’re really not going to get many cell phone numbers on it. It could be a problem.”
George Arzt, a political consultant who is skeptical of all polling, was a reporter for the New York Post when the newspaper commissioned a poll for the 1982 gubernatorial race. The poll showed Ed Koch beating Mario Cuomo by 17 points two weeks before the election. Arzt doubted the results, but the poll had been paid for, and he wrote up the story. Cuomo, who went on to win, made sure Arzt never forgot about it.
“I said to him, ‘You think I really wanted to look stupid?’ I’m still pained by it,” Arzt said. “I realize, because I was there too, that reporters love to do polls early, but rarely have I trusted the numbers that are in front of me.”
Tags: Baruch College, Bill Thompson, Doug Schwartz, Ed Koch, Gallup, Geoffrey Garin, George Arzt, Global Strategy Group, Mario Cuomo, Marist, Michael Bloomberg, Micheline Blum, Mitt Romney, Nate Silver, New York Post, Nick Gourevitch, Quinnipiac, SurveyUSA, Tony Avella