The Daily Background
| A note on polling (don’t believe it!) Part III
by Arlen Parsa
Shortly before the Iowa caucus (the Democratic of which I predicted the correct outcome and winning order of), I wrote two posts cautioning against paying too much attention to last minute polls:
Well, Tuesday’s New Hampshire primary is almost upon us, and so I suppose it’s time to write another such post. It seems like every blogger is posting every scrap of polling information they can get their hands on, as if it was all extremely valuable information. For example, many bloggers had cited the Zogby poll as evidence that Obama hadn’t gotten any benefit out of Iowa. The Clinton campaign’s pollster Mark Penn even sent out a memo to reporters proclaiming that Obama had gotten no bounce out of his decisive win in Iowa:
Where Is The Bounce?
Two polls that had the race within a few points before the Iowa caucuses have the race tied after the Iowa caucuses.
In today’s CNN/WMUR poll, Hillary Clinton and Senator Barack Obama are tied at 33 percent - their last two polls had Hillary up 4 points and before that had Hillary down 2 points, so there is no statistically significant change in their numbers before and after the Iowa caucuses.
And the Concord Monitor is out as well today with a poll showing the race at 33 percent for Hillary Clinton, 34 percent for Barack Obama and 23 percent for John Edwards - exactly the same margin as before Iowa.
Contrast that with the 17 points John Kerry gained in 2004 in the Boston Globe poll, which catapulted him from a 17-point deficit to a 20-point lead in New Hampshire after the Iowa caucuses. Or with the 7 points Al Gore gained in 2000 in the CNN/USA Today/Gallup poll, increasing his lead in New Hampshire from 5 points to 18 points.
New Hampshire voters are fiercely independent. They will make their own decisions about who to support.
There are a few problems with Penn’s memo.
First, the CNN/WMUR poll Penn cites was conducted on Friday and Saturday, a time that pollsters generally don’t trust. As Pollster.com’s Mark Blumenthal writes:
…Obama appears to be gaining. So how big will the bump be? Firm conclusions are premature for two important reasons.
The first involves the issue of weekend interviewing, or more specifically, surveys based on interviews completed entirely on Friday night and Saturday. Most campaign pollsters are reluctant to put too much faith in interviews conducted at those times, when younger and more mobile voters are less likely to be home. In my 20+ years of looking at surveys conducted for campaigns, I can remember only one we did based solely on Friday and Saturday interviewing
[…]
An aside: When pollsters like me worry about “weekends” we mean Friday night and Saturday, not Sunday. Actually, late Sunday afternoons and evenings are among the best times to catch people at home, especially in the winter. And I see much less to fear from a survey that begins calling on Friday and finishes on Sunday, so long long as all of the “no answer” numbers from Friday and Saturday get dialed again on Sunday night.
So to summarize, Obama’s support is likely under-counted because his supporters are among those least likely to be home on a Friday night or during the day Saturday.
Second, the Concord Monitor poll Penn cites was conducted over the exact same period of time, and was by Research 2000 which is considered one of the less respected polls among professional pollsters in New Hampshire.
Third, in his memo claiming that Obama won’t gotten a bump out of Iowa, Penn also cites the historical examples of Gore and Kerry, both of whom got significant bumps after a victory in the Hawkeye state caucus. The problem? Those bumps didn’t magically materialize just one day after the caucus took place. As Chris Bowers correctly notes:
Eight consecutive New Hampshire polls that have trendlines have shown Obama improving in New Hampshire relative to Clinton. Further, the post-Iowa polling bounces for Bush, Gore, and Kerry all peaked between three and five days after the caucus took place. After five days, in all three cases the Iowa winner began to slide in New Hampshire polls. However, this time around, there will only be five days separating Iowa and New Hampshire, thus providing less time for any bounce to fade.
In other words, if we are to believe historical precedents, Obama’s post-Iowa bounce should be building right now, not already materialized as Penn suggests.
Oh, and that Zogby poll mentioned earlier that so many bloggers were citing? 80% of its phone interviews were conducted before Obama’s victory in Iowa was even known. The same is true of the new McClatchy/MSNBC/Mason-Dixon poll, which was conducted between January 2nd and 4th (Iowa’s caucus was on the 3rd).
[…] sooner had I written part three in my series part three in my series do I see the latest CNN/WMUR poll. First, let’s summarize the points […]