Home Mail Articles Stats/current Supplements Subscriptions Links

This is a preview version of an article forthcoming in LBO #109, posted on October 1, 2004.

(c) Copyright 2004 Left Business Observer. All rights reserved.

Republican bias at Gallup?

by Doug Henwood

Connoisseurs of polling - a group that includes this newsletter - have been confused by the presidential preference polls of the last few months. Some are portraying a tight race, and others a Bush runaway. Some of the strangest numbers have been coming out of our most famous pollster, Gallup. Recall that Gallup reported a one-point decline in Kerry's popularity after the Democratic convention - a departure from historical precedent, since candidates usually have gotten, according to Gallup, an average of a six–point bounce after their nominating conventions. By contrast, Bush got a more normal bounce; the Gallup poll released right after the convention showed him gaining two points, widening his lead over Kerry to seven. That lead widened over the next few weeks to a peak of thirteen points in mid-September, which fell back to a mere eight at month's end.

Other pollsters have reported a solid Bush lead in recent weeks (though not all - the IBD/TIPP poll has the race as a dead heat). But Gallup was out in front in both speed in size, and their polls showing Kerry's negative bounce followed by Bush's extended surge have contributed mightily to a sense that Bush is now the favorite by a fairly wide margin on November 2. Gallup has also been fairly consistently reporting higher approval ratings for Bush than other pollsters over the last year or two.

What's going on? One clue can be found in looking at their reported party breakdown. In their late-September poll, they have 43% of repondents identifying as Republicans, and 31% as Democrats. We've never seen anything like a twelve-point Republican lead anywhere else before. A July New York Times/CBS poll reported the breakdown as 37% Democrat and 29% Republican, an eight-point Dem advantage. The exit polls in both the 2000 and 1996 elections reported the breakdown as 39% Democrat and 35% Republican, a four-point Dem advantage. (Independents were 25% of the Gallup universe, 30% of the Times/CBS universe, and 27% of the 2000 election's.) Party affiliations change over time, for sure, but a sixteen- or twenty-point shift seems highly unlikely over the course of four years, much less two months.

Gallup provides a breakdown of preference by partisan self-identification (see table below). If you rerun their numbers using the Times/CBS party ID weights, you get a ten-point Kerry lead; if you rerun them using the exit poll weights from the last two presidential elections, you get a six-point Kerry lead. Most pollsters object to weighting raw results by party, since partisan identification is unstable over time, and is a reflection of opinion and not a demographic category. True enough, but if you get party IDs so at odds with other renditions of reality, then you should suspect something's wrong with your techniques.

What could be wrong? There may be some mischief hidden in Gallup's "likely voter" filters - a series of questions about voting history, knowledge of the polling place, and intention designed to strip out the stay-at-homes from the polling results. (Other polls report the preferences of registered voters or even all adults.) While it may be reasonable to narrow the universe to likely voters in the days before an election, it may not be so sensible a month or two before, when people just haven't firmed up their intentions yet. And it may not even be reasonable in the final pre-election polls: a review of the last four elections by Ruy Teixeira shows that Gallup's estimates for all registered voters were actually more accurate than those for likely voters in three of the cases.

Likely voter filters may be especially inappropriate this year, with record registrations of new black voters, intense motivation among Bush haters to send him back to the hellish climes of Crawford, and uncommon interest expressed by young voters, who are often discarded in likely voter models.

"They're all Republicans!"

But is all this just a matter of mere technique, or is something else going on? There may well be some hidden partisan bias at work.

As the blogger Kos reported, in June 2003, Gallup's CEO James Clifton gave $2,000 to a very right-wing Republican who was running for Senate, Herman Cain. Journalists are not supposed to do these things; many newspapers are even forbidding reporters to go to the Kerry benefit concerts headlined by Bruce Springsteen, since the proceeds go to the Democratic campaign. Taking this concern for appearances to an absurd extreme, former Washington Post executive editor Ben Bradlee didn't even vote. But this notion of ethics doesn't extend to the CEO of the world's most famous pollster.

Since I'm a paying member of the American Association for Public Opinion Research (AAPOR), I subscribe to their electronic discussion list, AAPORnet. When I raised some questions about an apparent pro-Bush bias in the Gallup numbers on the list, I was met with mostly hostile responses; one participant even characterized my questions as "sleaze."

But the postings inspired a call from a former Gallup researcher last week, who opened the conversation by saying "They're all Republicans!" Well, not all, he clarified - just most of the senior people, like editor-in-chief Frank Newport and senior editor Lydia Saad. (An exception is Democrat David Moore, a senior analyst.) Saad and Newport have not yet responded to emailed requests for comments.

This isn't to say that Gallup cooks the books; I don't believe that. But in a field where you have to make a lot of choices about technique, your unconscious can easily lead you to embrace the ones that fit your preferences. But since polls can change the public opinion they're supposed to measure - people shun a loser and cluster around a winner - such choices can really matter.

Home Mail Articles Stats/current Supplements Subscriptions Links