Technical Documentation: Gender Difference in National and State-level Polling on Election 2012

October 28, 2012

Richard C. Eichenberg

Elizabeth Robinson

Tufts University

We have been following and recording gender difference in state and national polling during 2012 and provide a descriptive overview of the data here.

A few words on the contents of the database:  First, we track only polls for which the gender breakdown is available at no cost (this eliminates Rasmussen polls, to choose one example). Second, we track only polls for which the available cross tabulations are reported in a comprehensible fashion that facilitates rapid updating.  Finally, we have excluded national daily tracking polls because they would overwhelm the virtue of variety that comes from including a large number of different polling organizations (the Gallup and Rasmussen daily trackers are therefore excluded)  After the election has passed, we may add these tracking surveys back to the database for research purposes, perhaps choosing one reading per week. These exclusions have no effect on the state-level polls, as there are no state-level trackers –yet.

Does this introduce a “House Bias” into the data collection?

The excluded polling organizations mentioned above have produced results in 2012 that lean more pro-Romney than other organizations, although we have no way of knowing or assuming that this bias is more or less pronounced for gender differences in the polling results.  Nonetheless, their exclusion here might suggest that our collection is slightly more Obama-friendly than it would be were Rasmussen and Gallup (among others) included.

Partisans of both sides should be aware of that.

On the other hand, based on the sophisticated calculations performed by others, house bias is a smaller problem than one might expect when many polls with different drifts are averaged together, as we do here.  Based on the calculated “house bias” reported by Jackman and by Linzer,  our collection includes a number of polling organizations that lean pro-Obama by about 1 percentage point or slightly more, but it also includes organizations that lean slightly pro-Romney by about the same amount . Finally, the collection includes a large number of polls from organizations whose house bias is close to zero.

A fair guess –and it is only a guess—is that the average house bias in our collection leans only very slightly toward Obama, perhaps half a percentage point.  In fact, it may be less.  As we note below, we estimate the Obama/Romney percentage of the national electorate as of October 24th at exactly 50/50.  On this date, this is precisely the percentage estimated by the Huffpost’s pollster.com.  Also on this date, the NYT’s Nate Silver estimated a snapshot of the race at 50/49 (Obama/Romney).  In subsequent posts, we will also report the race in important states such as Colorado and Virginia at almost exactly 50/50 as of Oct 24th, which is also the precise number estimated by the professional poll aggregating sites.

Given how close our averages come to the averages yielded in professional model estimates, we have high confidence that our estimates are near the mark.  We will find out –and report– on election day.

Voter Base

The results reported here include likely voters and registered voters in survey samples.  We include registered voters because it allows us to extend the comparison back to January, when few polls had yet included likely voter screens.  As the year progresses, however, far and away the largest number of polls are based on likely voters only.  Specifically, for the entire year of 2012, 55 percent of our polls are Likely Voters.  After July, however, this increases to 84 percent Likely Voters, and it increases to 87 percent Likely Voters during October.

Polling Organizations Represented in the Database and Number of Polls for Each

ABC/WP 3
ANGUSREID 15
ARG 21
CBS 2
CBS/NYT 1
CNN/ORC 10
CNN/TIME/ORC 2
DK/PPP 13
DK/SEIU/PPP 3
FOX 19
GRAVIS 13
GROVE 5
IBBD/INV 2
PHILA INQUIRER 2
KIMBALL 2
LAKE 3
MARIST 2
MELL 1
MONM 6
MUHL 2
Marist/Mcc 2
NBC/Marist 15
NBC/WSJ/Marist 38
NEWSMAXZOBGY 3
NY1/YNN-Marist 2
PEW 10
POST/ABC 1
PPIC 4
PPP 120
PURP 18
QUINN 35
QUINN/CBS 1
QUINN/CBS/NYT 27
STRAT360 1
SUFFOLK 2
SVYUSA 41
TIME 1
UNH 10
WNEU 4
WP 2
YOUGOV 49
YOUGOV/ECON 44
Total 557