When it comes to predicting elections, pollsters are often judged on whether their results seem to consistently favor one party over another. The industry shorthand for this is “house effects.”
Poll critics are quick to argue that the existence of house effects proves deliberate partisanship and to suggest that many election surveys are deliberately skewed. But partisanship is not the only reason that a pollster’s work may repeatedly favor one party’s candidates.
In a series of articles, HuffPost Pollster will dig into the factors that push poll results in a partisan direction. The ultimate goal is to figure out how we can use an understanding of house effects to measure the quality of individual pollsters. We’ll start by focusing on how house effects are calculated and how the basic patterns of partisan bias played out in 2014 Senate and gubernatorial polling.
–
Additional technical information:
A technical description of the HuffPost Pollster model for the 2014 Senate and governor’s race forecasts can be found here. The model was adjusted to generate the house effect averages used in this article by comparing pollsters’ results to the actual election vote, instead of a polling average. The “calibration” to nonpartisan pollsters that were within one standard deviation of the 2012 outcome was not used in generating the house effects for this article, and the individual partisan polls (when the pollster only surveyed the race once) were not grouped into the Republican and Democratic pollster categories. The resulting house effect for each pollster was not restricted by any other pollster’s house effect.
The house effects calculation shows how far the pollster was from actual results, on average, by comparing the margin of the poll (Republican estimate minus Democratic estimate) to the actual margin (Republican result minus Democratic result) and combining these figures for all the polls that the pollster conducted in that race. The model treats polls conducted closer to the election as more influential than earlier polls, so pollsters are not penalized for conducting early polls. If a pollster surveyed in multiple races, the house effects for each race were combined into one overall measure by multiplying the house effect for each race by the number of polls in that race, adding those results together and dividing by the total number of polls.
