Bottom line: The devil is indeed in the details.
Let’s get really numbers geeky today ….
To satisfy, my curiosity, I dug a bit deeper into the 12 point difference between the recent Washington Post – ABC Poll (Clinton +12) and the historically accurate IBD poll (race tied).
Avid Trump supporters claim that WaPo over-sampled Dems … boosting Hillary’s numbers.
Break that into 2 parts: (1) Did WaPo over-sample Dems? (2) More generally, does over-sampling matter?
First, let’s draw a distinction between “over-sampling” and “over-weighting”.
Over-sampling means surveying more people in a particular group than their proportionate share of the population.
Over-weighting is disproportionately counting folks in a particular group when compiling an overall total.
Specifically, IBD says that it polls Democrats, Republicans and Independents in roughly the same numbers.
In other words, they over-sample Republicans and Independents since more than 1/3 of likely voters are Democrats (or, so they say).
But, IBD corrects for the over-sampling by re-weighting based on population proportions derived by separate studies called “enumeration surveys”.
Specifically, the IBD poll weights Democrats 36%, Republicans 30% and Independents 34%, assuming that mix is representative of likely voter turnout.
Note: overall, headline total numbers are highly sensitive to these turnout assumptions. Since roughly 85% of folks typically vote for their party’s nominee, each point shift in the turnout assumptions changes the weighted average by almost a point. And, these turnout ratios are derived outside of surveys based on the enumeration studies and political analysis.
As near as I can tell, the WaPo turnout weightings are about the same as IBD’s … maybe a couple of points more skewing to the Dems … maybe accounting for 2 or 3 of the 12 points.
No big deal.
Digging deeper is where things start to get really interesting.
Let’s look at what researchers call the “internals” … the components that go into the headline number.
For example, most pundits say that Trump is winning Independents, but that Hillary captures a larger percentage of her party’s likely voters.
IBD has Trump winning Independents by 9 points … 41% to 32%.
But, IBD has Trump winning more GOP likely voters than Hillary wins of likely Dem voters … 83% to 80%.
That may be right, but it’s contrary to the punditry.
WaPo has Hillary converting more Dems than Trump converts GOP … 89% to 83% … and has Hillary winning Independents by 8 points … 45% to 37%.
That squares with pundits re: own-party-conversion … but is contrary re: pundits and most other polls re: Independents.
In tech talk, those party affiliation results — IBD vs, WaPo –are wildly disparate.
They can’t both reflect reality.
Digging even deeper, let’s look at the gender splits:
IBD has Trump winning men by 13 points and Hillary winning women by 12 points.
Directionally, that matches the pundits.
But, WaPo has Hillary winning women and men …. Women by 20 points 55 to 35 …. Men 44 to 41.
Hillary winning women makes sense, but I haven’t seen any other polls with her winning men.
Makes no sense unless ….
There’s a significant anomaly in the WaPo survey … which is clearly noted.
Look at the asterisk and footnote below.
Apparently, WaPo doubled up two samples worth of Blacks & Hispanics.
Down at this level of detail … over-sampling and over-weighting tend to converge … since the turnout assumption start to get flakey.
For example, how do you estimate what percentage of black (or white) men are likely to vote?
So, if more of a group — say black men are over-sampled — they are probably over-weighted in the results.
If true, that would boost black men’s “contribution” to the party and gender results.
Since Hillary wins big with minority groups, that’s probably why her gender scores look out of line … and why the overall Clinton – Trump spread is so large.
Q.E.D. (<= stats talk for “I’m done”)
Takeaway: Look deeper than the overall number that’s reported.
Pay particular attention to the turnout ratios … and then to the splits by party, gender and race.
That’ll tell you more than the total number.
P.S. My bet: Hillary is probably leading by 4 or 5 points … as the poll-of-polls report.
Good new: in 2 weeks, we’ll know which poll was right.