male white rich
0.501 0.599 0.251
college vote.republican
0.551 0.501
PSCI 3802: Political Polling
We think of this:
Election polls are the visible tip of the iceberg. They hide how much of what we know about the world comes from surveys.
Because we get to see the truth.
The error is always there. Elections are the rare case where we get to see it.
According to Groves:
A survey is a systematic method for gathering information from a sample of entities for the purpose of constructing a quantitative description of the attributes of the larger population of which the entities are members.
A census (counting everyone) is not a survey. The population is the point.
“More data is better”: 10 million postcards, 2.4 million replies.
Two problems we’ll spend the semester on:
The “fix” targeted only the frame: set subgroup targets, fill them.
But quotas let interviewers choose who filled them…
Interviewers avoided the very poor and very rich (“the unshaven and sweaty”). Truman won.
Even with the right quotas, human choice in who gets interviewed introduces bias.
Stop hand-picking respondents to hit targets.
Let chance choose them.
Let’s play God: build a population where we know the truth, then sample from it.
50% male, 60% white, 25% rich, 55% college, vote split 50/50 (traits correlated).
male white rich
0.501 0.599 0.251
college vote.republican
0.551 0.501
A random sample recovers the population in expectation, even a tiny one.
male white rich college vote.republican
n = 50000 0.502 0.601 0.253 0.55 0.505
n = 1000 0.524 0.601 0.263 0.56 0.505
n = 100 0.490 0.630 0.200 0.54 0.530
Noisier as it shrinks, but no systematic tilt.
A random sample of 1,000 works as well in China (1.4B) as in Canada (40M).
The soup analogy: stir the pot well (that’s the randomness), and one spoonful tells you the seasoning. The size of the pot is irrelevant.
Break the randomness: let college grads be likelier to be interviewed (20% vs 15%).
male white rich
0.479 0.613 0.249
college vote.republican
0.622 0.489
A tilt on one variable quietly biases the ones you actually care about.
The population is the point.