How do the poll people decide the margin of error?

897 views

Google says that the margin of error is “The margin of error is a statistic expressing the amount of random sampling error in a survey’s results. The larger the margin of error, the less confidence one should have that the poll’s reported results are close to the “true” figures; that is, the figures for the whole population.”
But how do they know how wrong they are? Is it just bullshit?

In: Mathematics

4 Answers

Anonymous 0 Comments

If you’re curious about the actual math used, margin of error calculations typically depend on the Central Limit Theorem. This is a theorem that says if you take a sample from the population, the amount that your result will deviate from the true value follows a Normal distribution. The Normal distribution is a well-understood probability distribution, so you can assign a probability to any amount of deviation. The margin of error is usually selected to ensure that there is a 95% chance the true value is within the range defined by the margin of error.

The Central Limit Theorem also says that the sample size affects the resulting error distribution. The more people you survey, the tighter the distribution, so the smaller the margin of error.

Anonymous 0 Comments

Lets assume 2 candidates, A and B, with around 50% support each.

If I ask everyone in the electorate who they’re voting for, my results will be perfectly accurate (unless some of them change their minds), but I can’t do that.

I ask 100 people chosen randomly. Now it’s not improbably that I get exactly 50 people responding each way, but I wouldn’t be surprised if I get a 51/49 split, or even somewhere around a 45/55 split. It just happened that my random choice picked a few extra supporters of 1 candidate or another. Statistics are noisy like that.

We can use mathematical models to work out how likely it is that it’s within a range of 1% or 5% or whatever. With 100 people we can say that 90% of the time we’re accurate to within a range of 6%, 95% of the time we’re accurate within 9%, and 99% of the time accurate to within 12%.

By convention we look at what we can be 95% certain of. This is just a convention. We find that if we randomly select 1000 people, 95% of the time the number falls within 3%.

Be aware that this means that 5% of the time, we’re going to be outside that 3% range. So it’s always worth taking statistics with a pinch of salt.

Anonymous 0 Comments

FYI: when I took statistics, my professor said that you want to look for a sample of ~1,000 in a poll, and a good margin of error is 3%. That’s why you see political commentators lose it when races get within about 5%; that margin of error could reverse an election result.

Anonymous 0 Comments

A margin of error is a rigorously determined statistical calculation – there are precise formulas for it. It’s not a general statement of “how wrong could we be.” It really more of an assessment of how much your sample size may not be a good representation of the entire population you wish to sample.

If you want to know the preference between vanilla or chocolate of everyone in a town of 100,000 people, you could ask each of them and report what you find. Your finding would have no margin of error – you asked everyone and reported the percentages based on that. The point of a poll is that you don’t want to ask everyone, you want to take a much smaller sample and use that sample to extrapolate results for the entire population. The smaller your sample size, the greater the margin of error.

If you sampled 80,000 people (randomly, controlling for bias), you’d be pretty darn sure you’ll get very similar results as your total-population survey. If you asked 2 people, you’d have very little confidence that your results track the entire population. You can determine exactly how confident you should be that your sample results reflect the entire population, and that is expressed as a “margin of error.”