… Raleigh lawyer Press Millen hit just the right notes in this column about gross misuse of opinion poll data.
This is a common problem for media types, since few of them have enough statistics training to understand the significance (or lack of significance) of results tied to specific polls.
My statistics training is confined to one entry-level college stat course and one stat-heavy journalism course, so my knowledge of the topic is limited. But I remember enough from those classes to know that a poll has no statistical value unless it’s conducted properly. Internet polls, call-in polls … fun, yes, but not useful statistically.
Even if you conduct your poll scientifically (random sample, properly defined “population,” well designed questions), the accuracy of your results comes with caveats.
First, your poll will have a margin of error. With typical sample sizes, that margin is plus or minus 4 points. This isn’t a big deal if you find that 85 percent of the people surveyed prefer candidate A, while 15 percent support candidate B.
It’s a very big deal if you find that 51 percent support A and 49 percent support B. That means that the true support for candidate A could range from 47 percent to 55 percent, while support for candidate B could range from 45 percent to 53 percent.
In other words, your poll shows that candidate A might have a 10-point lead (55-45) or might face a six-point deficit (47-53). The results are unclear.
Plus there’s a great kicker that almost never merits mention in typical media coverage of poll results. Most well designed standard public opinion polls use a 95 percent standard. I could expose my ignorance and bore you to tears by trying to explain why that’s true. For the purposes of this blog, that information is not important.
What is important is the knowledge that most polls use this 95 percent standard. That sounds pretty good until you realize that there’s one chance in 20 that the poll is completely wrong. With the vast number of polls used every day, there’s a good chance that some of them fall into that 5 percent category.
Polls play a useful role in our public life. We just need to be careful when we use their results.
