Yesterday morning I noticed that the front page of the Toronto Sun had an Oscars theme, which made sense, considering the Academy Awards had happened the night before. (As a side note, I won’t comment on the Oscars in too much detail because, quite frankly, film isn’t much my area of expertise. That aside, it has made me love Jennifer Lawrence. Also, I thought Seth Macfarlane was alright as a host, despite a large swath of critics saying the opposite. People need to lighten up a little. Especially you, Peter Howell.)
But anyways, the Sun Oscar-themed front page wasn’t talking about the Oscars. No, it was talking about how, according to a poll that Forum Research recently did, apparently Torontonians love mayor Rob Ford or something. When I saw the headline, I became immediately suspicious, and with good reason. My journalist training has given me the impulse to look up a specific paragraph every time I read a story about an opinion poll. And anyone who reads these stories should look for this paragraph too.
What you should be looking for is the paragraph where it tells you how many people were surveyed, what the margin of error is and how many times out of 20 the poll is accurate to. When I searched out the paragraph in the accompanying Sun article, I found what I feared I might find. The poll randomly surveyed about 800 Torontonians. Therein lies the problem.
In my first year of journalism school, we were taught about the perils of polls. First of all, polls are conducted via random digit dialing (RDD). If they aren’t, the survey isn’t legitimate (on this factor, the Sun poll does alright). Secondly, though, the sample size needs to be at least 1,500 people. I don’t remember the exact reason why this is so, but it has something to do with improving the margin of error, if I recall.
So the Sun failed the second part of the test.
And I don’t mean to say that the Sun is the only paper that does this. While I admit I’m not a fan of that paper, a newspaper I do read, the Toronto Star, is equally guilty of making polls they conduct front-page news (most of the time, these days, the front-page poll news is usually how Torontonians wouldn’t vote for Ford, or something along those lines).
There are several reasons why newspapers should really, really stop using polls as top items. First and foremost, think about the segment of the population that these polls represent. Toronto is home to over two million people, and most surveys talk to no more than a few thousand. And who are these people pollsters reach by RDD, anyway? The always wonderful Rick Mercer has a brilliant rant on polls that echoes my sentiments almost to the letter.
And then think about the value of these things as news items. Quite frankly, who gives a shit what a thousand-and-someodd people think about pertinent issues of the day? Shouldn’t we be caring a little more about items that will actually affect our lives, like transit funding or government scandals? This is by no means an exhaustive list of issues, but it’s something that I think people should be a tad bit more interested in.
And the worst part is that, increasingly, the golden paragraph that tells you how it was done seems to be disappearing more and more often. The last few times I’ve looked at Star poll stories I’ve failed to find the information-bearing paragraph. For god’s sake, if you’re going to make a poll about Canadians hating Stephen Harper a front-page item, at least tell us how legitimate your data is! I end this rant with another person who echoes my sentiments, Arianna Huffington. She also brings up another pertinent point about the failure of polls.