Newsable: Why you should take individual polls with a pinch of salt

[ad_1]

Newsable is brought to you by BNZ. Listen to today’s episode in the player below.

Batten down the hatches, update the last will and testament, cover the floors in plastic – because when you hear the results of this latest political poll, your head may literally explode.

That’s not a real introduction, but it may as well be. It’s an election year after all: breathless coverage of the political polling is as inevitable as the tides, death, and taxes.

But the thing is, polls aren’t a crystal ball. They’re not deterministic. They’re a snapshot of a particular moment in time, taken as the remorseless slings and arrows of daily politics continue to fly overhead.

So what are polls actually useful for? What are their limitations? How are they actually conducted, and how do the political movers and shakers translate this data into rhetoric and actions?

Stuff’s daily podcast Newsable sat down with three experienced but very different operators in the polling ecosystem to gain an insight into these topics – and plenty more besides.

Curia Market Research founder David Farrar.

RICKY WILSON/STUFF

Curia Market Research founder David Farrar.

The pollster

“A poll is basically an attempt to find out what the public think on an issue”, says David Farrar, who founded and runs Curia Market Research.

“It attempts to be scientifically representative: what ISN’T a poll is me walking down the street and asking 100 random people, because that’s going to be determined by who I happen to walk into.

“Polls, you try to … represent New Zealand – do you have a good balance of gender, by age, by where people live? – so that when you get the results of a poll, it hopefully tells you something.”

Farrar describes Curia’s process as a ‘multi-mode model’, a pleasingly alliterative phrase meaning the firm uses a mixture of landlines, mobile phones and online panels to conduct polling.

It also sets quotas in an attempt to reflect the makeup of a population: 51% of the population is female, Farrar says, so a representative political poll would have 51% female respondents.

“One of the key tricky things is, how many different types of quotas do you have? If you have a quota for everything, you have to phone 10,000 people to find the 22 year-old left-handed Pacific Islander living on Waiheke Island … so you tend to do gender, age, area, income, sometimes ethnicity, to try and get that.

But do polls really reflect how people are feeling? Or is there an element of polls leading people to vote one way or another?

“Oh, absolutely … and there’s a number of factors for that.

“People don’t like to back losers. But if you see this party is sliding in the polls, the media coverage goes negative … this is why politicians, especially, are very focussed on the polls – I used to joke that John Key’s interest in the polls made Gollum look like a quitter when it came to the Ring.”

22122017 Photo: ROSS GIBLIN/STUFF Former Labour MP and United Future leader Peter Dunne

Ross Giblin/Stuff

22122017 Photo: ROSS GIBLIN/STUFF Former Labour MP and United Future leader Peter Dunne

The politician

Peter Dunne was the MP for Ōhāriu for more than 20 years – first as a member of the Labour Party, and then as leader of United Future. He served as a minister in both Labour-led and National-led governments.

“I think there’s always a risk in looking at the last poll, the last set of polls, and forming a view. You’ve got to take the long view, look at the trend line over time – certainly that’s what the politicians look at.”

While the polls that get the most oxygen tend to be those commissioned by media companies, political parties poll far more regularly. The difference, of course, is that political parties seldom release their internal polling.

“The larger parties are often polling weekly, and they’re polling on particular topics – not just the horse-race between the main parties, but they’re rating the leaders or looking at how particular policies are rating with voters, or even what it is the voters are expecting from them.

“They’re getting a lot more constant information, and they’re able to set that alongside what they got last week, the week before, the month before, and see what the developing trend is.”

Interestingly, Dunne says access to the information garnered from internal polling is often tightly controlled by those high up in a party.

“It’s a simple proposition: information is power. So you’re the leadership of a party, you’ve got some polling information that might be critical of you or critical of positions you’re advocating – do you really want to share that with your caucus and say the public think I’m a dickhead?

“You want to keep that very tight, just in case it gets misused by those in the caucus who might have a different view than you.

“It’s politics within politics, if you like: you’ll present the bits to the team that are broadly favourable, and keep the rest in-house to a small group so you can figure out what the next move is.”

Stuff senior journalist Andrea Vance.

Abigail Dougherty/Stuff

Stuff senior journalist Andrea Vance.

The political journalist

“I am not a big believer in the value of those polls”, says Stuff senior journalist Andrea Vance, who spent nearly a decade working in the press gallery.

“They’re very expensive … roughly $10,000 to $15,000 for a poll (though media companies do get package deals).

“The way I looked at it was: it was enough money to pay a senior journalist to do actual journalism for a year. So I was never a big fan of polls, and I was not silent about (it).”

The information garnered from sporadic media-commissioned polls is useful to particular people, Vance says: political strategists, for example, and political parties. It’s less valuable to political journalists, and much less valuable to the public at large.

She says part of the reason is that relatively few public polls are done in New Zealand. The more regular ones commissioned by political parties are rarely released, and when they are there’s often an ulterior motive.

“They only get leaked when someone wants them to be leaked, for good or for bad. So you don’t have a baseline, a regular data point to analyse.

“And then the way that we treat polls, individual polls, is fundamentally wrong. Journalists tend to interpret polls like they’re a crystal ball, like they’re going to predict what’s happening in the future, when actually they’re telling us what’s happening right there and then and in the past. We’re using them the wrong way around.

“Polls are valuable when you look at them as a trend over a long period of time. One poll won’t tell you what you need to know. It’ll tell you about a point in time … it definitely cannot tell you what’s going to happen at the election.”

Vance says the manner of coverage is part of the issue: often polls showing a marginal shift in voting are advertised as blockbuster events, appointment viewing. And she says the reason for this is simple: economics.

“You’ve gotta squeeze the hell out of it right? You paid 11, 12 grand for a poll – it’s gotta be the top story, it’s gotta be your lead.

“(This) goes to the gut of why I feel so strongly about it: it feels like we’re manufacturing news. We’re taking a piece of information that’s a snapshot of time, and we’re pretending we know the future.”

General advice for interpreting polls:

The trend is more important than any individual poll. Look at how polls trend over an extended period of time, three to four months.

The most indicative individual questions are preferred prime minister and, by some margin, right direction/wrong direction.

When it’s possible, take the time to look at how specific polling questions are phrased to see whether there’s a chance people are being led down a certain path by a question (as David Farrar says, polls are a science, but writing questions is an art).

Be wary of any internal party polling that leaks out – it’s coming into the public domain for a reason.

Newsable is Stuff’s daily news podcast, wrapping up what’s worth talking about in a short package every weekday morning. You can find new episodes and more detail on our stories here or in our newsletter. Make sure to like and follow us wherever you get your podcasts and across Instagram and TikTok.



[ad_2]

Leave a Comment