Monday, April 06, 2009

That poll

A couple of the papers plus radio news are carrying stories today about a online poll. Dubai Today on Dubai Eye had a discussion about it too.

It's been described as a poll of 22,708 respondents, which sounded implausibly large to me so I thought I'd have a look at it to see what it's actually all about.

As usual the reporting is muddled, the conclusions are iffy, the reports are misleading.

Khaleej Times incorrectly calls it a 'regional online poll' while EmBiz247 is more accurate with 'recent series of polls'.

In fact it could more accurately be described as a series of individual questions posed on the website over a period of time.

There may well have been over 22,000 respondents, but that was to a whole series of questions over a period of weeks. Many would have been the same people answering each weekly question so the 22,708 respondents figure is inaccurate.

Each question actually had very many fewer respondents.

For example, to the question: "How long do you see yourself remaining in your present place of employment? which ran from March 1 to 9 there were 5,090 respondents.

Take it all with a pinch of salt and as only a vague indicator of people's thinking. This is not depth research, it's just people clicking on options given below the question.

It's part of the website's 'Poll Corner' feature, such as you see on many newspaper websites. Today's example is:

Do you think extending work hours & increasing responsibilities of existing human resources is a good response to the current economic crisis?

Not sure

The news outlets have picked up on one question and chosen to go with the positive interpretation:

"Expatriates prefer to stay in Middle East" says KT.

"More than a third of expats in region will opt to stay put" according to EmBiz.

Here's the actual poll.

The question posed between February 8 and 16 was:

If you are an expat in the ME, do you see yourself having to return to your home country this year as a result of economic conditions?

Total Votes: 3053

Yes I may have no choice: 845/27.7%

Yes I prefer to: 473/15.5%

No there is still no reason for me to move: 1090/35.7%

No I will move to a different country as an expat: 645/21.1%

The headlines and the stories don't really have it right do they, with their 'will stay put' and 'prefer to stay' slant.

Only 15.5% said they'd prefer to go home, 27.7% were realistic when they said they may have no choice.

The 35.7% who said they had no reason to go home obviously haven't lost their jobs - yet. If they do it will be a very different answer.

And the 21.1% who said they'd move on to another country as an expat are really saying they don't want to return home. The reality is that every country is being affected by the recession so there are probably no jobs for them in other countries - maybe not even back home.

I've talked before about polls and statistics and how they're often misinterpreted. How the context isn't considered, how the methodology is ignored, how the phrasing of the questions isn't taken into account. This is another example.

The two newspaper reports are here; Khaleej Times EmBiz247.

Then you should go to the polls yourself to get the background and understand what the questions were, what answers were available to choose from and how many people actually responded. They're here, scroll to 'Poll Corner' and click on 'View past polls'


Dave said...

Who will win the forthcoming Ashes Cricket Series? Is it:-

a) Australia
b) Australia
c) Not England
d) All of the Above

100% of respondents believe Australia will win the Ashes Series.....

Seabee said...

Spot on Dave, questions are framed to give the answer already decided upon. Remember the referendum on the Aussie Head of State? Polls said the majority wanted to change to a republic, so the government phrased the question: "To alter the Constitution to establish the Commonwealth of Australia as a republic with the Queen and Governor-General being replaced by a President appointed by a two-thirds majority of the members of the Commonwealth Parliament."

The polls had also shown that we didn't want the government giving the job to one of their mates, so they added that bit to the question knowing we'd vote against it.

John Bloody Howard could then lie that "the people voted against a republic".