by Howard Fienberg
Public opinion research is a scientific enterprise. But in the service of activism, its scientific precision and accuracy can go awry.
For example, with the Johannesburg Summit on Sustainable Development throttling into gear, the company Politics Online has launched the "first ever Online Global Poll on the issues of the environment and sustainable development." The summit will be teeming with heads of state, government bureaucrats, business leaders, and all manner of non-profit activist groups. Politics Online seemingly wants to insert the opinions of ordinary folks from around the world into the cacophony of the summit.
The most interesting results from the poll will presumably come from the "Solutions and Outlook" section. Question 13 asks respondents to rate the importance of eight different potential policies for "making the environment better." They include: taxing fossil fuels; strengthening national government regulation; taxing consumers of "products that cause environmental damage"; educating people to "live more green"; and more. While two options at least would put the responsibility on individual choice, none of the proposed solutions offer any new or creative approaches.
This deck looks somewhat stacked.
Question 14 asks respondents to make a superficial choice between the environment and the economy. Most research shows that people fall somewhere in the "middle" -- that is to say, they want both. It is unfortunate that this poll seems infused with the notion that these issues are a zero-sum game: that the world can only choose between them, rather than work towards improving both, or that improving the economy could possibly improve the environment as well.
Question 15 asks how important a political candidate's environmental stance is when the respondent votes. This makes for interesting campaign research, but since the whole poll is focused on the topic, the respondent is primed to rate the issue higher than he or she might normally do so.
Questions 16-18 focus on views of regulation, free trade and responsibility (national versus international).
Question 19 asks, "What % of your income would you be willing to give up for a real improvement in our global environment?" If the respondent truly felt that giving up 5 percent of their family income would contribute to any such "real improvement," such a contribution could be made through effective philanthropic channels. Indeed, many nonprofit foundations and organizations should eagerly await the results. However, with most environmental initiatives being run through national and international governments, that presumably means the "% of your income" would be taken by taxes, either directly or indirectly. If that is the case, the question would be better worded as, "What % of everyone else's income" would you be willing to give up.
Question 23 asks how satisfied respondents are with the "amount of news coverage on environmental issues." Nowhere are respondents asked about the quality of coverage rather than the quantity. Environmentalists and environmental journalists frequently complain that they don't get enough column inches or air time. But they also complain about what makes it into the coverage and how - Surely, that should be of interest as well?
Aside from problems with limited options and poor wording,
many of the poll's results will be of little or no direct use for the policy
In this case, the real reason to doubt the poll's usefulness lies not in skewed perceptions, question wording, or options for answers. This poll is worthless because it is an Internet poll.
The "Online Global Poll" will probably get a much higher response rate than any classic telephone poll ever could (maybe in the tens or hundreds of thousands) and it will do so at minimal expense to the company running it. That is why Internet polls are so popular.
But in polling, size doesn't matter. While telephone polls randomly select participants (creating a random sample which accurately reflects the population), online poll respondents take the initiative to make their views known. Hence, online polls capture the opinions of a group that is self-selected and does not represent the general population. Internet polling is little different from the frequent practices of radio and TV talk show hosts, who canvas listeners' or viewers' opinions through call-ins. Such call-in polling is referred to by public opinion researchers as "Selected Listener Opinion Polls" - SLOP for short.
Demographics online also present a problem. There are significant demographic differences between the respondents of Internet and phone polls. This is to be expected, because there are social distinctions between those who have the time and inclination to accept a telephone poll from a stranger and those who surf the Internet and are willing to spend that time responding to these kinds of polls. In the case of this "Global" poll, how many people on Earth are online, available to answer? We don't really know. And no amount of statistical manipulation can get around this inconvenient fact.
Internet polls make for great entertainment, but have no value in decision-making, especially on important policy issues. Attendees at the Johannesburg Summit already have so many ideas and policies to contend with, it can't help them to inject even more SLOP into the debate.
See the original: http://www.tcsdaily.com/article.aspx?id=082602B
return to Howard Fienberg's page