Islamic Poll Daze
by Howard Fienberg
According to a recent poll, 57 percent of American Muslims
claim to have experienced bias or discrimination themselves since 9-11, and
87 percent say they know of another Muslim who has experienced
discrimination. The poll, conducted by the Council on American-Islamic
Relations (CAIR), surveyed 945 people in late July and early August. It
paints a mixed picture of
CAIR's sample of 945 people is quite typical for a random sample survey. But what is the margin of error? The "+/- X percentage" that has become ubiquitous when American journalists report on surveys is missing. The margin of error is the potential error in a poll due to the statistical model. While there are many sources of polling error, this one is the easiest to calculate. Revealing the margin of error is part of a pollster's reluctant duty: reminding consumers not to presume their polls to be 100 percent accurate.
But the sin of omission is not the only trouble afflicting CAIR's poll. Rather than conducting the poll by telephone, CAIR explains it "faxed, mailed and e-mailed" surveys to "Muslim individuals and organizations nationwide." Mail/fax/e-mail surveys are generally cheaper than telephone surveys, produce results uninfluenced by different interviewers and offer an increased sense of privacy (which should encourage more candid responses). The drawbacks to mail/fax/e-mail surveys include particularly low response rates and self-selected response. The Muslims who feel most aggrieved would be the most likely to respond to the survey, dramatically skewing the sample. A sample of self-selected (non-random) respondents such as this can only accurately reflect the opinion of those respondents, not an entire population. That is why random-sample surveys conducted over the phone remain the method of choice among American survey researchers.
Knowing that the survey mechanism was not a telephone, the essential piece of information, more so even than the margin of sampling error, would be the response rate. Now, it is true that few pollsters disclose such information.
However, without such information, there are two possible descriptions of CAIR's sample. Either CAIR contacted a probably very large number of people, and 945 of them responded, or CAIR contacted only 945 people and a smaller number bothered to respond.
CAIR is no stranger to polling hijinks.
In April, the organization's web site hosted an Internet poll, asking
respondents whether or not Israeli Prime Minister Ariel Sharon should be
tried for war crimes. Glenn Reynolds called attention to the poll on his Instapundit web log on the evening of April 15. At the
time, the poll registered 513 votes - and 94 percent of them were in favor of
CAIR later complained of a "nefarious attempt" to manipulate the poll, with one or two respondents voting thousands of times. Of course, CAIR should have known that the results of any such poll are meaningless. At best, Internet polls are for entertainment purposes. At worst, they are useful for interest groups looking to impact public opinion, rather than measure it.
The organization also continues to insist that, "There are an estimated seven million Muslims in America," based on its 2000 survey of mosques, while a STATS investigation in November estimated the number is closer to about two million, give or take a few hundred thousand. When it comes to dodgy polling, CAIR are serial offenders. Hope remains that they can learn to conduct better surveys in the future.
See the original: http://www.tcsdaily.com/article.aspx?id=090902B
return to Howard Fienberg's page