Islamic Poll Daze

by Howard Fienberg
September 9, 2002

According to a recent poll, 57 percent of American Muslims claim to have experienced bias or discrimination themselves since 9-11, and 87 percent say they know of another Muslim who has experienced discrimination. The poll, conducted by the Council on American-Islamic Relations (CAIR), surveyed 945 people in late July and early August. It paints a mixed picture of America's treatment of her Muslim population - three-fourths of respondents also claimed to have experienced acts of kindness or support from non-Muslims. Regardless, things are probably not that bad - there's no reason to believe this poll accurately captures the American Muslim experience, post-9-11.


CAIR's sample of 945 people is quite typical for a random sample survey. But what is the margin of error? The "+/- X percentage" that has become ubiquitous when American journalists report on surveys is missing. The margin of error is the potential error in a poll due to the statistical model. While there are many sources of polling error, this one is the easiest to calculate. Revealing the margin of error is part of a pollster's reluctant duty: reminding consumers not to presume their polls to be 100 percent accurate.


But the sin of omission is not the only trouble afflicting CAIR's poll. Rather than conducting the poll by telephone, CAIR explains it "faxed, mailed and e-mailed" surveys to "Muslim individuals and organizations nationwide." Mail/fax/e-mail surveys are generally cheaper than telephone surveys, produce results uninfluenced by different interviewers and offer an increased sense of privacy (which should encourage more candid responses). The drawbacks to mail/fax/e-mail surveys include particularly low response rates and self-selected response. The Muslims who feel most aggrieved would be the most likely to respond to the survey, dramatically skewing the sample. A sample of self-selected (non-random) respondents such as this can only accurately reflect the opinion of those respondents, not an entire population. That is why random-sample surveys conducted over the phone remain the method of choice among American survey researchers.


Knowing that the survey mechanism was not a telephone, the essential piece of information, more so even than the margin of sampling error, would be the response rate. Now, it is true that few pollsters disclose such information.


However, without such information, there are two possible descriptions of CAIR's sample. Either CAIR contacted a probably very large number of people, and 945 of them responded, or CAIR contacted only 945 people and a smaller number bothered to respond.


CAIR is no stranger to polling hijinks. In April, the organization's web site hosted an Internet poll, asking respondents whether or not Israeli Prime Minister Ariel Sharon should be tried for war crimes. Glenn Reynolds called attention to the poll on his Instapundit web log on the evening of April 15. At the time, the poll registered 513 votes - and 94 percent of them were in favor of trying Sharon. By early the next morning, Reynolds saw the tally had reversed itself - 94 percent were against trying Sharon - and the number of respondents had skyrocketed to 11,951. Within hours, the total tally of respondents had started to decrease, and was down to 2,083 votes. That turned the vote around to 93 percent in favor of trying Sharon. Within minutes of Reynolds noting this, CAIR removed the poll from their site.


CAIR later complained of a "nefarious attempt" to manipulate the poll, with one or two respondents voting thousands of times. Of course, CAIR should have known that the results of any such poll are meaningless. At best, Internet polls are for entertainment purposes. At worst, they are useful for interest groups looking to impact public opinion, rather than measure it.


The organization also continues to insist that, "There are an estimated seven million Muslims in America," based on its 2000 survey of mosques, while a STATS investigation in November estimated the number is closer to about two million, give or take a few hundred thousand. When it comes to dodgy polling, CAIR are serial offenders. Hope remains that they can learn to conduct better surveys in the future.

See the original:

return to Howard Fienberg's page