Bad Science Never Dies

by Howard Fienberg
May 20, 2002

Splashy science news reports draw eyeballs and move policy, but sometimes the scientific heart of the news comes up short. Worse, it can be dead wrong. So what happens in the news when a study is found to be faulty or false and ends up being retracted or thrown out?


Not much, usually. Science news revolves around news -- new studies, discoveries and achievements. The discovery that previous research has been dis-proven or shown to be worth less than the paper it was printed on just does not register as news to most journalists, no matter how said research was originally hyped to the public.


This is understandable. After all, journalists often work for publications that don't worry much about correcting the public record. Most newspapers and magazines print correction columns, but they can be hard to find. Few publications admit in big type that they were wrong.


Of course, when the media make a mistake, it usually is not earth shattering. By contrast, scientific errors can spread and leave even more bad science in their wake.


A study by Dr. John M. Budd et al. in the Journal of the American Medical Association (Jul. 15, 1998) examined 235 scientific journal articles that had been formally retracted due to error, misconduct, failure to replicate results, or other reasons. The researchers reported that, "Retracted articles continue to be cited as valid work in the biomedical literature after publication of the retraction."


Budd and his colleagues acknowledged that there is sometimes a significant time lag (an average of 28 months) between publication and retraction. But they found that the flawed articles were cited in the scientific literature an astonishing 2,034 times after they had been retracted. The vast majority of these post-retraction citations treated the work as still valid, making no reference to the retraction.


At a certain level, these studies have become urban myths. Despite no longer possessing scientific authority, their repeated publication has let them take on a life of their own -- regardless of any grounding in truth. Such scientific myths are worse than simple scare stories about kidney stealing or the influence of the full moon, because future researchers unwittingly depend upon their (invalidated) analyses.


For example, take the finding back in November that genetically modified corn had infiltrated regular strains in Mexico and was scrambling DNA chains (see Mexican Jumping Genes, Mar. 18). Scientists were suspicious of the study's claims from the start, but the examination of the research took time. Nature, the journal that had published the study, then spent time reviewing and considering the arguments of the critics as well as the counter-claims of the original authors. When Nature finally printed letters in April from two teams of scientists pointing out the extensive deficiencies in the research, the journal all but retracted the original article, admitting that "the evidence available is not sufficient to justify the publication of the original paper."


However, when the media responded to this news, they concentrated on political aspects, not the important scientific ones. The Washington Post reported that one of the authors of the original article "believed the effort to undermine" the study "was the work of biotechnology advocates, some of whom had personal reasons for attacking him." Rather than laying out the science and the criticism, the Post reduced the matter to a 'he said, she said' narrative, concentrating on personal and political motives rather than the merits of the research. This kind of narrative may follow the dictates of allegedly objective journalism, but it doesn't explain very much.


Further in the political vein, Guardian writer George Monbiot (May 14) dedicated a lengthy article to investigating the supposed public relations campaign against the article, a campaign "so severe" that it "persuaded" the journal to retract it (never mind the methodological problems in the article, which Monbiot called "hardly unprecedented in a scientific journal").


And, as if to demonstrate the conclusions of JAMA's 1998 study of retractions, the Washington Times (Apr. 30) ran a center-page spread on the Mexican corn infiltration, along with photos from an anti-GM crop demonstration outside the Mexican consulate in San Francisco and a photo of one of the original articles. While the Times did admit, deep inside the article, that the journal might have erred in publishing the study "because of a technical issue," understatement was not the biggest problem. In continuing to accept the retracted Nature article as gospel, the newspaper was simply following in the well-worn footsteps of news coverage earlier that month. When that media coverage reduced a scientific retraction into just another installment of political controversy, they reduced the need of other journalists to worry about the scientific part of the problem.


Since even scientists must rely on the news media for much of their science news (an endless array of journals defy any sane person to keep track of them all), they might miss the Nature retraction, too. It won't be long before other journal articles cite the (retracted) study.


When initial research becomes received wisdom and subsequent criticism and retractions fail to enter the public consciousness, journalism fails in its duty to both science and the public. As long as science is news, journalists should learn to take the mundane footnote with the exciting headline.

See the original:

return to Howard Fienberg's page