Bad Science Never Dies
by Howard Fienberg
Splashy science news reports draw eyeballs and move policy, but sometimes the scientific heart of the news comes up short. Worse, it can be dead wrong. So what happens in the news when a study is found to be faulty or false and ends up being retracted or thrown out?
Not much, usually. Science news revolves around news -- new studies, discoveries and achievements. The discovery that previous research has been dis-proven or shown to be worth less than the paper it was printed on just does not register as news to most journalists, no matter how said research was originally hyped to the public.
This is understandable. After all, journalists often work for publications that don't worry much about correcting the public record. Most newspapers and magazines print correction columns, but they can be hard to find. Few publications admit in big type that they were wrong.
Of course, when the media make a mistake, it usually is not earth shattering. By contrast, scientific errors can spread and leave even more bad science in their wake.
A study by Dr. John M. Budd et al. in the Journal of the American Medical Association (Jul. 15, 1998) examined 235 scientific journal articles that had been formally retracted due to error, misconduct, failure to replicate results, or other reasons. The researchers reported that, "Retracted articles continue to be cited as valid work in the biomedical literature after publication of the retraction."
Budd and his colleagues acknowledged that there is sometimes a significant time lag (an average of 28 months) between publication and retraction. But they found that the flawed articles were cited in the scientific literature an astonishing 2,034 times after they had been retracted. The vast majority of these post-retraction citations treated the work as still valid, making no reference to the retraction.
At a certain level, these studies have become urban myths. Despite no longer possessing scientific authority, their repeated publication has let them take on a life of their own -- regardless of any grounding in truth. Such scientific myths are worse than simple scare stories about kidney stealing or the influence of the full moon, because future researchers unwittingly depend upon their (invalidated) analyses.
For example, take the finding back in November that
genetically modified corn had infiltrated regular strains in
However, when the media responded to this news, they concentrated on political aspects, not the important scientific ones. The Washington Post reported that one of the authors of the original article "believed the effort to undermine" the study "was the work of biotechnology advocates, some of whom had personal reasons for attacking him." Rather than laying out the science and the criticism, the Post reduced the matter to a 'he said, she said' narrative, concentrating on personal and political motives rather than the merits of the research. This kind of narrative may follow the dictates of allegedly objective journalism, but it doesn't explain very much.
Further in the political vein, Guardian writer George Monbiot (May 14) dedicated a lengthy article to investigating the supposed public relations campaign against the article, a campaign "so severe" that it "persuaded" the journal to retract it (never mind the methodological problems in the article, which Monbiot called "hardly unprecedented in a scientific journal").
And, as if to demonstrate the conclusions of JAMA's 1998 study of retractions, the Washington Times
(Apr. 30) ran a center-page spread on the Mexican corn infiltration, along
with photos from an anti-GM crop demonstration outside the Mexican consulate
Since even scientists must rely on the news media for much of their science news (an endless array of journals defy any sane person to keep track of them all), they might miss the Nature retraction, too. It won't be long before other journal articles cite the (retracted) study.
When initial research becomes received wisdom and subsequent criticism and retractions fail to enter the public consciousness, journalism fails in its duty to both science and the public. As long as science is news, journalists should learn to take the mundane footnote with the exciting headline.
See the original: http://www.tcsdaily.com/article.aspx?id=052002A
return to Howard Fienberg's page