The Last Magazine

Doctoring the Data, Nursing the News?

by Iain Murray and Howard Fienberg

Summer 2000

There is a real emergency in the E.R. these days. Far from the professionalism of Dr. Ross and his colleagues, it seems the U.S. faces an epidemic of deaths and serious injury from medical screw-ups. Now the President has announced that he wants an integrated ‘serious' error reporting system and he is prepared to twist arms to get it. But this "cure" may be a erroneous prescription in itself; we aren't really sure what the ailment is.

According to an Institute of Medicine (IOM) report on dangers in American health care, between 44,000 and 98,000 hospital patients die each year because of medical mistakes, more than breast cancer, AIDS or highway accidents. The New York Times compared this figure on December 1 "to having three jumbo jets filled with patients crash every two days." But among the discussions of liability, confidentiality and reporting, few questioned the source of the numbers or the standards of measurement.

The jumbo jet analogy, while colorful, obscures the reality of the error count. The fatality figures originated in two studies, one of hospital discharges in New York in 1984, which found 129 fatalities in 30,000 cases, and another of discharges in Colorado and Utah in 1992, which counted 59 deaths in 15,000 cases. The IOM extrapolated these figures to the 33.6 million hospital admissions in 1997 - notably not to discharges, of which there were about 30.5 million in 1996 - and arrived at two guesstimates. These set the lower and upper limit for annual avoidable deaths at 44,000 from the Colorado/Utah evidence and 98,000 from the New York evidence.

We were given no idea as to whether or not these three states were representative of the country as a whole. And if the estimates had been made against hospital discharges, then the range would have been from 39,650 to 88,450 - a substantial decrease. As pointed out by the Detroit News (Dec. 6), "Any study whose estimates of errors varies by more than 100 percent is open to serious question." Perhaps the best indicator of the reliability of the figures is that, at the upper level, they would mean that 3 out of every 4 hospital doctors see one patient a year die as a result of medical error.

The measurement of medical error is as uncertain as the counting. One definition of the "adverse event," as the errors are euphemistically called, was when "an inappropriate decision was made when, at the time, an appropriate alternative could have been chosen." Could is emphasized because the alternative "should" was not used. By this definition, the doctor is assumed to be fully cognizant with all the circumstances, which is quite evidently not always the case. It fails to take potentially complicating circumstances into account. Surgeon Sherwin P. Nuland pointed out in the Wall Street Journal that, although evidence of error is sometimes clear, often the "association is indirect, hard to make, and debatable. Gathering such data simply isn't an exact science." The assessment of blame appears too subjective to allow an accurate calculation of total cases.

A particularly good example of this is deaths from medication-related errors. The IOM estimate of 7,000+ deaths per year is based on errors that resulted "from acknowledged errors by patients and medical personnel." In other words, if a patient takes an overdose or fails to inform their medical advisers of other conflicting medications they are taking, that is regarded as a medical error, rather than misadventure.

The American Medical Association is dead-set against any statutory reporting requirements, for the simple reason that in our litigation-hungry society, doctors do not need to be exposed to more opportunities for malpractice suits. Dr. Nancy Dickey, a past president of the AMA contends that these requirements would make doctors less likely to share experiences and discuss cases (New York Times, Feb. 22). Anything resulting in that is a backwards step on the road of medical progress. The fact that there is little evidence that reporting reduces errors, as the director of the Agency for Health Care Research and Quality freely admitted to a Senate hearing on 22 February, is another reason to make us ask why it is being proposed.

The complexity of modern medicine lends itself to greater error than we might have experienced in the past. Nonetheless, it also has the capacity to regularly treat an ever-widening number of ailments competently. With the growth of experience comes wisdom; perhaps that wisdom will eventually drip down to the media and public policy.

Iain Murray is senior analyst and Howard Fienberg is research analyst at STATS (the Statistical Assessment Service), a non-profit, non-partisan group based in Washington, DC dedicated to improving public understanding of scientific and statistical research.

The Last Magazine is from the people who brought you LM (there is life after the libel trial). Memo to lawyers: any resemblance to publications living or dead is entirely intellectual.

return to Howard Fienberg's page