Playing the Game with Statistics – and Playing it Wrong

Author of a recent article published  in Medicine, Health Care and Philosophy have taken a hard look at the 1986 article in the New England Journal of Medicine by Karin Nelson and Jonas Ellenberg that led the medical community to sing in chorus that lack of oxygen was rarely a cause of cerebral palsey.

The new article finds that the central argument of the 1986 article relies on "straightforwardly fallacious statistical reasoning."  The author’s concern is that the 1986 article  improperly influences "how carefully fetuses are monitored during labor and delivery, expert testimony in malpractice cases, and public policy decisions."

(Remember my recent post on the birth-related injuries prevented by Seaton hospitals after they instituted various measures to provide more uniform care to expectant mothers?  If lack of oxygen is rarely a cause of cerebral palsey why did those (and other) injuries decrease by almost 90% when new procedures were adopted?)

You can purchase the article here, although I must tell you that if you lack a solid background in statistics it is a challenge to read and comprehend.  I would post the article for you but I am not sure I have the right to do that since you have to pay ($30) to purchase it.

What I find interesting about the article is that it provides more evidence that we must never accept as true what we read in medical literature.  As recent experience as shown regarding the risks associated with consumption of Vioxx, information contained in medical journals is not necessarily true or complete.  The authors of some of the articles  have an agenda other than helping other doctors treat patients, or they may be receiving compensation from, say, a drug or device manufacturer that may cause reasonable people to give different weight  to the findings and conclusions expressed.

Over time, things tend to work out.  By that I mean that erroneous positions in medical articles will be exposed over time.  One problem is that in the meantime tremendous harm can be done to patients and litigants when "experts" rely on medical or scientific literature that lacks a solid basis.

The other problem is the resources to attack "bad" literature.  It costs alot of money to conduct solid scientific research?  Who pays for the effort?  A competing drug company or manufacturer?  The government?  Plaintiff’s lawyers?

Three things must be done at a minimum.  First, the editors of the publications must conduct true peer review of the research, insisting that the researchers disclose all underlying data, assumptions, etc.  Second, the editors must insist upon complete disclosure of all economic and other ties between the authors, their institutions and families with any entity that could economically benefit from the publication of the research.  The disclosure should be a "black box" disclosure, such as you find in the PDR about very significant adverse risks attendant to the use of certain medications.

Finally, judges must be educated that the mere fact that something is published in medical or scientific literature doesn’t make it Gospel.  There is a tendency for all of us to assume that if something finds its way to a publication it must be accurate, particularly if the publication is well-known.  But, as the recent experiences with Vioxx tell us, even the folks at the NEJM can be lead astray.