Hello and welcome. I'm Dr George Lundberg, and this is At Large at Medscape.
Back when I was in graduate school, my biochemistry professor Alan Hisey referred me to his favorite journal, called the Journal of Irreproducible Results . As a biochemist, he particularly liked the way that journal focused on studies about the chemical noacetol.
The same publication, long before our emphasis on sensitivity, specificity, and predictive values, published a surefire test to distinguish your a... from a hole in the ground. 100% sensitivity and specificity, never wrong. A perfect lab test.
If any reader wants the punch line, email me individually.
I had not the slightest idea back then that I would spend the bulk of my professional life as a medical journalist. And I surely had no inkling that most rank and file medical science journals would themselves become unwitting repositories of a trove of irreproducible, or at best unreproduced, results.
Of course, John Ioannidis has famously blown the whistle on how much of what is published in even the leading medical journals is simply wrong.[1]
And in June 2017, the Friends of the National Library of Medicine hosted a 2-day program at the National Institutes of Health (NIH) titled "Consequential and Reproducible Clinical Research: Charting the Course for Continuous Improvement." Nice positive spin on a very disturbing past and present.
Who knows how much published preclinical and clinical work has not been or could not be reproduced? We do know that vast sums have been expended trying to develop useful clinical drugs, devices, and procedures on the basis of seriously flawed preclinical research.
Longtime National Public Radio reporter Richard Harris has a new book out entitled Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions.[2] It details some of the most flagrant examples of errors. They will curl your hair or cause it to fall out.
How to fix this mess?
All involved in the scientific enterprise should recognize that irreproducible (or unreproduced) published reports are common and a serious problem.
The scientific community should cultivate a culture of increased scrutiny and criticism about research reports.
Real (not fake) journals should improve their prepublication peer review process and procedures.
Medical and science journals should welcome and give higher priority to studies that attempt to reproduce studies that have already been published, even in the same journals, rather than always insisting that the results that they publish must be novel.
The NIH and other funding agencies, and their review committees, should give a higher priority to funding projects intended to support or refute research that has already been published.
The academic personnel processes of the best medical schools should reward faculty whose research tests the reproducibility of prior published studies. Such work is needed and worthy.
Academic personnel processes should cease to rely on misleading measures, such as the impact factors of journals, as a prime way to evaluate the quality of a faculty member's work. Primacy of such measures precludes incentive to reproduce studies.
Postpublication peer review should be enhanced by widespread use of more formal methods of assessment.
This is a massive problem; it demands a massive change in values, processes, and rewards, and it must be done. The time to begin is now.
That's my opinion. I 'm Dr George Lundberg, At Large for Medscape.
Medscape Internal Medicine © 2018 WebMD, LLC
Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.
Cite this: George D. Lundberg. Irreproducible 'Scientific' Results - Medscape - Mar 06, 2018.
Comments