Recent Journal Retractions: Flawed or Fraud?

Clinicians, researchers, and health reporters are accustomed to believing articles published in the best medical journals. No wonder, then, that many people were shocked when the two oldest medical journals, The Lancet and The New England Journal of Medicine, recently retracted articles on a highly relevant topic at about the same time.

I have no inside information, but I do have some insights on medical journals generally, having served as editor-in-chief of JAMA for 17 years. I also founded the International Congress on Peer Review in Biomedical Publication (now called the International Congress on Peer Review and Scientific Publication) and co-founded the World Association of Medical Editors (WAME).

So, what happened?

The NEJM paper, published May 1, 2020, had five authors, including two involved in the retracted Lancet study. The conclusion read:

Our study confirmed previous observations suggesting that underlying cardiovascular disease is associated with an increased risk of in-hospital death among patients hospitalized with Covid-19. Our results did not confirm previous concerns regarding a potential harmful association of ACE inhibitors or ARBs with in-hospital death in this clinical context.

On June 4, 2020, all five authors retracted the paper, stating that they and a third-party auditor were not granted access to the primary data in order to validate the study.

The Lancet paper, published online May 22, 2020, was based on a massive multinational hospital registry. This time the authors concluded:

We were unable to confirm a benefit of hydroxychloroquine or chloroquine, when used alone or with a macrolide, on in-hospital outcomes for COVID-19. Each of these drug regimens was associated with decreased in-hospital survival and an increased frequency of ventricular arrhythmias when used for treatment of COVID-19.

The retraction notice on June 5, 2020, written by three of the four original authors, gave the ostensible reason for retraction as the fact that new peer reviewers assigned by The Lancet were prevented from examining the original data, due to contractual terms between the data company and the hundreds of institutions that participated.

Critics hurried out of the woodwork, impugning the motives of the authors, editors, and institutions involved.

Errors in the medical literature are really quite common. After all, there are literally tens of thousands of medical periodicals. Most errors are neither noticed nor commented upon because the work involved is of little meaning and few readers probably even notice. But some matter a lot.

No medical editor promises that what he/she published is the "truth." But all the best ones promise that they faithfully follow processes that are intended to produce a result as close to the "truth" as is possible at that point in time. Since 1978, the International Committee of Medical Journal Editors (ICMJE) has published best practices for medical journal authors, editors, publishers, and others.

The ICMJE document defines the responsibility of authors as an "agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved."

Both the NEJM and The Lancet state that they follow ICMJE recommendations. Thus, one may assume that all authors of both articles signed forms attesting to the veracity of the data and assured the editors that questions about the data would be resolved by additional queries.

Increasingly, medical research relies on large databases. Unable to peer into the "black box," an editor must trust the ability and the morality of the authors. If the editors or peer reviewers raise serious questions about the integrity of the data (including its very existence), the named authors must be willing and able to satisfy those questions.

Medical journals can easily be duped by skillful liars. The editors, at their heart, trust the authors to tell the truth. That is why successful perpetrators of research fraud often become repeat offenders.

But there is a big difference between "flawed" and "fraud." Flawed studies are typically discovered by editorial or peer review and not published, unless revised to correct the problems. If flaws are discovered after publication, these flaws are typically reported and debated in the Letters column. If serious, the journal process usually leads to a published "correction."

Fraudulent work, discovered after publication, typically leads to retraction. Characteristically, the retraction notices are vague, smell of lawyer language, and do not reveal what really happened.

No one has ever claimed that the peer-review process is a guarantor of truth. That is why we invented the International Congress in Chicago in 1985, intended to turn the "art of peer review" into a science. Meeting quadrennially ever since, this congress has developed and widely disseminated much new knowledge and understanding about peer review.

Postpublication peer review is often the best test of a study's integrity. That is what happened in this case. All readers and potential readers become the reviewers.

Recognition of error was rapid and retraction of both articles probably established world speed records. What really happened? Do the data actually exist or were they all fabricated? Did the Chicago data company, Surgisphere, do legitimate work for other clients over its many years of existence?

We don't have those answers, but we do know that of the parameters for editing medical journals — speed, economy, and accuracy — the greatest of these is accuracy.

Let the medical meetings, the listservs, the preprints, and social media go fast. That is their role and they serve it well as a form of data sharing, crowdsourcing, unvarnished open peer review, even postpublication peer review of drafts.

But once an article appears in the top medical journals, I expect to be able to trust that the information is as accurate as it can be, speed and cost be damned.

That's my opinion. I'm Dr George Lundberg, at large at Medscape.

Follow Medscape on Facebook, Twitter, Instagram, and YouTube