Find the latest COVID-19 news and guidance in Medscape's Coronavirus Resource Center.
This transcript has been edited for clarity.
Welcome to Impact Factor, your weekly dose of commentary on a new medical study. I'm Dr F. Perry Wilson from the Yale School of Medicine.
When we look back on 2020, what will we call it? The year of the pandemic? The year of democracy? From a medical publishing standpoint, it's clear: 2020 is the year of the preprint.
Preprints: medical manuscripts published for all to see — before peer review.
The promise of preprint servers is nothing less than the democratization of medical science. Free, open publishing so researchers and readers of research can come together and make science better. But like all good ideas, it's about the execution.
While preprint servers like arXiv have been running for decades servicing the math and physics community, the medical research world has only more recently embraced bioRxiv (often for basic science papers) and the newcomer to the scene, medRxiv, for the clinical sciences. Full disclosure: medRxiv is run out of Yale, and I have nothing to do with it.
And according to this research letter in JAMA, medRxiv is taking off in 2020, thanks, of course, to COVID-19.
medRxiv only began in June of 2019. Check out the growth in submissions over the past year.
You don't need to be a statistician to see the sharp uptrend starting in February of 2020. The site went from a median of five submissions per day pre-2020 to 51 per day this year. Fully 73% of those submissions were COVID-19 related — a staggering share of research for a single pathogen, and a testament to the fact that when speed of publication is an issue, preprint servers really shine.
Now, medRxiv is not the only preprint server. Far from it, though it has become the go-to for COVID-19 preprints. Another research letter in JAMA examines the other major preprint servers — 57 in total — and shows us that preprints still have a ways to go to live up to their promise.
The researchers identified 18 best practices for research transparency and reporting and reviewed which preprint servers required which policies.
These are policies like data sharing, ethics approval, funding declarations — that sort of thing. The median server addressed just one of the 18 best practices. As beacons of open science, some preprint servers are sputtering.
And preprint servers have had their controversies. Many of the manuscripts — 86% on medRxiv, for example — have not (yet) made it into the peer-reviewed literature. This doesn't mean that they are fatally flawed, but the absence of peer review means there is little opportunity for scientific quality control. Many of these papers would never have seen the light of day had it not been for a preprint server. Whether that's a good thing or a bad thing depends on the paper.
There have been some notable retractions in the COVID-19 era, like this article, which implied that smoking might be protective in COVID-19, and this one, which addressed the fraught issue of hydroxychloroquine for COVID treatment.
But peer review does more than just find shaky data or the occasional fraud. I've been through a lot of peer review myself, and while people often think of peer review as being about rejecting bad science or suggesting new experiments, a lot of it is about moderating language. I've conducted an experiment, I believe it shows X, and I write that. The peer reviewers often act to say, Be careful; you can't be sure you've proven that. Tone it down a bit. That's a critical part of the process and one that doesn't happen at all on the preprint servers.
And that's really where we get into trouble with preprint servers. It's not the servers; it's what we do with them. And that includes people like me who write about medical studies. News outlets have been trolling through medRxiv to get the scoop on the latest COVID-19 science, often with a minimal nod to its preliminary, non–peer-reviewed nature.
If I were in charge of an editorial desk, I would simply instruct my reporters not to use sources from preprint servers; the risk for misinterpretation is just too high. Peer review reduces hype. That means that the peer-reviewed literature doesn't always make for the most exciting headlines — but it does make for better science.
F. Perry Wilson, MD, MSCE, is an associate professor of medicine and director of Yale's Clinical and Translational Research Accelerator. His science communication work can be found in the Huffington Post, on NPR, and here on Medscape. He tweets @fperrywilson and hosts a repository of his communication work at methodsman.com.
Follow Medscape on Facebook, Twitter, Instagram, and YouTube
Medscape © 2020 WebMD, LLC
Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.
Cite this: The Problem With Preprints in Clinical Science - Medscape - Nov 11, 2020.
Comments