Cancer Data Reproducibility Problem Undermines Public Trust

This transcript has been edited for clarity.

I'm Dr Maurie Markman from Cancer Treatment Centers of America. Over the next couple of minutes, I would like to discuss with you a very sobering, and in my opinion very distressing, paper entitled "Investigating the Replicability of Preclinical Cancer Biology" that appeared in eLife in 2021.

There has been information over a number of years, initially presented by an investigator from the MD Anderson Cancer Center and a lead investigator from Amgen, showing preclinical data in high-impact medical journals that were not reproducible. An effort has since been undertaken by investigators to look very systematically at this question in a number of high-profile preclinical research subjects and try to replicate selected experimental results from these high-impact papers.

These investigators, as outlined in detail in this manuscript, used multiple methods to assess reproducibility. They concluded — distressingly — that overall, in only 51 of 112 experiments (46%) were they able to reproduce the results. Furthermore, in 92% of the cases, the overall effect sizes were smaller than the original report.

Now, obviously, it's not easy to discuss this. And it's certainly inappropriate to come up with a single conclusion or reason for these outcomes. You can say that the original papers did not have enough information to be able to reproduce the results. You can say that the people presenting the results didn't know what they were doing, that they didn't have the proper equipment or expertise. There are many things you can say, but half of the results in high-impact papers couldn't be reproduced.

This, in my opinion, is a challenge to the cancer community. It is a challenge to the leaders of academic medicine that perhaps there's something wrong here.

Why can't these results be reproduced in such a high percentage [of studies]? Is there excessive [pressure] to publish positive results and one doesn't really that much care if they can't be reproduced? Or are investigators not expecting their results to be challenged?

Again, I'm raising a question, not providing an answer. One can only hope, however, that the academic leaders in this country at the NIH, which funds much of this research, will look at this issue very closely because it is the public that is actually reading the results of this research. Some may very well question whether money supporting research that can't be reproduced is well spent.

I would encourage you, if you have an interest in this topic, to read this paper and follow how the academic community deals with these questions. In my opinion, they're very important for the future of the public trust and support of cancer research in this country. I thank you for your attention.

Maurie Markman, MD, is president of medicine and science at Cancer Treatment Centers of America in Philadelphia. He has more than 20 years of experience in cancer treatment and gynecologic oncology research.

Follow Medscape on Facebook, Twitter, Instagram, and YouTube

Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.

Cite this: Maurie Markman. Cancer Researchers' Data Reproducibility Problem Undermines Public Trust - Medscape - Apr 27, 2022.

Commenting is limited to medical professionals. To comment please Log-in.

Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.