Early Cancer Drug Results: Are We Celebrating Too Soon?

Each year, we see tremendous enthusiasm surrounding top-line results for cancer drugs presented at the American Society of Clinical Oncology (ASCO) and European Society for Medical Oncology (ESMO) annual meetings. The fanfare often continues on social media, with some calling these drugs a "game changer" or "new standard of care," despite only seeing an abstract or presentation about the trial.

But, for the sake of our patients and science, it is important for oncologists to read these clinical trials objectively and critically. I know this is easier said than done. And with the ESMO meeting right around the corner, I recently teamed up with my colleagues from the ESMO-Magnitude of Clinical Benefit Scale working group to develop a checklist to evaluate issues and identify flaws in clinical trials. Here, I have highlighted a series of important questions in 13 areas to help oncologists, as well as journalists, take a step back and carefully assess new cancer drug findings before getting swept up in the hype.

1. The trial type: First, is the study a single-arm or a randomized controlled trial (RCT)? Single-arm trials may be fraught with limitations, and responses are not necessarily interchangeable with those from RCTs (see examples in hepatocellular cancer trials).

2. The patient population: When evaluating RCTs, is the trial conducted in an appropriate or representative sample of patients? Look at the eligibility criteria and ask yourself, would your typical patient get into this trial or are the criteria too strict or narrow?

3. The control arm: Is the control arm treatment standard and of high quality? Is that what you would have used for your patients as standard of care?

4. The primary endpoint: What is the trial's primary endpoint? Is it overall survival or progression-free survival? For trials using progression-free survival, research shows this endpoint is not necessarily an adequate proxy for overall survival or indication of patients' quality of life.
   
   Also ask yourself, is the primary endpoint a surrogate marker, and is the surrogate validated in that context? In breast cancer, for example, here's an overview of what surrogates are valid and what surrogates are not valid.

5. Conflicts of interest: Who wrote the abstract, and what are the authors' conflicts of interest? Do the authors, for instance, receive money from industry? What conflicts of interest do the discussants or people celebrating the abstract have?

6. Noninferiority trials: Was a noninferiority design justified? What criteria were used to define noninferiority, and would you consider that limit acceptable? Noninferiority designs are justified if the newer agent provides any compensatory benefits, such as greater ease of use or less invasive (oral vs intravenous, less frequent administration), lower cost, or better quality of life.

7. The crossover design: Was a crossover design included in the trial, and was it appropriate? If a drug is approved in later lines of treatment and this new trial is testing the same drug in earlier lines of treatment, a crossover design is desirable; however, for a drug that has never been approved in that setting, crossover is not desirable.

8. The duration: Was the trial stopped early? If the trial is stopped early, the actual effect size may be less impressive than the reported effect size.

9. The trial location: Where was the trial conducted? In the case of multinational trials, do the results differ substantially across the geographies? What percentage of patients received standard-of-care treatment after progression?

10. Censoring: What percentage of patients were censored? For trials reporting progression-free survival, does the duration differ substantially from time to treatment failure, and what were the treatment dropout rates? The impact of informative censoring is particularly important for trials reporting progression-free survival but not overall survival benefit.

11. Quality of life: Are quality-of-life data reported, and was there an improvement? For trials focused on progression-free survival benefit, patients' quality of life must improve to conclude that the drug has a clinical benefit. "Not worsening" quality of life does not signify clinical benefit.

12. The toxicities: What are the fatal or serious adverse events from a drug, and how many patients were affected? What were common grade 1 or 2 side effects that could also affect quality of life? Trials that report "tolerable," "manageable," or "acceptable" drug toxicity profiles often underestimate the harms of these agents.

13. The subgroup analysis: Subgroup analyses can come with a host of limitations and misconceptions. If a trial highlights a subgroup finding, was the subgroup analysis planned? Did alpha-splitting occur, or is it a post hoc subgroup? Are the findings consistent with what is observed in other trials?

I hope this checklist of questions helps people get started, and for a deeper dive, check out our paper.

Bishal Gyawali, MD, PhD, is an associate professor in the Department of Oncology and Department of Public Health Sciences and a scientist in the Division of Cancer Care and Epidemiology at Queen's University in Kingston, Ontario, Canada. He is also affiliated faculty at the Program on Regulation, Therapeutics, and Law in the Department of Medicine at Brigham and Women's Hospital in Boston. His clinical and research interests revolve around cancer policy, global oncology, evidence-based oncology, financial toxicities of cancer treatment, clinical trial methods, and supportive care. He tweets at @oncology_bg.

Follow Medscape on Facebook, Twitter, Instagram, and YouTube