Five Reasons I Don't Believe an Imaging Test Improves Outcomes

In recent years the obvious trend in cardiovascular medicine has been toward the incremental. This is the cost of progress; easy gains have already been made.

Yet on the first day of the 2018 European Society of Cardiology (ESC) meeting, authors of the SCOT-HEART trial ask us to believe that an imaging test—not a drug or a procedure—can reduce cardiovascular outcomes in low-risk patients who present with chest pain by 41%.^[1]

Their conclusion in the New England Journal Medicine reads: "The use of coronary computed tomographic angiography (CTA) in addition to standard care in patients with stable chest pain resulted in a significantly lower rate of death from coronary heart disease or nonfatal myocardial infarction at 5 years."

If you believe that, well, you could believe just about anything.

I will first describe SCOT-HEART, then offer five reasons to doubt their conclusion. In the end, I also hope to have shown the need for basic critical appraisal.

The Trial

SCOT-HEART is an open-label multicenter trial of more than 4000 patients with stable chest pain who were referred by primary care doctors to 12 outpatient cardiology clinics in Scotland. All patients had regular clinical evaluation, including exercise electrocardiography (ECG) if deemed appropriate, before being randomly assigned to either standard care or standard care plus CTA. Nearly 90% of both groups had ECG stress tests at baseline.

In the first publication from SCOT-HEART,^[2] researchers reported that CTA clarified the diagnosis of coronary artery disease and led to changes in subsequent therapy—more preventive therapy and more coronary angiography. At 1.7 years of follow-up, death due to coronary heart disease (CHD) or nonfatal myocardial infarction (MI) was lower in the CTA arm (38% relative; 0.7% absolute risk reduction) but the difference did not reach statistical significance.

This most recent analysis of SCOT-HEART reported 5-year outcomes of the long-term primary outcome of death from CHD or nonfatal MI. In the standard care group, 81 (3.9%) patients had such an event vs 48 (2.3%) in the CTA arm. The hazard ratio was 0.59 (95% confidence interval, 0.41 - 0.84). Nonfatal MI accounted for almost all of the difference in event rates.

The other major finding concerned subsequent therapies. Short-term follow-up of SCOT-HEART showed higher rates of coronary angiography in the CTA arm, but at 5-years, rates of procedures were similar in the two groups. Second, 97 more patients in the CTA arm received preventive therapies than the standard group (19.4% vs 14.7%).

Reasons to Doubt This Finding

1. Preventive therapies are not that good

Because the rate of coronary procedures did not differ between groups, the authors, and editorialists, suggest that more preventive therapy in the CTA arm led to the 33 fewer events. Cursory thinking makes sense of this claim because both aspirin and statins reduce the rate of cardiac events. The problem comes when you run the numbers.

I reached out to Andrew Foy, MD, from Penn State Heart and Vascular Institute, my coauthor on a meta-analysis^[3] of CTA vs standard functional stress testing. By email, he wrote that if you apply the modest average risk reductions with aspirin and statins to the 97 more patients in the CTA arm who received the two drugs, there's no way to get 33 fewer events. If you assume a number needed to treat (NNT) for aspirin/statin therapy of 50 to prevent one death/nonfatal MI, then only 2 of the 97 patients would have had events prevented, he explained.

Perhaps a better way to explain the folly of saying preventive therapy caused the lower event rate in the CTA arm is that it would require the NNT of aspirin/statin therapy for primary prevention to be 3!

2. Implausible effect sizes

Foy also noted that implausibility is evident when considering the prespecified power analysis. The authors assumed a 5-year-event rate of 13.1%, which would give the trial 80% power to detect an absolute difference of 2.8 percentage points lower in the CTA group than in the standard-care group. Instead, the actual rate of events of 3.1% was far lower than expected and the actual absolute difference was 1.6%, which yields a 41% relative risk reduction, almost double what was expected.

Fragility of results

Not all P values are equally robust. Trials that have small numbers of events can be susceptible to the play of chance or patients lost to follow-up.

The fragility index^[4] of a trial describes the number of nonevents that need to become events in order to render a result nonsignificant. Using an online tool, I calculated the fragility index of SCOT-HEART to be 10, meaning it would take 10 misclassified nonfatal MIs for the primary endpoint to become not statistically significant.

Ten events seems like a lot, except nonfatal MI events are hard to sort out, even when one has clinical data. This quote comes the methods section of SCOT-HEART: "There was no formal event adjudication, and end points were classified primarily on the basis of diagnostic codes."

The other reason to suggest fragility of the results is that the number of patients lost to follow-up, 40 in the standard arm and 26 in the CTA arm, exceeded the number of misclassified events (10) it would take to make the results nonsignificant.

4. Bias

The authors wrote that this was an open-label trial with nonblinded event adjudication, and ascertainment bias is inherent to the trial design.

In simpler words: Attending physicians who favor CTA might treat patients in the CTA arm with more care, or they might code a possible event in a patient in the CTA arm as a troponin leak rather than an MI. And the opposite could happen in the standard care arm, where MI coding could have been more rigorous.

5. Less aggressive stress testing in standard care arm is not a factor

The two authors of the supportive editorial^[5] argued that a possible reason for SCOT-HEART's positive results in contrast to the PROMISE trial,^[6] which found no incremental benefit for CTA vs functional testing in low-risk patients, was less use of functional stress testing in the standard care arm of SCOT-HEART. In the comparator arm of PROMISE, most participants received a functional stress test (stress echo or stress nuclear), but in SCOT-HEART only 10% of the control arm had a functional stress test. The argument is that suboptimal detection of coronary disease in the standard care arm of SCOT-HEART may explain its higher rate of events.

That argument belies data on functional testing in low-risk patients. A recent secondary analysis of ROMICAT-II,^[6] a randomized controlled trial studying use of CTA as a first diagnostic test in patients with chest pain, looked specifically at outcomes in patients who did not undergo any noninvasive testing. The authors found that in low-risk patients presenting with acute chest pain, those who received clinical evaluation alone did not have higher rates of major adverse events but did have shorter length of stays, less downstream testing, less radiation exposure, and lower costs of care.

Conclusion

My specific message is that these results should not convince you that detecting the presence of disease within the coronary artery with CTA improves outcomes. This is important because if CTA becomes accepted common practice, radiation exposure and costs would increase. In an era of expanding costs of healthcare, adding another flat-of-the-curve ^[7] low-value practice hardly helps patients or society.

The broader and perhaps more important message of this critique is that such an implausible conclusion reached the highest level of medical science. The editorialists conclude that "the more general message from these trials is that the information provided by a diagnostic test can resonate therapeutically beyond making a correct diagnosis of coronary artery disease and that clinicians should aggressively pursue preventive measures to achieve the best outcomes possible while minimizing daily symptoms."

I would choose a different general message from these trials. It would be to cite the late physicist Richard Feynman, who said, "The first principle is that you must not fool yourself—and you are the easiest person to fool."

Comments

Commenting is limited to medical professionals. To comment please Log-in.

Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.