PREDIMED Trial of Mediterranean Diet Retracted and Republished

The New England Journal of Medicine (NEJM) has retracted, and republished a new version of, the PREDIMED study of the Mediterranean diet following concerns about the validity of some of the data.

The journal has also made minor corrections to five other studies. All these corrections were published online June 13 in NEJM.

The corrections follow concerns raised about baseline data distributions by a UK anesthesiologist with a track record of exposing fraudulent research. 

John Carlisle, MD, Torbay Hospital, Torquay, United Kingdom, last year reported use of a screening tool that raised question about data in some studies.

These questions "may turn out to involve misinterpretation, statistical error, or plain simple mistakes," he said in a statement released at that time by his own journal, Anaesthesia, which published Carlisle's research June 5, 2017, and is among the journals implicated.

"However, on the basis of previous studies it is likely that some of the data highlighted in this latest research may have been deliberately falsified. At a minimum, it is clear that the reporting of some randomized, controlled trials may be seriously flawed," he wrote.

Carlisle told theheart.org | Medscape Cardiology that he is satisfied with the NEJM response. "They have done the responsible thing. They have found some errors and have corrected them. I am happy that these errors do seem to be minor technical mistakes in five out of the six papers they have investigated."

The PREDIMED study, originally published in 2013, showing a beneficial effect of the Mediterranean diet on cardiovascular outcomes, was found to have more extensive errors and so has been retracted and republished.

"When the authors looked at the data again, it was also found that the original results included some patients who had not been randomized correctly," Carlisle said. "That is quite a fundamental error. That paper has now been retracted and republished with the correct patient data, but the results have not changed materially." 

PREDIMED senior author, Miguel A. Martinez-Gonzalez, MD, University of Navarra Medical School,​​ Spain, commented to theheart.org | Medscape Cardiology that there was a "small departure from the protocol" during the randomization process.

"PREDIMED was a study of the Mediterranean diet and patients were supposed to be randomized individually, but about 10% of the population were allocated by household or by clinic," he said. "This is known as cluster randomization. This wasn't a cluster randomized study, although some studies do randomize n this way." 

Martinez-Gonzalez said the study authors did not realize this had happened for the initial publication, but they discovered the errors when they reanalyzed the data because of questions raised by Carlisle.

"We took the initiative to honestly report what we had found," he said. "We have reanalyzed the data taking into the account the cluster effect and found no difference in the results. The proper way to adjust for potential clustering effects in this small percentage was applied (robust estimate of standard errors, accounting for intracluster correlation)."

The 2013 unadjusted results suggested a hazard ratio for the risk for cardiovascular events of 0.70 for the Mediterranean diet plus virgin olive oil vs control, and the new 2018 adjusted version shows a hazard ratio of 0.69.

For the Mediterranean diet plus nuts, the 2013 unadjusted results suggested a hazard ratio of 0.70 vs control and the new 2018 adjusted version shows a hazard ratio of 0.72.

In addition, a secondary analysis removing those patients also had similar results — showing an even stronger beneficial effect of the Mediterranean diet, Martinez-Gonzalez added.

"Consequently, both Mediterranean diet interventions did lower the incidence of hard cardiovascular events."

Carlisle said the randomization errors raised questions about the overall conduct of the trial. "My concern is that a well-run trial should have very clear lines of management with a paper trail of each patient.  This does raise questions about whether there are other errors that the authors are unaware of."

Commenting for theheart.org | Medscape Cardiology, Steve Nissen, MD, Cleveland Clinic, Ohio, said the journal and the authors "deserve credit for their commitment to scientific accuracy in correcting this manuscript."

"Although the findings don't substantially change, our confidence in the results is weakened by the errors in randomization that may have introduced bias into the trial," Nissen said. "We need additional trials to clarify the potential benefits of a Mediterranean diet. Dietary trials are very challenging to conduct. Unfortunately, the PREDIMED trial was not conducted using the best possible methods. Although I do think the findings are probably still correct, the findings now need to be replicated."

"Standard Error" vs "Standard Deviation"

The minor errors in the other studies are reported to have been caused by inadvertent interchanging of the terms "standard error" and "standard deviation" in the baseline variables in the original papers.

"This is just a technicality, but it could be seen as a barometer of error rates," Carlisle commented. "If there are errors like this in the paper, then there could also be other errors in more important data, such as the main results."

"I questioned the baseline variables in many papers as the data did not seem feasible, which brought to light these errors in mixing up the standard deviation and standard error terminology," he added. "This was possible to do as baseline values should be similar as they are governed by chance. It is much more difficult to use the same techniques for the actual results of the study, which do not rely purely on randomization."

Carlisle analyzed data from a total of 5087 trials published in various journals. These included 934 studies published in the NEJM with baseline data that, he said, did not follow expected distribution patterns.  

"There have been cases in the past where study data has been fabricated, but it is quite difficult to make up data that follows a random distribution, and this is how the fabrication has come to light. So I thought the journals should check that out," he explained.

"The NEJM has looked at 11 trials which they say had the least likely distribution of baseline variables and in 6 of these it was found that the terms 'standard deviation' and 'standard error' had been mixed up."

An Editor's Note in the NEJM explains that: "An analysis of reports of randomized trials for improbable distributions of baseline data included 934 reports published in the Journal and identified 11 with distributions of baseline variables that did not appear consistent with randomization."

"We reviewed these 11 trial reports and replicated the Carlisle analysis," the note said. "In 5 reports, standard errors were inadvertently reported as standard deviations, or vice versa. These errors explained the findings and have been corrected."

The five reports are as follows:

• "High-Dose Atorvastatin after Stroke or Transient Ischemic Attack" (published in 2006);  

• "Treatment of Periodontitis and Endothelial Function" (2007);

• "Extended Antiretroviral Prophylaxis to Reduce Breast-Milk HIV-1 Transmission" (2008);

• "Effect of Bronchoconstriction on Airway Remodeling in Asthma" (2011); and

• "Horse versus Rabbit Antithymocyte Globulin in Acquired Aplastic Anemia" (2011).

"For 5 other reports, limitations in the Carlisle analysis, such as lack of accounting for correlation among baseline variables, most likely explained the findings," the NEJM Editor's Note suggests.

On the PREDIMED study, it says: "The authors' subsequent review of their data and randomization procedures identified protocol deviations, including the enrollment of participants who were not randomized. The authors have therefore withdrawn their original report.  The Journal has now published their new report of the study, which describes the protocol deviations and reports re-analyses of the data."

"Mixing 'standard deviation' and 'standard error' are very minor errors, similar to typos, and were made by the authors," Jennifer Zeis, manager, communications and media relations at NEJM, told theheart.org | Medscape Cardiology. 

They have now added "an extra level of review" to tables of baseline characteristics, she said. "After outside peer review, tables of baseline characteristics are now reviewed by a graduate student in biostatistics as well as by the editors and one of our consulting statisticians."

"We conduct a rigorous peer-review, and careful and extensive internal review, to assess the quality and validity of the research," Zeis added. "We interact extensively with the authors during the review and editing phases of the publication process. And at NEJM, it is the singular responsibility of our staff — a team of editors, statistical experts, illustrators, manuscript editors, proofreaders, and production staff — to ensure that each published paper meets our exacting standards."

"Healthcare professionals around the world, and the patients they serve, depend on us and we take that responsibility seriously," she concluded.

N Engl J Med. Published online June 13, 2018.  Retraction notice/editor's note; Republication  

For more Medscape Neurology news, join us on Facebook and Twitter