What does the PREDIMED trial retraction and reboot mean for the Mediterranean diet?

A very influential nutrition trial just tanked. It was retracted from the New England Journal of Medicine (NEJM) on 13 June, and re-published with new analyses and toned-down conclusions.

Both Gina Kolata, writing in the New York Times, and Alison McCook, writing at NPR, imply, at least to some extent, that it might make no difference to the evidence. But I disagree.

Here's what's happened to the trial, and where I think it leaves the overall evidence.

Called PREDIMED, it was a multi-center trial from Spain, with the NEJM final report published in 2013. Altogether, 7,447 people at risk of cardiovascular disease (CVD) – heart attack and stroke – were reported as randomized to one of 3 groups:

• Mediterranean diet with free olive oil provided, along with individual and group training sessions at the start, and then quarterly;
• Mediterranean diet with free nuts provided, along with individual and group training sessions at the start, and then quarterly;
• Advice to reduce fat intake, with a leaflet – but after the first 3 years, people in this control group could also participate in the Mediterranean diet training sessions.

The primary endpoint for the trial was a composite one of major cardiovascular events: myocardial infarction, stroke, or CVD-related death. And the trial was stopped early. More people dropped out of the control group than the Mediterranean diet groups.

There are several alarm bells here already, and we'll come back to those.

The road to retraction of PREDIMED didn't start with those issues, though. It began with a piece of meta-research by John Carlisle in 2017. He listed PREDIMED as an example of a trial branded "randomized", where the data didn't look consistent with randomization to him. NEJM followed up – and the authors dug into their data.

It turned out Carlisle was right. The authors have now reported that randomization had gone awry for 21 percent of the participants – 1,588 of the 7,447 people in the trial. About a third each were for one or more of these reasons:

When more than one person in a house was recruited, instead of randomizing each, they were all assigned to the same diet;

• At one site, the randomization table hadn't been used properly; and
• At one site, clinics were randomized instead of people.

The authors have gone to a lot of trouble to re-analyze. And they say that even without these 1,588 people, the results are much the same. They tone down their conclusion because as the trial wasn't properly randomized, the evidence is now weaker.

Unfortunately, though, I couldn't find data showing what the impact of these 1,588 people were on the components of the composite endpoint. (The 3 components are all secondary endpoints, but I couldn't find a sensitivity analysis for them – let me know, please, if you see them!)

Even for the full group, there was no statistically significant difference on myocardial infarction or CVD mortality – just for stroke. And in the supplementary information, there wasn't a difference in the Kaplan Meier analysis for stroke either.

So what have critical systematic reviewers had to say about the quality and reliability of the PREDIMED trial, even before this bad news landed? And how strong is the evidence overall for the Mediterranean diet after this retraction?

The authors list a large number of systematic reviews/meta-analyses in their supplementary information, to support their certainty that health benefits for the Mediterranean diet are established. But they don't report the basis for selecting that list. And there are, generally, more unreliable systematic reviews than very good ones.

I looked at two sources of reviews for a quick echeck: the systematic reviews listed in PubMed as citing the 2013 paper (n = 9) and cited in the HANDI assessment of the Mediterranean diet (n = 1). The systematic reviews listed with the trial in PubMed are those selected for PubMed Health with citations accessible. That means they are more likely to be higher quality systematic reviews – here's a post from me of some rules of thumb on this. (Disclosure: In a previous role, I had recommended the providers of systematic reviews included in PubMed Health.)

Out of those 10 reviews, 6 of them were directly relevant and of acceptable quality. Only 1 of them was included in the PREDIMED paper's list of systematic reviews/meta-analyses. (You can see a summary of all 10 reviews below this post.)

I think the strongest and most recent of these reviews were from organizations that specialize in systematic reviews: 2 from the Cochrane Collaboration and 1 from NICE. The 2 Cochrane reviews rejected the PREDIMED trial from consideration, even before these revelations, primarily because of taking issue with the control group.

The reviewers from NICE assessed the PREDIMED trial as at "serious" risk of bias for individual CVD outcomes, with "low or very low quality" data – and underpowered for mortality outcomes (even at the full complement). In addition, they wrote, the trial

was difficult to interpret because the control group was advised to reduce their fat intake and to follow some of the components of the Mediterranean diet.

Even before this retraction, the evidence base for the Mediterranean diet wasn't really a slam dunk. There have been several trials, but they are riddled with problems – and what's called "the Mediterranean diet" varies a lot.

So how does this evidence apply to you? Keep in mind, these people were at relatively high risk of CVD, with the men 55 and older, and the women 60 and older. So if you're at less risk that that, these results don't necessarily apply anyway. And we don't really know what aspects of a Mediterranean diet might make a health difference to people from other countries.

If you are keen on trying to stick to a Mediterranean diet, HANDI has a guide here. It's not lots of pasta and pizza and red wine! And speaking of red wine, check out this article by Julia Belluz in Vox, on the shifting evidence about even moderate alcohol use. If there does turn out to be an adverse effect to the popularization of the Mediterranean diet, it could be in the idea that the best diet includes regular red wine.

The Mediterranean-food-and-red-wine hypothesis is really appealing – at least to some social groups. But that could be a problem. If we're not particularly careful with health claims that are attractive to us, there's a good chance we're going to be disappointed.

This story is republished courtesy of PLOS Blogs: blogs.plos.org.