Larger sample sizes needed to avoid false negative findings in vitamin D trials

Researchers from Trinity College Dublin have developed a novel set of tools for designing vitamin D clinical trials that capture large seasonal and population-wide differences in vitamin D status, typically seen in individuals. Their study published in the journal Scientific Reports (today, Monday 31st May 2021) provides a framework for clinical trials to establish whether vitamin D supplementation is effective against a given disease.

The study also reveals that many trials which failed to find any association between vitamin D and disease prevention may have been underpowered or conducted without enough subjects to detect a benefit of vitamin D.

• Vitamin D deficiency has been linked with approximately 200 diseases, including most recently Covid-19. The link between vitamin D and disease can be causal—in such cases, vitamin D treatment is an effective way of preventing or treating the disease, for example rickets. However, due to an interplay with other factors vitamin D may only appear to affect health. In these cases, supplementation is not of benefit.
• Results from many observational and experimental studies have strongly suggested that vitamin D supplementation is beneficial for health. However, randomized control trials (RCTs) have, in many cases, failed to show benefit. RCTs can help us distinguish between direct (causal) and indirect links, and therefore are the gold standard approach to identifying conditions where treatment is truly beneficial. The substantial disagreement in conclusions drawn from observational studies and RCTs on benefits of vitamin D has fuelled a heated debate in public health and limited the range of disease for which vitamin D is recommended as a treatment, or preventative aid.
• Some concerns about RCT design that are specific to vitamin D are coming to the fore of this discussion. In particular, results from trials that found no benefit have been called into question, because the ability to spot effects of vitamin D supplementation might have been weakened by large differences in vitamin D status between participants. For example, it is not reasonable to expect improved health in individuals who, at the outset, have good vitamin D status: even if disease could 100% be prevented with vitamin D, a trial would find no benefit of supplementation if all participants were vitamin D sufficient, because there is no "room for improvement".

Study synopsis

• The researchers approached the problem from the individual perspective, simulating vitamin D status over a year, with peaks in the summer and troughs in the winter. The modelled fluctuations were unique for each person, to allow for the differences that exist between people.
• This study for the first time systematically examined key issues which can interfere with vitamin D trials, such as the differences between people's baseline vitamin D status and natural seasonal fluctuation due to changing intensity of solar radiation.
• The study demonstrated that individual differences in vitamin D status among trial participants, coupled with seasonal fluctuations, can have detrimental impacts on a trial's ability to detect true causal effects.
• Based on this work, it is possible to take into account specific characteristics of the study population when planning a trial, to ensure sufficiently large sample size that will enable investigators to see a benefit of vitamin D supplementation, if such a benefit exists.
• The study particularly focused on determining the right sample size requirements for a successful vitamin D RCT. Given the population and seasonal variations, a minimum size is needed to give a trial enough statistical power to be able to detect a treatment effect when one exists.
• The results suggest that the sample sizes used in some RCTs that failed to detect a treatment effect were underpowered. In these cases, we cannot know if vitamin D supplements didn't work (true negative), or if the sample size was too small to pick up an effect obscured by noise (false negative). The results suggest many of these questions may need to be re-examined, with sufficiently large samples.
• In the case of vitamin D supplementation, statistical power can be improved by appropriate trial design; as this study shows, this should include looking at baseline vitamin D status, considering the time of year the trial takes place and most importantly, how many individuals are going to be recruited to the trial.
• The implications of establishing whether those null-findings from RCTs are true or false negatives are major. If associations with vitamin D and some of those 200 diseases are causal, enormous public health impact could be achieved by advocating seasonal supplementation for vitamin D deficient individuals.

Dr. Jason Wyse, Assistant Professor in Statistics at Trinity College and a senior author on the study, said, "Vitamin D levels in our blood have an annual rhythm, keeping time with the seasons of the year. Our tools allow researchers to take account of these important characteristics when planning a trial, modeling vitamin D benefit at the individual micro-level and bringing these together to get a view of what would happen at the macro-level of a trial."

Dr. Lina Zgaga, Associate Professor in Epidemiology at Trinity College and a senior author on the study, said, "We simulated a wide range of scenarios and approximated how many participants we would need, given the range of starting points, to have sufficient statistical power to detect an effect. We found that once we take seasonal and population differences into account, we would need a larger number of participants than traditional approaches would suggest. The signal—a treatment benefit of Vitamin D—may have been lost to noise in many trials.

We now have new understanding about the factors that might have interfered with our ability to detect benefit in vitamin D trials. Going forward, we need appropriately designed and adequately powered vitamin D trials, and we hope this new tool will help researchers to ensure this. While we await these trials, I would encourage everyone to take vitamin D supplements."