Adding stratification factors can reduce a small trial's statistical power

Credit: Pixabay/CC0 Public Domain

To ensure all arms in a randomized clinical trial have roughly equal percentages of participants from key subgroups, statisticians often stratify patient assignment to each arm and account for these stratification factors in the final analysis.

A computational study led by researchers with the SWOG Cancer Research Network has found that for modestly sized 2 trials, stratified analysis with more than one or two stratification factors can significantly reduce the power of a trial to detect a positive result.

The work will be presented at the American Society of Hematology's 2022 Annual Meeting and Exposition, on December 12 in New Orleans (ASH 2022 Abstract #4027).

To test the effect of stratification on statistical power, the researchers simulated trial data using design assumptions from the protocol of a randomized phase 2 acute myeloid leukemia trial that was being finalized. They ran multiple simulations based on the design, using increasing numbers of stratification factors in randomization and in analysis. They then estimated the statistical power for each simulation while keeping the allowable type-1 error roughly constant. Also known as a , a type-1 error concludes an effect when no effect actually exists.

The researchers found that for the small phase 2 trial design they worked from, which had a total sample size of 84 participants (42 per arm), a stratified analysis that included one or two stratification factors did not significantly decrease the statistical power from that of an unstratified design. When four or six stratification factors were used, however, statistical power decreased from around 88% in the unstratified setting to 75% or 55%, respectively. Generally, clinical trial designs aim to have at least 80% power to detect a difference between treatment arms.

The analysis was led by Anna Moseley, a SWOG biostatistician based at the Fred Hutchinson Cancer Center. Moseley noted there is currently a lack of concrete data in the literature on the impact of additional stratification factors on statistical power.

"The impetus for the experiment was that there weren't any data to look at when trying to advise study teams on how many factors to use in stratification, at least not in the setting of a phase 2 trial—so with a —with potentially many stratification factors."

The new findings, she said, "can help statisticians give more informed advice when designing trials, and a justification for limiting our number of factors to those that are going to really make a difference in how likely each patient is to meet the primary endpoint."

"There might be a tendency to assume that we want the arms to be as equal as possible," she added, "which makes sense, because you want all of these to be distributed equally across the to make the trial most likely to be unbiased, but there is a drawback to that: it will hit your power to show a positive result."

Provided by SWOG Cancer Research Network
Citation: Adding stratification factors can reduce a small trial's statistical power (2022, November 3) retrieved 23 June 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Multi-modal AI models for intermediate- to high-risk localized prostate cancer


Feedback to editors