Researchers reveal that low-quality studies of autism early interventions dominate the field

The Centers for Disease Control and Prevention report that autism is becoming more common in young children. In an effort to improve the challenges young autistic children face as part of their early development, researchers have focused on developing and evaluating nonpharmaceutical interventions that can be provided in early childhood.

Micheal Sandbank, Ph.D., an assistant professor in the Department of Health Sciences at the UNC School of Medicine, is an expert on the research supporting these early interventions, which informs clinical practice across the United States. A new comprehensive meta-analysis, led by Sandbank, shows that many low-quality studies dominate the field, and ultimately dictate intervention recommendations and patient outcomes. The results were published in the British Medical Journal.

"What our study shows is really that the evidence is poor for a lot of interventions and that we have not done a very good job of monitoring for potential adverse effects or harms," said Sandbank. "We, as pediatricians or therapists, need to be clearer with families about what that landscape of evidence looks like and then also be clear that these interventions might have negative impacts that we just don't know about."

A fractured landscape

Early intensive behavioral intervention is the "gold standard" for early autism care in the United States. The adult-led, highly structured intervention provides young autistic children with one-on-one support for 20-40 hours per week.

Some clinicians claim it is too intense, and prefer to promote more developmentally-informed interventions, which are often provided at lower intensities and can be easily embedded in the daily family routines. However, not everyone agrees.

"There's really a lot of different interventions that are available to this population, but the researchers studying them don't agree with one another," said Sandbank. "As a result, there's a fractured landscape both of what is offered and how researchers feel about what is the best support for the children and their caregivers."

A change-up in study approaches

Clinicians choose interventions for their patients based on outcomes from the most up-to-date research studies.

Several different types of studies can be used to evaluate the efficacy of a particular intervention. Often, these studies compare a group of participants who have received the intervention to participants who did not receive the intervention. Sometimes, participants are assigned to each group randomly through a randomized controlled trial. Other times, the participants are assigned to each group based on non-random factors through a quasi-experimental study.

In autism research, quasi-experimental studies have given way to randomized-controlled trials. Randomized-controlled trials can offer more accurate findings because random assignment reduces the influence of other factors that may skew the outcomes. However, a plethora of quasi-experimental studies still exist in the field and inform clinical practice.

The research team's meta-analysis, which searched all recent scientific literature and compared study methods and results for different types of interventions, found that the number of studies, including randomized controlled trials, had doubled in just four years.

However, even though randomized controlled trials are increasing, other problematic study methods are prevalent. Sandbank and her colleagues found that even when they considered evidence from randomized controlled trials alone, many of the studies had not adequately controlled for two important risks of bias: placebo-by-proxy and detection bias.

Eliminating biases and improving accuracy

Researchers strive to carefully design studies so that the answers they provide are as accurate as possible. There are cases, though, where aspects of study design threaten the accuracy of results and can make interventions appear more effective than they are.

When scientists measure outcomes through caregiver report, placebo-by-proxy bias may occur. This happens when caregivers subconsciously report better outcomes because they know their child is receiving an intervention that they believe to be effective. Even when scientists directly measure outcomes themselves, they may subconsciously overestimate the effect of the intervention if they are aware of which group each participant is assigned to, a type of bias known as detection bias.

Sandbank and her colleagues found that when they only considered the best evidence that adequately guarded against all of these risks, few of the most popular interventions had support from the best evidence.

"However, this does not mean that these interventions are not effective," said Sandbank. "What we need are more rigorous studies of the types of interventions that are offered to children and families, so that we can really understand both their positive and negative effects."

In the meantime, clinicians struggling to choose the best support for their patients can simply be more transparent about current evidence and remove absolute phrasing when presenting interventions to autistic children and their caregivers. Sandbank finds this to be especially important for supporting caregiver decision-making and their own mental health.

"Just as children can thrive with certain interventions, they can have adverse reactions, too," said Sandbank. "There are families that have very negative experiences, but they are not quick to let go of an intervention approach that is not working for them because they have been told that it is supported by incontrovertible evidence. That can cause a lot of self-doubt and self-blame."