Integrating multiple models provides best control of pandemic as states reopen

Leaders deciding how to contain COVID-19 most effectively should use multiple different mathematical models together in policy design instead of just one, according to new research.

A study co-authored by Ross School of Business professors Hyun-Soo Ahn and John Silberholz proposes a practical way to combine multiple COVID-19 models and generate the best guidance for individual U.S. states as they reopen, while balancing public health against the economic impact of continued restrictions.

The researchers also found the most effective strategies to contain the pandemic can vary considerably from state to state, and their paper offers specific guidance on how and when individual states should increase or relax COVID-19 restrictions.

"To tackle COVID-19, there has been an unprecedented level of research done by many scholars and researchers around the world, which has led to many parallel and collaborative research projects," the researchers wrote. "Our study shows the value of integrating these great intellectual outputs in a sensible way—rather than cherry-picking one—to design public health policy."

Ahn and Silberholz discuss their research and how it could apply to future pandemics. Their answers below were the result of a collaboration via email.

What are some of the dangers of using the wrong model to decide on strategies to contain the pandemic?

There are more than 50 COVID-19 forecast models. We find that, even for one-month forecasts, the range these models generate is incredibly wide, with gaps of tens of thousands of deaths in the forecasts.

Our study finds that if you decide on the response policy from a wrong model, both health and economic consequences can be devastating: You might either select a policy that is too lax, causing out-of-control COVID-19 deaths, or you might select a policy that is too restrictive, causing extensive economic damage.

Why isn't it possible to simply find one good model and use that?

When modeling any infectious disease, there's a huge amount of complexity and unknowns, because you need to understand both properties of the disease (such as transmissibility) as well as information about behavior (such as where people travel and the degree to which they interact with others). This is why even for established diseases, there have often been many models that estimate how they will spread and the effects of various interventions.

For a novel pandemic, the complexity is even greater. There are often many unknowns about the disease and how it spreads. Indeed, we found that sometimes the best-performing model for Michigan may not perform well for some other state. Our view is that there's never going to be a "perfect model," and that using decisions that many high-quality models find to be effective is the best way forward.

How did you decide which models to include in your framework?

Groups like the U.S. Centers for Disease Control and Prevention have actually done a great job of collecting many COVID-19 forecasting models and aggregating their predictions. We reviewed the models being used by the CDC, limiting our study to a subset that clearly describe their methodology, have good predictions in 2020, make their source code available, and cover a variety of different methodologies and data sources to make predictions.

What's the key finding of your paper, and what should happen as a result?

A key message we have for policymakers is the COVID-19 response policy recommended by a single model might actually perform quite poorly. It's essential for policymakers to use multiple models, in parallel, that make different structural assumptions, such as how they model the geographic spread of the disease. The techniques we describe in our paper can be used to find a response policy that all of these different models find to be effective.

You found that the most effective containment strategies vary considerably from state to state. Why is that?

The policy options considered in our study are based on actual policies that states used during the first surge, in April or May 2020, and the third surge this winter. Since many states have selected different COVID-19 response strategies and their populations have responded differently to those strategies, this leads to differences in how long a state might need to stay at a particular response level before moving to less stringent strategies.

What else should states keep in mind as they make decisions about reopening?

Throughout the pandemic, many states have defined tiers or response levels, without specific thresholds that trigger moving from one tier to another. Our study finds that a response policy based on caseload is simple to administer and communicate. We believe that having clear thresholds for changing the response levels is important, and we present a set of tools that can be used to select what those thresholds should be.

How much of this work is applicable to pandemics beyond this COVID-19—to potential future pandemics?

It would map well. The key finding of our research is that there is a huge value of considering multiple models at the same time when choosing a policy, and our study has shown how to do that using the existing COVID-19 forecasting models. The method applies not only to future pandemics, but also to other health policy decisions like screening for cancer, where different mathematical models may suggest different screening policies—such as the ages when you start and stop screening, the screening frequency and intensity.