Knowing the model you can trust: The key to better decision-making

As much of Europe is engulfed by a second wave of COVID-19, and track and trace struggles to meet demand, modeling support tools are being increasingly used by policymakers to make key decisions. Most notably, models have been used to predict the COVID-19 R0 rate—the average rate of secondary infections from a single infection, which has formed the basis for many lockdown decisions across the UK.

Models can represent the most effective tool for identifying interventions that can balance the risks of widespread infection and help assess socio-economic disruption until an effective treatment is established. However, not all models are equal, and differences in model predictions during the COVID-19 pandemic have caused confusion and suspicion.

A recent paper "Three questions to ask before using model outputs for decision support" published in Nature Communications aims to help decision makers choose the best available model for the problem at hand. The paper proposes three screening questions that can help critically evaluate models with respect to their purpose, organization, and evidence, and enable more secure use of models for key decisions by policy makers.

One of the authors of the paper, Dr. Alice Johnston, Lecturer in Environmental Data Science at Cranfield University, said, "From COVID-19 to the stock market, models are increasingly used by policymakers to support their decisions.

"However, different models are based on different assumptions and so can produce conflicting results, even when they represent the same system. Models used early on in the COVID-19 pandemic were a prime example of this, which led to confusion over which models to trust to support the decision-making process. This really highlights the need for clear communication of a model's context, so that policymakers have confidence in which models to trust.

"We propose that before engaging with a model, policymakers ask themselves three screening questions focusing on the model's purpose, organization and evidence base, with the aim of bringing greater clarity to the decision-making process."