Researchers find substantial uncertainties in COVID-19 pandemic simulations

Computer modeling to forecast Covid-19 mortality contains significant uncertainty in its predictions, according to an international study led by researchers at UCL and CWI in the Netherlands. Their article 'The Impact of Uncertainty on Predictions of the CovidSim Epidemiological Code', which was earlier announced in a Nature news item, was published in Nature Computational Science on 22 February 2021.

While in a physical experiment it is common practice to provide error bars along with  the measured values, the predictions from a computer model often lack a measure of uncertainty. This is despite the fact that such models undeniably are uncertain, and are used in high-level decision making. The international research team argues that computational predictions without error bars can paint a very incomplete picture, which they demonstrated in a recent study with a  computer model used for evaluating COVID-19 intervention scenarios.

This study was done within the VECMA project, a European Union Horizon 2020 research and innovation program. At CWI, researchers Wouter Edeling an Daan Crommelin from the Scientific Computing group were involved. Edeling, first author of the article, and a specialist in quantifying modeling uncertainties, made part of the software for the VECMA toolkit. This was used to couple a uncertainty quantification (UQ) technique to the well-known epidemiological CovidSim model from Neil Ferguson of Imperial College in the UK and data from the UK from 2020.

'Curse of dimensionality'

Edeling explained: "For models with a high number of parameters like CovidSim, it is very difficult to study which effect uncertainties in the input parameters have on uncertainties in the output. Having many parameters means that the computational costs will be inordinately high – often referred to as the 'curse of dimensionality'. We investigate how to do the computations as efficiently as possible, by finding out which parameters matter most for the output uncertainties. By focusing on these parameters it becomes possible to make good probabilistic predictions, which can be used by governments for their decisions."

The new methods are very effective. In testing the robustness of CovidSim the research team found that, although the code contained 940 parameters, 60 were important and, of those, only 19 dominated the variance in the output predictions. Half of the overall variation in their results was down to just three of the 940 input parameters: the disease's latency period, the delay in an infected person self-isolating, and the effectiveness of social distancing. Whereas the latency period is a biological parameter, the other two (and quite a few others which were influential) are related to the intervention scenarios and human behavior. While they represent a difficult modeling task, unlike the biological aspects, these parameters (and the phenomena which they model), can be influenced by governmental policy.