COVID-19 studies based on potentially 'weak and over-optimistic' data, warn researchers

Doctors may be making decisions about the care of Covid-19 patients based on weak and over-optimistic evidence, Keele researchers have found.

Researchers from Keele University and institutions from across Europe, including the University of Oxford, reviewed 27 research studies that propose models for the diagnosis and prognosis of patients with Covid-19.

The researchers, whose paper has been published today in the British Medical Journal (BMJ) (link here), found that the data and methods used in these studies were potentially at high risk of bias, while some of the studies included recommendations that were questionable if put into practice.

The researchers warn that the potentially flawed models may result in doctors making inappropriate decisions about whether patients have the virus, need a ventilator or should remain in hospital.

Since the outbreak in December, health care systems across the world have been under severe strain.

More than one million people have been diagnosed with the virus worldwide and the death toll has surpassed 51,000.

Doctors face significant pressure to detect and diagnose patients who are infected with the virus and to give a prognosis for each confirmed case.

The researchers reviewed 27 studies—25 used data from China, one used data from Italy, while another used international data. The data was collected between 8 December 2019 and 15 March 2020.

They found that all the studies had a high risk of bias. Some of the studies had a non-representative selection of patients, while others excluded patients who were still ill at the end of the studies. Others had poor statistical analysis.

The researchers acknowledge that clinical data from Covid-19 patients is scarce and that the studies were done under severe time constraints so that they could help medical decision-making as quickly as possible.

However, given the identified flaws, the researchers said it was a concern that some of the proposed models were already being used to support medical decisions.

They are calling on researchers to immediately share anonymised high-quality patient data through the World Health Organisation to support collaborative efforts to build and validate more rigorous prediction models.

Richard Riley, professor of biostatistics at Keele University, said:"Doctors should be making decisions based on robust evidence. Unfortunately, current studies fall short in providing robust models for diagnosis or prognosis of Covid-19.

"Researchers around the world should unite in sharing their data immediately to improve the quality of future studies in this area. Health professionals and methodologists must work together, to pool high-quality and representative Covid-19 data that is then analysed using appropriate statistical methods."

Dr. Kym Snell, Lecturer in Biostatistics at Keele University, said: "Many researchers are focusing their efforts on trying to assist healthcare professionals in this very difficult time—this should be applauded. However, we still need to ensure that models for the diagnosis and prognosis of patients with Covid-19 are properly tested.

"We will continue to critically appraise new models so that healthcare professionals have an up-to-date summary of the evidence."

Laure Wynants, Assistant Professor of Epidemiology at Maastricht University, which led the research, said:"This research has shown how important it is for researchers working in this field to share good quality evidence. This helps quickly identify models that work well and can potentially support decision-making in clinical practice."