Dynamic tool accurately predicts risk of COVID-19 progressing to severe disease or death

Clinicians often learn how to recognize patterns in COVID-19 cases after they treat many patients with it. Machine-learning systems promise to enhance that ability, recognizing more complex patterns in large numbers of people with COVID-19 and using that insight to predict the course of an individual patient's case. However, physicians sworn to "do no harm" may be reluctant to base treatment and care strategies for their most seriously ill patients on difficult-to-use or hard-to-interpret machine-learning algorithms.

Now, Johns Hopkins Medicine researchers have developed an advanced machine-learning system that can accurately predict how a patient's bout with COVID-19 will go, and relay its findings back to the clinician in an easily understandable form. The new prognostic tool, known as the Severe COVID-19 Adaptive Risk Predictor (SCARP), can help define the one-day and seven-day risk of a patient hospitalized with COVID-19 developing a more severe form of the disease or dying from it.

SCARP asks for a minimal amount of input to give an accurate prediction, making it fast, simple to use and reliable for basing treatment and care decisions. The new tool is described in a paper first posted online March 2 in the Annals of Internal Medicine.

"SCARP was designed to provide clinicians with a predictive tool that is interactive and adaptive, enabling real-time clinical variables to be entered at a patient's bedside," says Matthew Robinson, M.D., assistant professor of medicine at the Johns Hopkins University School of Medicine and senior author of the paper. "By yielding a personalized clinical prediction of developing severe disease or death in the next day and week, and at any point in the first two weeks of hospitalization, SCARP will enable a medical team to make more informed decisions about how best to treat each patient with COVID-19."

The brains of SCARP is a predictive algorithm called Random Forests for Survival, Longitudinal and Multivariate Data (RF-SLAM), described in a 2019 paper by its creators, Johns Hopkins Medicine researchers Shannon Wongvibulsin, an M.D./Ph.D. student; Katherine Wu, M.D.; and Scott Zeger, Ph.D.

Unlike past clinical prediction methods that base a patient's risk score on their condition at the time they enter the hospital, RF-SLAM adapts to the latest available patient information and considers the changes in those measurements over time. To make this dynamic analysis possible, RF-SLAM divides a patient's hospital stay into six-hour windows. Data collected during those time spans are then evaluated by the algorithm's "random forests" of approximately 1,000 "decision trees" that operate as an ensemble. This enables SCARP to give a more accurate prediction of an outcome than each individual decision tree could do on its own.

"The same way that individual stocks and bonds perform better as a portfolio—with the overall value staying strong as individual items balance each other's rises and falls in price—the trees as a group create a flexible and adaptable forest that protect each other from individual errors," says Robinson. "So, even if some trees predict incorrectly, many others will get it right and move the group in the correct direction."

Robinson says that most machine-learning systems used to make clinical prediction can only consider static data at a single point in time. "RF-SLAM enables us to be nimble and predict the future at any point," he explains.

To demonstrate SCARP's ability to predict severe COVID-19 cases or deaths from the disease, Robinson and his colleagues used a clinical registry with data about patients hospitalized with COVID-19 between March and December 2020, at five centers within the Johns Hopkins Health System. The patient information available included demographics, other medical conditions and behavioral risk factors, along more than 100 variables over time, such as vital signs, blood counts, metabolic profiles, respiratory rates and the amount of supplemental oxygen needed.

Among 3,163 patients admitted with moderate COVID-19 during this time, 228 (7%) became severely ill or died within 24 hours; an additional 355 (11%) became severely ill or died within the first week. Data also were collected on the numbers who developed severe COVID-19 or died on any day within the 14 days following admission.

Overall, SCARP's one-day risk predictions for progression to severe COVID-19 or death were 89% accurate, while the seven-day risk predictions for both outcomes were 83% accurate.

Robinson says that further SCARP trials are planned to validate its performance on a large scale using national patient databases. Based on the results of the first study, Johns Hopkins Medicine has already incorporated a version of SCARP into the electronic medical record system at all five of its hospitals in the Maryland and Washington, D.C., area.

"Our successful demonstration shows that SCARP has the potential to be an easy-to-use, highly accurate and clinically meaningful risk calculator for patients hospitalized with COVID-19," says Robinson. "Having a solid grasp of a patient's real-time risk of progressing to severe disease or death within the next 24 hours and next week could help health care providers make more informed choices and treatment decisions for their patients with COVID-19 as they get sicker."