Study finds use of mobile app improved physician lab test ordering and diagnosis

Mobile health applications (apps) for improving diagnostic decision-making often lack clinical evaluation, but one app that has undergone testing by researchers is the Centers for Disease Control and Prevention PTT Advisor. In a recently published study in the Journal of the American Medical Informatics Association, researchers from Baylor College of Medicine and the CDC evaluated whether the app improves diagnostic and test ordering decisions of physicians for certain coagulation and bleeding disorders and if they find the app to be useful for learning.

"There are increasingly more physician-focused health apps available, and while physicians are becoming more comfortable using these apps, they don't necessarily know which ones can help with diagnosis because they haven't been evaluated," said lead author Dr. Ashley Meyer, assistant professor of medicine at Baylor and researcher in the Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety (IQuESt). "Test ordering and diagnosis for certain hematologic disorders is particularly difficult for physicians. The team at the CDC's Clinical Laboratory Integration into Healthcare Collaborative developed PTT Advisor to address these issues and approached our research group to conduct an evaluation."

Meyer and her team looked to the literature for established methodologies to evaluate these types of apps. Finding none, they took this opportunity to establish a foundational methodology to evaluate the usefulness of these apps.

"Our first step was figuring out how to rigorously evaluate an app that aimed to improve physician decision-making, such as PTT Advisor, and then to use our methodology to evaluate the app," Meyer said.

To evaluate the app, the evaluation team used eight vignettes based on real and challenging clinical cases that would prompt physicians to make clinical laboratory test ordering and diagnosis decisions. The scenarios tested physician decision-making while they sought help from one of two resources: PTT Advisor or usual clinical decision support (including any internet, text or personnel resources the physicians wanted). Forty-six physicians from seven different healthcare institutions participated in the study.

Each physician solved four of the eight vignettes using the app and the other four using usual clinical decision support.

"We found that using the app increased diagnostic accuracy, which is a good step forward given that we previously found 5 percent of U.S. adults are misdiagnosed every year," Meyer said.

Results indicated that when physicians used the app, the accuracy of their diagnosis and testing decisions was 13 percent higher than when they used usual clinical decision support. Additionally, using the app helped them come to a diagnosis about 51 seconds faster than when they used usual clinical decision support – a reduction in time of 22 percent.

On a subsequent survey, physicians reported positive perceptions of the app's potential for improved clinical decision-making and benefit to patients.

"Moving forward, we hope that such evaluation methods will be adopted by others to determine usefulness of apps to improve diagnosis. Apps that add value to physician decision-making will be much easier to integrate with physician workflow and help impact clinical practice and reduce misdiagnosis," said senior author Dr. Hardeep Singh, chief of the Health Policy, Quality and Informatics Program at the Houston VA's IQuESt and professor of medicine at Baylor.