AI-generated responses to patient portal messages are feasible, usable

Physicians who utilize artificial intelligence (AI)-generated draft replies to patient portal messages find the technology easy to adopt and use and beneficial to their overall well-being, according to a study published online March 20 in JAMA Network Open.

Patricia Garcia, M.D., from the Stanford University School of Medicine in California, and colleagues evaluated the implementation of a large language model to draft responses to patient messages in an electronic inbox. The analysis included 162 primary care and gastroenterology clinicians, 73 of whom completed surveys before and after implementation.

The researchers report that the mean AI-generated draft response utilization rate across clinicians was 20 percent. Compared with the prepilot period, there was no change in reply action time, write time, or read time during the pilot periods. In the prepilot and postpilot surveys, there were statistically significant reductions seen in the four-item physician task load score derivative (mean, 61.31 presurvey versus 47.26 postsurvey) and work exhaustion scores (mean, 1.95 and 1.62, respectively).

"These findings suggest that the use of large language models in clinical workflows was spontaneously adopted, usable, and associated with improvement in clinician well-being," the authors write. "Further code-to-bedside testing is needed to guide future development and organizational strategy."

One author disclosed receiving grants from Google, Omada Health, and PredictaMed.