AI medical coding research adds to big picture

Much like the game of connect the dots, Megan McDougal's academic and professional career share points that have come together to form one big picture.

An associate professor in health informatics and information management in the West Virginia University School of Medicine, McDougal has recently added research to her itinerary. She never thought either would be part of her future.

"It's quite interesting how things line up and you don't realize how each experience you have in your life really helps prepare for the next big thing," she said.

She once aspired to become an X-ray technician. While she liked the medical side of her studies, patient care just didn't fit with what she wanted to do. She gave psychology a try, then moved on to elementary education. Both had that something she was looking for, but neither fit the bill. Then along came health information technology, a job in medical records at J.W. Ruby Memorial Hospital and a degree in allied health administration. It seemed she'd found her place. That was until a supervisor recognized her underlying potential and added student preceptor to her job duties.

"One aspect was working with students who were just like me previously," she explained. "They needed internships and practical experience to graduate and to see if those careers were right for them. That sparked my love of educating others again."

Still, she felt her own education was incomplete. She went on to earn a master's degree in health information management with no concrete plans for what came next.

"You never know what will happen in life. It's so weird how everything falls into place."

That's exactly what happened.

When a position for an online adjunct instructor in health information technology came open at another university, McDougal decided to make a move. It wasn't long after that WVU began to build a new bachelor of science in health informatics and information management program, the only one of its kind in West Virginia. The new HIIM program director, Sally Lucci, heard of McDougal's background mentoring students at the hospital and her recent adjunct experience, and connected with her to gauge her interest in working with the program. Her path led her back to WVU, first as an adjunct and as full-time faculty in 2017.

"My favorite thing about the field is it's never the same day-to-day and constant new things emerge to challenge you. It's good to remain curious and continue to keep updated on changes and how you can make an impact."

Next came the research project, something else McDougal said she "stumbled upon" when a group of WVU physicians began wondering what future role artificial intelligence will play in medical coding.

As a result, McDougal, another HIIM faculty member, Ashley Simmons, and WVU Medicine colleagues Drs. Brian Dilchner, Jami Pincavitch, Ankit Sakhuja and Lukas Meadows set out to determine how effective AI can be for the medical coding workflow.

"Medical coding is an important aspect of the U.S. health care system, and currently, computerized assistive coding technology—CAC—is used to assist medical coding professionals with their workflow," she said. "Some studies have looked into powering CAC with use of artificial intelligence, which potentially opens up new opportunities for the coding workflow."

Using 50 fictional patient records, the team divided duties. A certified coder on the team coded the clinical notes and kept track of the time she put in and resulting diagnoses. The others fed the same clinical notes into various open source AI large language models following specific prompts, and also kept track of the time spent and resulting diagnoses.

Although all the LLM platforms extracted codes quicker, there seemed to be minimal agreement between the human coder and the system.

"While the study is still underway, what we are finding from the data we have collected so far is that the LLMs have limited performance in their ability to accurately abstract ICD-10 diagnostic codes from the clinical notes provided," McDougal said. "The actual performance of the LLMs doing the specialized tasks asked have been very poor."

McDougal emphasized the purpose of the study wasn't to see if AI could replace humans, but that did spark the idea.

"You're still going to need a human to make sure you get the right diagnostic code and make sure you are compliant with regulations."

The team is considering experimenting with other LLMs to make comparisons in efficiency.