Researchers explore generative AI benefits and shortfalls in medical education

AI is a rapidly growing technology that is now influencing the world of medical education. For educators, this emerging technology is showing its potential in bringing real value to classrooms. That said, the use of AI has led to the question about whether these intelligent systems can play a significant role in medical institutions.

Academia, including medical education, has become divided in this new arena due to ethical concerns arising over misuse, cheating, and privacy when using AI.

To better understand the current evidence and best practices behind how to implement generative AI in teaching and medical education, one study—led by third-year UNC School of Medicine student Joshua Hale—is aiming to provide clarity for educators and students about best practices for implementation of these tools, as well as identify gaps in the literature to provide future investigation into why AI should be included in the classroom and curriculum.

"We found some interesting ideas about how generative AI can augment professors and students," said Hale. "Specifically, many articles suggested that generative AI's natural language processing ability will make them good candidates for future implementation as virtual patients, enabling students to practice independently and receive personalized feedback."

The study, published in the Journal of Medical Education and Curricular Development, reviewed 521 articles that were released before June 30, 2023, on how generative AI is and could be used and implemented in undergraduate medical education (UME).

A total of 41 articles were extracted and showed how the majority of them were opinion pieces, case reports, letters to the editor, editorials, and commentaries about the use of generative AI while seven articles used qualitative and/or quantitative methods.

The literature was also divided into five categories acknowledging the role AI can implement in UME as well as its challenges: nonclinical learning assistant, content developer, virtual patient interaction, clinical decision-making tutor, and medical writing. Results also showed that generative AI tools' greatest potential is for use as a virtual patient and clinical decision-making tutor.

Identifying whether these areas will yield the most significant benefits is a question researchers aimed to address as they searched for the integration of AI tools into medical education. The study further explored how these five categories were proposed to enhance undergraduate medical education based on current literature.

"Nonclinical learning assistant" was the first in which articles suggested the use of AI as a reference for students to search for and find medical information. Virtual assistant ChatGPT 3.5 showed how it could answer clinical free-response questions above a passing threshold 43% of the time. Articles suggested that AI could be used as a learning assistant or personalized tutor for nonclinical content.

"Content developer" appeared in articles discussing the ability of ChatGPT to write multiple-choice questions and provide explanations. Researchers found that AI could write multiple-choice questions at a similar level of proficiency to that of a human in significantly less time.

For "curriculum," it was found that AI could develop cases and simulations and suggest the implementation of guidelines and detectors to prevent misuse by students. Four articles provided insight into how AI models could create a type of virtual patient interaction and tutoring. It will be able to display how ChatGPT could act as a virtual patient and provide feedback to students on their clinical decision-making, as well as a clinical reasoning tutor.

"Generative AI could simulate virtual patients that respond to student queries with specific diagnoses or learning objectives in mind," said Hale. "While current virtual patients exist, generative AI has the capability to interpret text and produce unique responses based on student interactions.

"For example, students could conduct interviews with AI virtual patients on their computers, receive follow-up questions, and obtain automated feedback on their performance. This technology expands the potential educational tools available to educators," he said.

Lastly, medical writing was frequently cited as an area of use and an area of concern. AI displayed creative writing assistance, research manuscripts, and grant writing as well as support in writing medical notes. To help with the concern of misusing the tool in this category, researchers recommended the implementation of surveillance systems for generative AI use and tailoring assignments towards its limitations, so students are forced to address its shortfalls.

"Generative AI-generated content can evade automated detection, posing challenges to academic integrity," said Hale. "Tools like ChatGPT are valuable for generating search strategies and brainstorming. Schools are increasingly acknowledging these capabilities and educating students on appropriate usage and citation of generative AI."

The study suggests that ideas put forth in these articles can provide a starting point for educators looking to innovate in the classroom and prepare students for a medical field that increasingly embraces AI tools. Advantages of these tools can lead to personalized insight into cognitive or practical mistakes that the student can possibly make and provide immediate feedback.

By incorporating AI and its new capabilities, educators can spend more time focusing on the next generation of medical professionals and innovation. However, Hale emphasized the need for more research.

"Further research is essential to demonstrate the specific benefits of integrating AI tools into classroom learning," said Hale. "Nevertheless, these tools are already proving to be valuable adjuncts for interested users.

"My classmates frequently leverage ChatGPT as a study partner to develop cognitive aids, study templates, and more. In a research setting, it can serve as a resource for generating ideas for literature searches, learning coding or data analysis, and much more."