This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

proofread

How accurate is ChatGPT at rating common allergy myths? Pretty accurate, says research

chatgpt
Credit: Pixabay/CC0 Public Domain

Artificial intelligence language models such as ChatGPT are increasingly being used in many different professions, and medicine is not an exception.

In a new study being presented at the American College of Allergy, Asthma and Immunology (ACAAI) Annual Scientific Meeting in Anaheim, Calif., ChatGPT was asked to rate 10 common allergy , and it judged their accuracy correctly 91% of the time.

"We searched the PubMed and Google Scholar databases to identify all articles on common allergy myths between 2000-2023," says Ricardo Estrada-Mendizabal, MD, ACAAI member and lead author of the study.

"Our goal was to assess the of ChatGPT in correctly identifying common allergy myths. We know allergy and asthma patients are constantly exposed to misinformation on the internet. We thought that the surge of public language models could represent a bigger threat to spread, especially because ChatGPT is unable to provide references."

Ten myths were selected and entered in ChatGPT 4.0 with the prompt "true or false." ChatGPT answers were graded by 24 allergists on a Likert scale of 1–4 (1=inaccurate [100% not true], 2=somewhat inaccurate [>30% not true], 3= somewhat accurate [<10% not true], 4=accurate [100% true]).

Finally, allergists were asked their likelihood of using ChatGPT for patient education with a Likert scale of 1–5 (1=not likely at all to 5=very likely). Overall, ChatGPT had an accurate or somewhat accurate response in 91% of cases and an inaccurate or somewhat inaccurate response in 9%.

Said Dr. Estrada-Mendizabal, "The response to whether a positive allergy test indicates a clinical was the most accurate response (96% accurate), whereas the response to if hypoallergenic animals were better for people with and allergies was the most inaccurate (17% inaccurate and 21% somewhat inaccurate response). Eighteen allergists (75%) were open (likely or very likely) to using ChatGPT in the future for patient education."

The authors of the paper say more studies are required to interpret the validity of ChatGPT in providing accurate medical information.

More information: Abstract D015 (A037): Dear ChatGPT, should I get a hypoallergenic dog?: Accuracy of ChatGPT for common allergy myths https://annualmeeting.acaai.org/

Top 10 myths, ChatGPT responses, and response rating by allergists.

Citation: How accurate is ChatGPT at rating common allergy myths? Pretty accurate, says research (2023, November 9) retrieved 24 February 2024 from https://medicalxpress.com/news/2023-11-accurate-chatgpt-common-allergy-myths.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Language barriers may cause some children to be underdiagnosed for allergic conditions

2 shares

Feedback to editors