ChatGPT found to display lower concern for child development 'warning signs' than physicians

Artificial intelligence (AI) tool ChatGPT displayed lower concern than physicians in 36% of potential developmental delays, according to a new study. The research will be presented at the Pediatric Academic Societies (PAS) 2024 Meeting, held May 2–6 in Toronto.

Researchers found ChatGPT made different conclusions about the abnormality of a potential delay than pediatricians 41% of the time.

The study investigated how ChatGPT responded to parents' concerns whether their child's development was normal or abnormal, including if the response aligned with a pediatrician's diagnosis. The research found that ChatGPT rarely categorized a case as abnormal, underscoring pediatricians' concerns that the tool is not prepared to be a reliable source of guidance for child behavioral patterns.

"Artificial intelligence tools like ChatGPT can provide accurate information for parents regarding their child's development, but still do not perform like physicians in certain tasks," said Joseph G. Barile, BA, research assistant at Cohen Children's Medical Center and presenting author.

"This study reveals how pediatricians may have more conviction than ChatGPT when it comes to denoting certain developmental delays as 'abnormal.'"

While ChatGPT showed higher concern than physicians in only 5% of cases, the research found that pediatricians identified approximately 30% more potential developmental delays than ChatGPT.

ChatGPT and pediatricians were most inconsistent with social, emotional, and behavioral concerns rather than physical, and for children older than one.

The study looked at 108 concerns in children up to five years old. The results were scored by board-certified physicians for accuracy.