New perspective argues pursuing fair AI for health care requires cross-disciplinary collaboration

Pursuing fair artificial intelligence (AI) for health care requires collaboration between experts across disciplines, says a global team of scientists led by Duke-NUS Medical School in a new perspective published in npj Digital Medicine.

While AI has demonstrated potential for health care insights, concerns around bias remain. "A fair model is expected to perform equally well across subgroups like age, gender and race. However, differences in performance may have underlying clinical reasons and may not necessarily indicate unfairness," explained first author Ms Liu Mingxuan, a Ph.D. candidate in the Quantitative Biology and Medicine (Biostatistics & Health Data Science) Program and Center for Quantitative Medicine (CQM) at Duke-NUS.

"Focusing on equity—that is, recognizing factors like race, gender, etc., and adjusting the AI algorithm or its application to make sure more vulnerable groups get the care they need—rather than complete equality, is likely a more reasonable approach for clinical AI," said Dr. Ning Yilin, Research Fellow with CQM and a co-first-author of the paper.

"Patient preferences and prognosis are also crucial considerations, as equal treatment does not always mean fair treatment. An example of this is age, which frequently factors into treatment decisions and outcomes."

The paper highlights key misalignments between AI fairness research and clinical needs. "Various metrics exist to measure model fairness, but choosing suitable ones for health care is difficult as they can conflict. Trade-offs are often inevitable," said Associate Professor Liu Nan also from Duke-NUS' CQM, senior and corresponding author of the paper.

He added, "Differences detected between groups are frequently treated as biases to be mitigated in AI research. However, in the medical context, we must discern between meaningful differences and true biases requiring correction."

The authors emphasize the need to evaluate which attributes are considered 'sensitive' for each application. They say that actively engaging clinicians is vital for developing useful and fair AI models.

"Variables like race and ethnicity need careful handling as they may represent systemic biases or biological differences," said Assoc Prof Liu. "Clinicians can provide context, determine if differences are justified, and guide models towards equitable decisions."

Overall, the authors argue that pursuing fair AI for health care requires collaboration between experts in AI, medicine, ethics and beyond.

"Achieving fairness in the use of AI in health care is an important but highly complex issue. Despite extensive developments in fair AI methodologies, it remains challenging to translate them into actual clinical practice due to the nature of health care—which involves biological, ethical and social considerations," said co-author Associate Professor Daniel Ting, Director of SingHealth's AI Office and Associate Professor from the SingHealth Duke-NUS Ophthalmology & Visual Sciences Academic Clinical Program.

"In order to advance AI practices to benefit patient care, clinicians, AI and industry experts need to work together and take active steps towards addressing fairness in AI." Ting is also Senior Consultant at the Singapore National Eye Center and Head of AI & Digital Innovation at the Singapore Eye Research Institute (SERI).

"This paper highlights the complexities of translating AI fairness techniques into ethical clinical applications. It represents our collective commitment to developing AI that augments clinicians with trustworthy insights to provide quality and equitable care enhanced by technology," said co-author Clinical Associate Professor Lionel Cheng Tim-Ee, Chief Data & Digital Officer, Clinical Director (AI) Future Health System Department, and Senior Consultant, Department of Diagnostic Radiology at Singapore General Hospital (SGH).

"Clinicians must be actively engaged in iterative communication with AI developers to ensure models align with medical ethics and context," added senior co-author Professor Marcus Ong, Director of the Health Services & Systems Research (HSSR) Program at Duke-NUS, who is also Senior Consultant at SGH's Department of Emergency Medicine.

"Good intentions alone cannot guarantee fair AI unless we have collective oversight from diverse experts, considering all social and ethical nuances. Pursuing equitable and unbiased AI to improve health care will require open, cross-disciplinary dialogues."

The perspective published in npj Digital Medicine represents an international collaboration between researchers from institutions across Singapore, Belgium, and the United States.

Authors from across the SingHealth Duke-NUS Academic Medical Center (including Duke-NUS, SingHealth, SGH, Singapore Eye Research Institute and Singapore National Eye Center) worked together with experts from the University of Antwerp in Belgium as well as Weill Cornell Medicine, Massachusetts Institute of Technology, Beth Israel Deaconess Medical Center and Harvard T.H. Chan School of Public Health in the United States.

Professor Patrick Tan, Senior Vice-Dean for Research at Duke-NUS, commented, "This global cooperation exemplifies the cross-disciplinary dialogues required to advance fair AI techniques for enhancing health care. We hope this collaborative effort spanning Singapore, Europe, and the US provides valuable perspectives to inspire further multinational partnerships towards equitable and unbiased AI."