Can we trust AI not to further embed racial health inequalities?

Artificial Intelligence
Credit: CC0 Public Domain

Heralded as an easy fix for health services under pressure, data technology is marching ahead unchecked. But is there a risk it could compound inequalities, asks journalist Poppy Noor in The BMJ today?

Artificial intelligence (AI) systems simulate by learning, reasoning, and self correction. The technology is promising: it could reduce patient waiting times, improve diagnosis, and relieve an overburdened NHS—benefits which apply across the field, regardless of race.

But there is growing concern that if the data being used to train AI are not ethnically representative, the technology may not work as well for some groups.

For example, images used to train algorithms to detect melanoma are predominantly, if not all, images of caucasian skin. In reality that could be the difference between AI that works as well as the best-trained clinicians for a white person, but doesn't recognise cancerous moles on a .

What's more, if companies are not obliged to show this data, people may continue to think that AI is relevant to everyone when it is not.

Professor Adewole Adamson at the University of Texas says it is fundamental to AI's success that products work on all regardless of , and that where they do not, a warning is provided.

Eleonora Harwich from the think tank Reform warns that bias and discrimination are difficult things to regulate, and suggests that market incentives could provide more representative data.

But others argue that market incentives can't overcome the fact that there are hundreds of years of medical research missing for people from ethnic minority backgrounds.

According to Adamson, the cost of not fixing this could be huge. AI could misdiagnose people, under diagnose people, and potentially increase health-care disparities, he warns.

For that reason, he says we need to keep in mind that AI might be an amazing tool, but it is no replacement for the complex human thinking required of ethical decision making.

Explore further

Artificial intelligence helps with earlier detection of skin cancer

More information: Can we trust AI not to further embed bias and prejudice? BMJ (2020). DOI: 10.1136/bmj.m363 ,
Journal information: British Medical Journal (BMJ)

Citation: Can we trust AI not to further embed racial health inequalities? (2020, February 12) retrieved 24 February 2020 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments