This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


peer-reviewed publication

trusted source


AI uncovers bias in dermatology training tools

AI uncovers bias in dermatology training tools
STAR-ED framework overview. A STAR-ED framework takes academic materials (e.g., in .pdf format) as input followed by extraction of skin images in the given academic material. Specifically, image pixels that are identified as skin are then utilized to estimate the skin tone category. B Corpus Conversion Service (CCS) (7) is an existing document ingestion tool employed to parse different document entities, such as all images and tables in the data. We extracted all images using the JavaScript Object Notation (JSON) (8); output from the ingestion step contains the coordinates and page number of identified images. C Since our focus is on images related to skin diseases, non-skin images (e.g., graphical illustrations and pathology images) are discarded using an XGBoost (9) classifier. D For each image depicting skin, we masked out non-skin related pixel regions in the foreground and background (e.g., pixels of clothes, laboratory equipment). We employ color-based skin pixel segmentation that extracts pixels that meet a predefined threshold. E Finally, the segmented skin regions are fed into a pre-trained deep learning framework, i.e., ResNet fine-tuned as described in Materials and Methods, to estimate the skin tone category as either light (FST I–IV) or dark (FST V–VI). Images adapted from Wikimedia commons. Credit: npj Digital Medicine (2023). DOI: 10.1038/s41746-023-00881-0

Skin diseases do not look the same across the skin-tone spectrum, and medical textbooks and presentations used to train dermatologists often lack example images of darker skin tones. During the recent pandemic, for instance, studies showed annotated photos of COVID-19's dermatologic symptoms lacked adequate representation of darker tones.

Other research has gone so far as to suggest that such disparities may contribute to why skin cancer diagnosis—particularly melanoma, which is among the most metastatic of all cancers—is often "significantly delayed" for people of color and why death rates are higher in those same populations.

"Unfairness in the equates to unfairness in society," says Roxana Daneshjou, a dermatologist and biomedical data scientist at Stanford University. Daneshjou recently co-authored a study published in npj Digital Medicine introducing the Skin Tone Analysis for Representation in EDucational materials (STAR-ED) framework that uses machine learning to assess bias in skin tones in frequently used medical .

The team trained STAR-ED on thousands of images in medical textbooks, lecture notes, presentation slides, and journal articles to determine just how underrepresented black and brown skin is in the materials. They found that just one in ten images throughout these materials is in the black-brown range on the Fitzpatrick Scale used to evaluate skin tone.

"We're turning AI-bias storyline on its head a little bit," Daneshjou says. "There's lots of news out there of bias in AI models, but in this case we've trained an AI model that detects human bias."

STAR-ED is not the first study to highlight this disparity, Daneshjou acknowledges, but it is the first to use automatic skin tone grading by computer algorithms and is therefore far more scalable than the human-labeled data of the past. Human annotators, Daneshjou says, are prone to fatigue and variability in judging skin tone. While STAR-ED is not perfect, the results closely mirror the findings of human annotators, who spent many hours labeling the data.

In highlighting the disparity in existing training materials, Daneshjou's hope is that authors and editors will use STAR-ED to evaluate their soon-to-be-published textbooks, journals, and slides for potential biases and to remedy any biases prior to publication.

"Our suggestion is physicians are not being trained adequately and that this shortfall may contribute to why people of color with psoriasis, eczema, melanoma, and other don't get diagnosed and treated sooner and better," Daneshjou adds. "The bottom line is that we need to get more images of black and brown skin diseases into the training literature."

STAR-ED works on a comprehensive range of file formats—pdf, png, jpeg, pptx, and docx among others—and can be applied to materials beyond textbooks, such as research papers, image study sets, and lecture slides, to bring greater representation into classrooms, reading materials, and seminars.

"We envision STAR-ED helping medical educators, publishers, clinicians, and even students quickly and more easily assess their for bias," she says. Ultimately, she hopes that would translate to earlier, more frequent, more accurate dermatological diagnoses for some already underserved populations.

Future plans are to get STAR-ED into the hands of publishers, editors, and and, potentially, to extend STAR-ED to other educational domains—such as history textbooks—to identify similar gaps in diverse representation.

In this iteration, STAR-ED did not consider texts, tables, and other written content, which its creators hope to integrate later to further reduce bias. One limitation of the methodology the authors noted is that it did not fully exclude diseased or lesional skin, which can obscure the appearance —and thus the tone grading—of the subject's healthy skin. Last, Daneshjou also noted that the Fitzgerald Scale itself is known to be biased, a concern the team hopes to rectify in future versions with the use of alternative tones scales.

STAR-ED is , and both code and dataset are available for download by interested parties.

More information: Girmaw Abebe Tadesse et al, Skin Tone Analysis for Representation in Educational Materials (STAR-ED) using machine learning, npj Digital Medicine (2023). DOI: 10.1038/s41746-023-00881-0

Journal information: npj Digital Medicine

Citation: AI uncovers bias in dermatology training tools (2023, September 7) retrieved 6 December 2023 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Training physicians and algorithms in dermatology diversity


Feedback to editors