Milestone rating system improved residency knowledge ratings bias

Adoption of the Milestone ratings system in 2014 was associated with improvement in internal medicine (IM) residency knowledge ratings bias in Black or Latino residents, who are underrepresented in medicine (URiM), and Asian residents, according to a study published online Dec. 25 in the Annals of Internal Medicine.

Bradley M. Gray, Ph.D., from the American Board of Internal Medicine in Philadelphia, and colleagues examined bias in IM residency knowledge ratings against Black or Latino residents (URiM) and Asian residents before and after the 2014 adoption of the Milestone ratings system. Participants included 59,835 IM residents completing residencies during 2008 to 2013 (pre-Milestone) and 2015 to 2020 (post-Milestone).

The researchers found that ratings biases against minoritized groups were large during the pre-Milestone period (−0.40 standard deviations [SDs] for URiM residents, −0.24 SDs for U.S.-born Asian residents, and −0.36 SDs for non-U.S.-born Asian residents). After adoption of Milestone ratings, these estimates decreased to less than −0.15 SDs for all groups except U.S-born Black residents, among whom substantial bias persisted, although at a lower level (−0.26 SDs). Coincident with adoption of Milestone ratings, substantial deviations for pre-Milestone linear bias trends were seen.

"The current study is an important step in unraveling and combating the persistent nature of bias in the evaluation of internal medicine residents of color," write the authors of an accompanying editorial. "Bias is difficult to confront and must be continually and proactively addressed to counter its unconscious and negative effects."