Report shows score comparability in-person, remote proctoring

Residents taking the 2020 Internal Medicine In-Training Examination (IM-ITE) performed similarly across in-person and remote proctoring—providing evidence of score comparability, according to an American College of Physicians (ACP) research report, "A Comparison of Remote vs. In-Person Proctored In-Training Examination Administration for Internal Medicine", published in Academic Medicine.

Authors of the study included members of ACP's Medical Education division, ACP's research division and researchers from the National Board of Medical Examiners, which collaborates with ACP in providing exam administration and scoring services for the IM-ITE.

The purpose of the study was to compare the scores from in-person and remote test proctoring modalities initially made available to residents in 2020 by ACP in response to the COVID-19 pandemic and associated restrictions.

Data were analyzed from residents from all U.S.-based Accreditation Council for Graduate Medical Education–accredited IM residency programs and participating Canadian and international programs who completed the IM-ITE in 2020. The final sample contained 27,115 IM residents: 9,205 postgraduate year (PGY) 1, 9,332 PGY-2, and 8,578 PGY-3 residents.

According to the study, 42% were tested remotely and 58% in person. It revealed that the statistically significant effects were the interaction effects between mode and PGY.

Differences between in-person and remote predicted scores were slightly larger for PGY-1 than for PGY-2 and PGY-3 residents (controlling for the other predictors in the model), but the magnitude of these differences across residency training was well under a single percentage point. Because these statistically significant effects were deemed educationally nonsignificant, the study concluded that performance did not substantively differ across in-person and remote examinees.

"Residency programs choose remotely proctored testing to accommodate a myriad of scheduling challenges, and they should be reassured that they can continue to enjoy the flexibility of this self-assessment testing modality, with results comparable to in-person testing results," said Margaret Wells, MEd, vice president of assessment, accreditation, and research.

This study provides evidence of score comparability across the two testing modalities and supports continued use of remote proctoring for the IM-ITE.