New tool can assess the climate of equity and inclusion in medical schools

Yale researchers have developed a new tool that can assess the state of equity and inclusion in medical school learning environments and provide feedback on how schools can make improvements. Using the tool could yield the timely and recurrent information needed to develop effective, evidence-based interventions, said the researchers.

The findings were published Feb. 21 in JAMA Network Open.

Creating inclusive and equitable learning environments has been a longstanding priority for medical organizations like the Association of American Medical Colleges (AAMC) and the National Academy of Sciences. A lack of regular, timely data on such environments, however, has limited how well organizations and medical schools can target their efforts. It was this challenge that led the researchers to develop the new tool.

"We wanted to make something generalizable and scalable, without the need for additional investment," said lead author Dr. Dowin Boatright, who is currently vice chair of research in the Department of Emergency Medicine at New York University Grossman School of Medicine but initiated this research while an assistant professor at Yale School of Medicine (YSM) where he served as the YSM Teaching and Learning Center faculty associate for diversity, equity, and inclusion educator development.

The AAMC conducts several surveys of U.S. medical students throughout their education. The Yale researchers believed that two of these surveys specifically might already contain the information needed to evaluate whether the nation's medical schools achieve a meaningful climate of equity and inclusion: the Year 2 Questionnaire, given to students in the middle of their second year of medical school, and the Graduation Questionnaire, distributed to students in their final year. The former has a historical response rate of 59% while the latter has a response rate of approximately 80%.

To assess the value of these surveys, the research team first convened a diverse panel of experts that included medical school faculty and students, as well as individuals with expertise in diversity, equity, and inclusion, medical education, and organizational psychology. That panel identified 146 survey questions that they believed were relevant to equity and inclusion.

The team then applied rigorous evaluations of the survey items. Using survey data from 2015 to 2019, they mapped those survey questions to specific practices or attributes that are related to equity and inclusion—such as discrimination, faculty support, and fostering a collaborative and safe environment.

"We were also able to confirm that similar questions were being answered in a consistent manner," said senior author Marney White, a professor of public health (social and behavioral sciences) at the Yale School of Public Health who also has affiliations with YSM and the Yale Jackson School of Global Affairs. "And we determined that the tool performed as predicted, matching theory based on previous research evidence."

According to the researchers, the tool is able to take the answers to the relevant survey questions and yield a score for how students perceive equity and inclusion in their schools.

Based on the test data, the researchers found that scores were higher from students who reported no marginalized identities. Students from backgrounds historically marginalized in medicine—for example, female students, non-white students, and lesbian, gay, or bisexual students—scored their schools less favorably than their peers.

"And students with multiple marginalized identities scored their learning environments increasingly lower in a stepwise fashion," said Mytien Nguyen, a co-author of the study and an M.D.-Ph.D. student at YSM. "These findings were consistent with those of previous studies, leaving us confident that the tool was measuring what it was designed to measure."

There are several benefits to using this tool, said the researchers. It offers a look at the state of equity and inclusion at the national level and within individual schools, and it can provide insight into how these factors are changing over time. Accrediting bodies—such as the Liaison Committee on Medical Education, which accredits schools in the U.S. and Canada—could use it to hold schools accountable for reaching certain standards or demonstrating improvement. And schools themselves could compare their scores to national averages to see where they stand.

"The tool's specific data would also be a great way for schools to dissect what they can target in terms of directing resources toward more effective initiatives," said Nguyen.

The researchers are now in talks with the AAMC to see whether scores and their related data could be included in the annual reports the organization sends to medical schools.

Going forward, the researchers want to explore how these scores relate to schools' rates of student burnout and attrition. They also want to see if the tool can help pinpoint effective efforts to improve equity and inclusion.

"We'd like to use these scores to identify schools that have been successful in creating equitable and inclusive environments," said Boatright. "We could then look at what they're doing in terms of intervention to promote equity and inclusion and amplify what's working."