Examiners tend to grade relative to work already seen, research finds

December 5, 2012, University of British Columbia

(Medical Xpress)—How well you fare on a subjective evaluation – whether it's of you treating a patient, auditioning for a play or even interviewing for a job – may depend largely on the person who was examined just before you. And the worse the person before you did, the better for you.

That sobering conclusion by University of British Columbia professor Kevin Eva resulted from a that examined whether physician-educators' grading of younger doctors could be influenced by the quality of those they had previously evaluated.

The study was one of two articles by Eva, to be published Dec. 5 in (The ).

Conducted with researchers at the University of Manchester, it asked physician educators in England and Wales to grade videos of young doctors in their first year of post-graduate training. The doctors portrayed in the videos followed scripts to represent three levels of performances – good, poor, and borderline – as they examined patients.

Some physician-educators were primed by first viewing the good performances while others viewed the poor performances. Then both groups were compared on their grading of the borderline performances.

Educators who had been primed by poor performances consistently gave better grades than those who had just watched the good performances – depending on the patient case, the grades were 30 to 100 per cent better.

"This experiment shows that judging someone's performance – whether it's clinical skills, essay-writing, or figure skating – is likely to be relative," says Eva, Director of and Scholarship in the Department of Medicine and a Senior Scientist at UBC's Centre for Scholarship.

"While such assessments are unavoidable in determining whether a student has mastered the required competencies in a field like medicine, we need to take steps to minimize contrast bias," Eva says. "It is important to ensure that there are a sufficient number of evaluations – the more data points, the more reliable the aggregate score will be."

In the second study published in the same edition of JAMA, Eva and collaborators examined the validity of multiple mini-interviews (MMIs) – a dozen, brief encounters in which applicants are asked to solve a problem through discussion with an interviewer, or through interaction with an actor or another applicant. MMIs have come to replace longer, more traditional interviews at most Canadian medical schools (including UBC's Faculty of Medicine).

The researchers found that students accepted to McMaster University, the first Canadian medical school to adopt MMIs, performed better years later on licensing examinations than students whom McMaster rejected but who were accepted at other, non-MMI schools.

Eva is an expert in the education of health professionals but he has also demonstrated how seemingly unrelated factors can affect judgment in a very different domain – men's physical attractiveness.

In 1999, he and a fellow graduate student at McMaster University asked women to view photographs and brief descriptions of men, and to rate their attractiveness on a 7-point scale. Men labeled as married had a mean attractiveness score that was 23 per cent higher than when the same men were labeled as single. As he and his collaborator wrote in CMAJ (Canadian Medical Association Journal): "Our findings do support the notion that being 'taken' influences perceptions of 'goodness."

Explore further: Bias may exist in rating of medical trainees

Related Stories

Bias may exist in rating of medical trainees

December 4, 2012
Peter Yeates, M.B.B.S., M.Clin.Ed., of the University of Manchester, United Kingdom, and colleagues conducted a study to examine whether observations of the performance of postgraduate year 1 physicians influence raters' ...

Research evaluates possible benefit of mini-interviews as part of medical school admission process

December 4, 2012
Kevin W. Eva, Ph.D., of the University of British Columbia, Vancouver, and colleagues conducted a study to determine whether students deemed acceptable through a revised admissions protocol using a 12-station multiple mini-interview ...

Study shows persistence pays off in the mating game

December 23, 2011
(Medical Xpress) -- A new study co-authored by a University of Texas at Austin psychology professor suggests that self-deception may help men succeed in the mating game, while women will benefit more from effective communication.

Recommended for you

Wine is good for you—to a point

January 18, 2018
The Mediterranean diet has become synonymous with healthy eating, but there's one thing in it that stands out: It's cool to drink wine.

Sleep better, lose weight?

January 17, 2018
(HealthDay)—Sleeplessness could cost you when it's time to stand on your bathroom scale, a new British study suggests.

Who uses phone apps to track sleep habits? Mostly the healthy and wealthy in US

January 16, 2018
The profile of most Americans who use popular mobile phone apps that track sleep habits is that they are relatively affluent, claim to eat well, and say they are in good health, even if some of them tend to smoke.

Improvements in mortality rates are slowed by rise in obesity in the United States

January 15, 2018
With countless medical advances and efforts to curb smoking, one might expect that life expectancy in the United States would improve. Yet according to recent studies, there's been a reduction in the rate of improvement in ...

Teens likely to crave junk food after watching TV ads

January 15, 2018
Teenagers who watch more than three hours of commercial TV a day are more likely to eat hundreds of extra junk food snacks, according to a report by Cancer Research UK.

Can muesli help against arthritis?

January 15, 2018
It is well known that healthy eating increases a general sense of wellbeing. Researchers at Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) have now discovered that a fibre-rich diet can have a positive influence ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.