Examiners tend to grade relative to work already seen, research finds
(Medical Xpress)—How well you fare on a subjective evaluation – whether it's of you treating a patient, auditioning for a play or even interviewing for a job – may depend largely on the person who was examined just before you. And the worse the person before you did, the better for you.
That sobering conclusion by University of British Columbia professor Kevin Eva resulted from a controlled experiment that examined whether physician-educators' grading of younger doctors could be influenced by the quality of those they had previously evaluated.
Conducted with researchers at the University of Manchester, it asked physician educators in England and Wales to grade videos of young doctors in their first year of post-graduate training. The doctors portrayed in the videos followed scripts to represent three levels of performances – good, poor, and borderline – as they examined patients.
Some physician-educators were primed by first viewing the good performances while others viewed the poor performances. Then both groups were compared on their grading of the borderline performances.
Educators who had been primed by poor performances consistently gave better grades than those who had just watched the good performances – depending on the patient case, the grades were 30 to 100 per cent better.
"This experiment shows that judging someone's performance – whether it's clinical skills, essay-writing, or figure skating – is likely to be relative," says Eva, Director of Educational Research and Scholarship in the Department of Medicine and a Senior Scientist at UBC's Centre for Health Education Scholarship.
"While such assessments are unavoidable in determining whether a student has mastered the required competencies in a field like medicine, we need to take steps to minimize contrast bias," Eva says. "It is important to ensure that there are a sufficient number of evaluations – the more data points, the more reliable the aggregate score will be."
In the second study published in the same edition of JAMA, Eva and collaborators examined the validity of multiple mini-interviews (MMIs) – a dozen, brief encounters in which applicants are asked to solve a problem through discussion with an interviewer, or through interaction with an actor or another applicant. MMIs have come to replace longer, more traditional interviews at most Canadian medical schools (including UBC's Faculty of Medicine).
The researchers found that students accepted to McMaster University, the first Canadian medical school to adopt MMIs, performed better years later on licensing examinations than students whom McMaster rejected but who were accepted at other, non-MMI schools.
Eva is an expert in the education of health professionals but he has also demonstrated how seemingly unrelated factors can affect judgment in a very different domain – men's physical attractiveness.
In 1999, he and a fellow graduate student at McMaster University asked women to view photographs and brief descriptions of men, and to rate their attractiveness on a 7-point scale. Men labeled as married had a mean attractiveness score that was 23 per cent higher than when the same men were labeled as single. As he and his collaborator wrote in CMAJ (Canadian Medical Association Journal): "Our findings do support the notion that being 'taken' influences perceptions of 'goodness."
Provided by University of British Columbia
- Bias may exist in rating of medical trainees Dec 04, 2012 | not rated yet | 0
- Research evaluates possible benefit of mini-interviews as part of medical school admission process Dec 04, 2012 | not rated yet | 0
- Study supports students not repeating a school year Oct 14, 2011 | not rated yet | 0
- Medical student gender and self-confidence Oct 03, 2008 | not rated yet | 0
- Allowing native language in school benefits Mexican-American students, researcher finds Sep 21, 2011 | not rated yet | 0
- Motion perception revisited: High Phi effect challenges established motion perception assumptions Apr 23, 2013 | 3 / 5 (2) | 2
- Anything you can do I can do better: Neuromolecular foundations of the superiority illusion (Update) Apr 02, 2013 | 4.5 / 5 (11) | 5
- The visual system as economist: Neural resource allocation in visual adaptation Mar 30, 2013 | 5 / 5 (2) | 9
- Separate lives: Neuronal and organismal lifespans decoupled Mar 27, 2013 | 4.9 / 5 (8) | 0
- Sizing things up: The evolutionary neurobiology of scale invariance Feb 28, 2013 | 4.8 / 5 (10) | 14
Classical and Quantum Mechanics via Lie algebras
Apr 15, 2011 I'd like to open a discussion thread for version 2 of the draft of my book ''Classical and Quantum Mechanics via Lie algebras'', available online at http://lanl.arxiv.org/abs/0810.1019 , and for the...
- More from Physics Forums - Independent Research
More news stories
Talking on a hands-free device while behind the wheel can lead to a sharp increase in errors that could imperil other drivers on the road, according to new research from the University of Alberta.
Health 3 hours ago | not rated yet | 0
(HealthDay)—More than one in four of those eligible for new premium assistance tax credits under the Affordable Care Act (ACA) do not have a checking account and will not be able to receive premiums from ...
Health 5 hours ago | not rated yet | 0
After studying noise in one French Quarter neighborhood of New Orleans to determine whether or not noise levels exceeded municipal ordinances, Annette Hurley, PhD, Assistant Professor of Audiology at LSU Health Sciences Center ...
Health 6 hours ago | not rated yet | 0
Young children who missed more than half of recommended well-child visits had up to twice the risk of hospitalization compared to children who attended most of their visits, according to a study published today in the American Jo ...
Health 6 hours ago | not rated yet | 0
The individualisation of drug treatments to support patients to self-manage their conditions is a concept that sits at the heart of policy, but a recent study in BMJ Open shows that there is no concrete defini ...
Health 9 hours ago | 3 / 5 (1) | 0
3 hours ago | 5 / 5 (1) | 0 |
9 hours ago | 5 / 5 (3) | 0 |
6 hours ago | 5 / 5 (1) | 0 |
6 hours ago | 5 / 5 (2) | 0 |
Kate O'Reilly's spring allergy survival kit includes the usual stuff - nasal sprays, allergy pills and a box of tissues. This season, she's added a new weapon to her line of defense: an app on her smartphone.
4 hours ago | not rated yet | 0