Linguistic and cultural factors influence GPs exams
Research at King's College London, with the Universities of Nottingham and Cardiff, has identified features of both candidate performance and of structured clinical skills examination design in general that contribute to our understanding of the reasons why International Medical Graduates (IMGs) have a lower success rate than their UK counterparts in the licensing examination for GPs in the UK. This research and associated development work, carried out in partnership with the Royal College of General Practitioners, was designed to have a positive impact on the ongoing concern about this gap in pass rates.
Many post-graduate medical exams have differential pass rates between UK graduates and those with their primary medical qualifications from overseas. These frequently include a standardised assessment of clinical consulting skills with role-players as patients. The findings of this research have implications for other similar assessments of medical clinical practice.
Celia Roberts, Professor Emerita in Sociolinguistics, Centre for Language, Discourse and Communication at King's, who led the research, said: 'The published report demonstrates that there are features of candidate performance associated with lower grades, but it is unhelpful to describe them as difficulties with 'language' or 'culture' in an undifferentiated way, set aside from discussions of fairness. There are also aspects of structured, simulated examinations which, unintentionally, contribute to the weight of the exam for the IMG group.'
This research focused on linguistic and cultural factors that could affect pass rates (excluding those cases where serious medical errors and omissions were causes of failure). Differences between successful and unsuccessful candidates were only apparent at quite subtle levels of communication. These related to speech style (for example, sounding formulaic or using stereotyped phrases), misunderstandings between role player patients and candidates and difficulties in giving clear explanations about the diagnosis or clinical management. Such apparently small differences can have large consequences for outcomes and for examination marks. The findings suggest that more explicit training and support is required, particularly for the IMG group.
Professor Roberts said: 'Simulated structured examinations, such as the Clinical Skills Assessment, have their own internal 'culture' and linguistic 'fingerprint' – ways of talking developed specifically in response to performing simulated consultations with role player patients. This is most evident in assessment of interpersonal skills. This area of the exam is the most subjective and the most challenging for IMG candidates.'
The research confirms other studies that have raised questions about how best to measure interpersonal skills (IPS), adding a cultural/ linguistic dimension. While observable aspects of communication can be assessed, more subjective features of IPS, such as 'empathy' and 'rapport' are not best assessed in a highly standardised, high stakes exam of simulated consultations. The authors have made recommendations on how interpersonal effectiveness can be better defined and assessed, using a new analytic language.
The results of the research are being used to develop new e-learning materials in preparation for the CSA, which will be useful for all candidates, but particularly for overseas trained candidates. These e-learning materials are being developed by Dr Sarah Atkins of the Centre for Research in Applied Linguistics at the University of Nottingham and should be available on the Royal College of GPs website in the next few months.
She said: 'It's exciting to build upon the findings of this original King's College London research project here at Nottingham under an Economic and Social Research Council grant. We hope the e-learning and workshop materials we're producing here, in conjunction with the RCGP, will provide valuable evidence-based learning tools for GPs who are preparing to take the exam and give them every chance of success.'
Professor Kamila Hawthorne, Associate Dean at the School of Medicine, Cardiff University, and one of the authors, said: 'The findings of this research are by no means unique to the Membership of the RCGP examinations, but are applicable in other clinical settings both in undergraduate and postgraduate medicine, where standardised exams with simulated cases and simulated patients are used. More widely, this research addresses a central debate in institutional life – how to balance validity in assessments and be fair to a diverse group of candidates in an increasingly diverse society, while maintaining reliability with standardised and universal marking criteria.'
