Development of a tool to assess medical oral language proficiency Journal Article
| Authors: | Diamond, L. C.; Gregorich, S. E.; Karliner, L.; González, J.; Pérez-Cordón, C.; Iniguez, R.; Alberto Figueroa, J.; Izquierdo, K.; Ortega, P. |
| Article Title: | Development of a tool to assess medical oral language proficiency |
| Abstract: | Purpose To communicate with linguistically diverse patients, medical students and physicians often use their non-English-language skills. However, there is no standard protocol to determine whether those skills are adequate before patient care. This causes many physicians, institutions, educators, and learners to forgo non-English-language proficiency assessment altogether. The purpose of this study is to report on the development, refinement, and interrater reliability of the Physician Oral Language Observation Matrix (POLOM), a rater-based tool assessing 6 language skill categories observed during clinical interactions: comprehension, fluency/fluidity, vocabulary, pronunciation, grammar, and communication. This study focused on the use of the POLOM in Spanish interactions. Method The authors adapted an existing language observation tool for use in clinical settings, creating the preliminary POLOM. Next, they iteratively refined the tool from April to July 2021 using videorecorded medical student-standardized patient encounters from a U.S.-based medical Spanish program. In each refinement iteration, 4 bilingual raters (2 physicians and 2 linguists) independently rated 3 to 6 encounters and convened to discuss ratings with the goals of improving instrument instructions, descriptors, and subsequent rater agreement. Using the final POLOM, raters independently rated 50 videos in rotating interdisciplinary pairs. Generalizability theory was applied to estimate reliability via interrater agreement (dependability) coefficients (range 0-1) for each POLOM category and the total score. Results POLOM total score dependability equaled 0.927 (single rater) and 0.962 (averaged across 2 raters). The highest mean score was observed for the comprehension category (4.15; range 1-5) while the lowest was for communication (3.01; range 1-5). Conclusions Raters achieved a high level of agreement on POLOM assessments of students' medical oral Spanish proficiency. The POLOM is the first assessment tool that provides examinees and instructors with both a holistic and detailed review of clinician non-English oral language skills as contextualized for patient care. © 2023 Lippincott Williams and Wilkins. All rights reserved. |
| Keywords: | reproducibility; reproducibility of results; communication; interpersonal communication; language; medical student; students, medical; humans; human; vocabulary |
| Journal Title: | Academic Medicine |
| Volume: | 98 |
| Issue: | 4 |
| ISSN: | 1040-2446 |
| Publisher: | Lippincott Williams & Wilkins |
| Date Published: | 2023-04-01 |
| Start Page: | 480 |
| End Page: | 490 |
| Language: | English |
| DOI: | 10.1097/acm.0000000000004942 |
| PUBMED: | 36484536 |
| PROVIDER: | scopus |
| PMCID: | PMC10373794 |
| DOI/URL: | |
| Notes: | The MSK Cancer Center Support Grant (P30 CA008748) is acknowledged in the PubMed record and PDF. Corresponding author is MSK author Lisa C. Diamond -- Source: Scopus |
Altmetric
Citation Impact
BMJ Impact Analytics
Related MSK Work