Surgery Clerkship Evaluations Are Insufficient for Clinical Skills Appraisal: The Value of a Medical Student Surgical Objective Structured Clinical Examination. J Surg Educ 2017;74(2):286-294
Date
10/04/2016Pubmed ID
27692808DOI
10.1016/j.jsurg.2016.08.018Scopus ID
2-s2.0-84999767782 (requires institutional sign-in at Scopus site) 18 CitationsAbstract
OBJECTIVE: Optimal methods for medical student assessment in surgery remain elusive. Faculty- and housestaff-written evaluations constitute the chief means of student assessment in medical education. However, numerous studies show that this approach has poor specificity and a high degree of subjectivity. We hypothesized that an objective structured clinical examination (OSCE) in the surgery clerkship would provide additional data on student performance that would confirm or augment other measures of assessment.
DESIGN: We retrospectively reviewed data from OSCEs, National Board of Medical Examiners shelf examinations, oral presentations, and written evaluations for 51 third-year Harvard Medical School students rotating in surgery at Massachusetts General Hospital from 2014 to 2015. We expressed correlations between numeric variables in Pearson coefficients, stratified differences between rater groups by one-way analysis of variance, and compared percentages with 2-sample t-tests. We examined commentary from both OSCE and clinical written evaluations through textual analysis and summarized these results in percentages.
RESULTS: OSCE scores and clinical evaluation scores correlated poorly with each other, as well as with shelf examination scores and oral presentation grades. Textual analysis of clinical evaluation comments revealed a heavy emphasis on motivational factors and praise, whereas OSCE written comments focused on cognitive processes, patient management, and methods to improve performance.
CONCLUSIONS: In this single-center study, an OSCE provided clinical skills data that were not captured elsewhere in the surgery clerkship. Textual analysis of faculty evaluations reflected an emphasis on interpersonal skills, rather than appraisal of clinical acumen. These findings suggest complementary roles of faculty evaluations and OSCEs in medical student assessment.
Author List
Butler KL, Hirsh DA, Petrusa ER, Yeh DD, Stearns D, Sloane DE, Linder JA, Basu G, Thompson LA, de Moya MAAuthor
Marc Anthony De Moya MD Chief, Professor in the Surgery department at Medical College of WisconsinMESH terms used to index this publication - Major topics in bold
AccreditationClinical Clerkship
Clinical Competence
Databases, Factual
Education, Medical, Undergraduate
Educational Measurement
Female
General Surgery
Humans
Male
Massachusetts
Medical History Taking
Physical Examination
Retrospective Studies
Schools, Medical
Students, Medical
Young Adult
