Faculty Collaboration Database

Comparing PDA- and paper-based evaluation of the clinical skills of third-year students. WMJ 2011 Feb;110(1):9-13

Date

04/09/2011

Pubmed ID

21473507

Scopus ID

2-s2.0-79957648676 (requires institutional sign-in at Scopus site)   13 Citations

Abstract

BACKGROUND: The mini-Clinical Evaluation Exercise (mini-CEX) is used to assess medical students' clinical skills during medicine clerkships.

PURPOSE: To evaluate reliability, feasibility, and user satisfaction of a paper vs PDA-based mini-CEX in a third-year medicine clerkship.

METHODS: The mini-CEX was reformatted as a PDA-based rating form for a medicine clerkship over 1 year. Faculty and residents were instructed to use either paper-based or the PDA form to assess clinical skills of students. A 9-point Likert scale was used to assess clinical skills and user satisfaction. Independent t-tests were used to assess differences between delivery formats.

FINDINGS: Nearly all (98%) students completed 2 Mini-CEXs, with 275 PDA- and 101 paper-based records performed. Form reliability (Cronbach alpha) exceeded 0.9 for both. Overall resident satisfaction scores with the PDA form (7.2 +/- 1.8) were higher (P = 0.01) than the paper-based form (6.6/1.7). However, faculty satisfaction scores with the PDA form (6.9 +/- 1.6) were significantly lower (P = 0.01) than the paper form (7.6 +/- 1.5). Mean scores for all 7 clinical competencies of PDA format (7.9 +/- 0.9) were higher than the paper-based (7.6 +/- 1.1) version (P = .01). Mean observation (26 min; +/- 16) and feedback time (11 min +/- 8) were longer (both P < .05) with PDA-based form compared to the paper version (22 min +/- 14);(8.7 +/- 6.3). Student and evaluator satisfaction ratings were not significantly different by form.

CONCLUSIONS: Both PDA- and paper-based mini-CEX delivery was acceptable to evaluators and students with both formats demonstrating high reliability. However, because evaluators' satisfaction, observation, and feedback time differed by form, further studies are needed to determine factors influencing rating variability.

Author List

Torre DM, Treat R, Durning S, Elnicki DM

Author

Robert W. Treat PhD Associate Professor in the Academic Affairs department at Medical College of Wisconsin

MESH terms used to index this publication - Major topics in bold

Clinical Clerkship
Clinical Competence
Computers, Handheld
Education, Medical, Undergraduate
Educational Measurement
Feasibility Studies
Female
Humans
Male
Reproducibility of Results
Wisconsin