The reliability and validity of the American Board of Internal Medicine Monthly Evaluation Form. Acad Med 2003 Nov;78(11):1175-82
Date
11/08/2003Pubmed ID
14604883DOI
10.1097/00001888-200311000-00021Scopus ID
2-s2.0-0242458245 (requires institutional sign-in at Scopus site) 19 CitationsAbstract
PURPOSE: To report indicators of reliability and validity of the American Board of Internal Medicine Evaluation Form (ABIM-MEF) at one institution (Wright-Patterson Medical Center).
METHOD: Completed ABIM-MEFs from 1990-1999 were reviewed. Reliability measures included Cronbach alpha, interrater reliability, and rating consistency between different types of staff and rotations. Construct validity was investigated by tracking ABIM-MEF scores over time and with factor analysis. Predictive validity was assessed by correlating ABIM-MEF scores with the In-training Examination and ABIM Certifying Examination results
RESULTS: The 71 residents averaged 12 ABIM-MEFs per year. The forms had a Cronbach alpha of 0.96 and high interrater reliability (intraclass correlation coefficients > 0.80). Ratings did not differ by type of attending or rotation, except that noninternists rated residents lower on procedural skills. ABIM-MEF questions about judgment, knowledge, and clinical skills showed significant improvement from month to month during each academic year as well as year to year. In contrast, questions on professional attitudes, humanism, and procedural skills sections improved between postgraduate year 1 and postgraduate year 2 only. ABIM-MEF questions collapsed into two domains in factor analysis: judgment-knowledge-skills and attitude-humanism. ABIM-MEF questions from judgment and knowledge sections modestly predicted In-training Examination and ABIM Certifying Examination results. In contrast, professional attitude, humanism, and clinical as well as procedural skill questions had little discriminative ability.
CONCLUSIONS: The ABIM-MEF appears to be reliable and valid. Further, factor analysis results support the ABIM's movement to simplify the monthly evaluation form to the new Accreditation Council for Graduate Medical Education core competencies.
Author List
Durning SJ, Cation LJ, Jackson JLAuthor
Jeffrey L. Jackson MD Professor in the Medicine department at Medical College of WisconsinMESH terms used to index this publication - Major topics in bold
Academic Medical CentersAdult
Female
Humans
Internal Medicine
Male
Professional Competence
Reproducibility of Results
Sensitivity and Specificity
Specialty Boards
Surveys and Questionnaires
