Faculty Collaboration Database

An assessment of the inter-rater reliability of the ASA physical status score in the orthopaedic trauma population. Injury 2015 Apr;46(4):542-6

Date

03/25/2014

Pubmed ID

24656923

DOI

10.1016/j.injury.2014.02.039

Scopus ID

2-s2.0-84925136842 (requires institutional sign-in at Scopus site)   36 Citations

Abstract

OBJECTIVES: Although recent literature has demonstrated the utility of the ASA score in predicting postoperative length of stay, complication risk and potential utilization of other hospital resources, the ASA score has been inconsistently assigned by anaesthesia providers. This study tested the reliability of assignment of the ASA score classification by both attending anaesthesiologists and anaesthesia residents specifically among the orthopaedic trauma patient population.

METHODS: Nine case-based scenarios were created involving preoperative patients with isolated operative orthopaedic trauma injuries. The cases were created and assigned a reference score by both an attending anaesthesiologist and orthopaedic trauma surgeon. Attending and resident anaesthesiologists were asked to assign an ASA score for each case. Rater versus reference and inter-rater agreement amongst respondents was then analyzed utilizing Fleiss's Kappa and weighted and unweighted Cohen's Kappa.

RESULTS: Thirty three individuals provided ASA scores for each of the scenarios. The average rater versus reference reliability was substantial (Kw=0.78, SD=0.131, 95% CI=0.73-0.83). The average rater versus reference Kuw was also substantial (Kuw=0.64, SD=0.21, 95% CI=0.56-0.71). The inter-rater reliability as evaluated by Fleiss's Kappa was moderate (K=0.51, p<.001). An inter-rater comparison within the group of attendings (K=0.50, p<.001) and within the group of residents were both moderate (K=0.55, p<.001). There was a significant increase in the level of inter-rater reliability from the self-reported 'very uncomfortable' participants to the 'very comfortable' participants (uncomfortable K=0.43, comfortable K=0.59, p<.001).

CONCLUSIONS: This study shows substantial agreement strength for reliability of the ASA score among anaesthesiologists when evaluating orthopaedic trauma patients. The significant increase in inter-rater reliability based on anaesthesiologists' comfort with the ASA scoring method implies a need for further evaluation of ASA assessment training and routine use on the ground. These findings support the use of the ASA score as a statistically reliable tool in orthopaedic trauma.

Author List

Ihejirika RC, Thakore RV, Sathiyakumar V, Ehrenfeld JM, Obremskey WT, Sethi MK

Author

Jesse Ehrenfeld MD, MPH Sr Assoc Dean, Exec Director, Professor in the Advancing a Healthier Wisconsin Endowment department at Medical College of Wisconsin

MESH terms used to index this publication - Major topics in bold

Anesthesiology
Fractures, Bone
Health Status Indicators
Humans
Length of Stay
Multiple Trauma
Observer Variation
Orthopedic Procedures
Radiographic Image Interpretation, Computer-Assisted
Reproducibility of Results
Research Design
Societies, Medical
United States