Methodology to improve data quality from chart review in the managed care setting. Am J Manag Care 2002 Sep;8(9):787-93
Date
09/18/2002Pubmed ID
12234019Scopus ID
2-s2.0-0036733986 (requires institutional sign-in at Scopus site) 34 CitationsAbstract
BACKGROUND: Because inherent variability may exist in data collected by multiple reviewers or from potential difficulties with data abstraction tools, we developed a standardized method of evaluating interrater reliability (IRR) for clinical studies, HEDIS effectiveness of care measures, and onsite/medical record reviews.
OBJECTIVE: To demonstrate the ability of our standardized methods of data collection and analysis of results to determine the extent of agreement between multiple reviewers; identify areas for improvement in data collection procedures; and improve data reliability.
STUDY DESIGN: A prospective chart review with concurrent IRR.
METHODS: A subsample of patient records included in the Highmark Blue Cross Blue Shield/Keystone Health Plan West basic medical review for each HEDIS measure was selected for the IRR study. An experienced nurse ("gold standard") conducted a blinded concurrent review of these records. Using the kappa statistic (kappa) we evaluated interobserver agreement between results of the onsite reviewers and the "gold standard" from 1997 through 2000. Revised data collection methods and enhanced reviewer training were incorporated for measures showing areas for rater improvement.
RESULTS: Results across years showed excellent IRR for most measures; however, each year 1 or 2 measures showed areas for rater improvement (1997 Papanicolaou kappa = 0.50; 1998 well-child visits 3 to 6 years kappa = 0.37; 1999 comprehensive diabetes kappa = 0.73; high blood pressure kappa = 0.73). After reevaluating these measures, the results of the kappa showed excellent interrater agreement in subsequent years.
CONCLUSIONS: Standardized methods of data collection and evaluation of IRR results provides health plans increased confidence in data collection, statistical analyses, and in reaching conclusions and deriving relevant recommendations.
Author List
Cassidy LD, Marsh GM, Holleran MK, Ruhl LSAuthor
Laura Cassidy PhD Associate Dean, Professor in the Institute for Health and Equity department at Medical College of WisconsinMESH terms used to index this publication - Major topics in bold
Blue Cross Blue Shield Insurance PlansConcurrent Review
Data Collection
Humans
Managed Care Programs
Medical Audit
Medical Records
Prospective Studies
Quality Control
Quality Indicators, Health Care
Reproducibility of Results
United States
