Faculty Collaboration Database

Approaches to Assessing and Adjusting for Selective Outcome Reporting in Meta-analysis. J Gen Intern Med 2022 Apr;37(5):1247-1253

Date

10/21/2021

Pubmed ID

34669145

Pubmed Central ID

PMC8971211

DOI

10.1007/s11606-021-07135-3

Scopus ID

2-s2.0-85117505380 (requires institutional sign-in at Scopus site)   2 Citations

Abstract

BACKGROUND: Selective or non-reporting of study outcomes results in outcome reporting bias.

OBJECTIVE: We sought to develop and assess tools for detecting and adjusting for outcome reporting bias.

DESIGN: Using data from a previously published systematic review, we abstracted whether outcomes were reported as collected, whether outcomes were statistically significant, and whether statistically significant outcomes were more likely to be reported. We proposed and tested a model to adjust for unreported outcomes and compared our model to three other methods (Copas, Frosi, trim and fill). Our approach assumes that unreported outcomes had a null intervention effect with variance imputed based on the published outcomes. We further compared our approach to these models using simulation, and by varying levels of missing data and study sizes.

RESULTS: There were 286 outcomes reported as collected from 47 included trials: 142 (48%) had the data provided and 144 (52%) did not. Reported outcomes were more likely to be statistically significant than those collected but for which data were unreported and for which non-significance was reported (RR, 2.4; 95% CI, 1.9 to 3.0). Our model and the Copas model provided similar decreases in the pooled effect sizes in both the meta-analytic data and simulation studies. The Frosi and trim and fill methods performed poorly.

LIMITATIONS: Single intervention of a single disease with only randomized controlled trials; approach may overestimate outcome reporting bias impact.

CONCLUSION: There was evidence of selective outcome reporting. Statistically significant outcomes were more likely to be published than non-significant ones. Our simple approach provided a quick estimate of the impact of unreported outcomes on the estimated effect. This approach could be used as a quick assessment of the potential impact of unreported outcomes.

Author List

Jackson JL, Balk EM, Hyun N, Kuriyama A

Author

Jeffrey L. Jackson MD Professor in the Medicine department at Medical College of Wisconsin

MESH terms used to index this publication - Major topics in bold

Bias
Computer Simulation
Humans
Meta-Analysis as Topic
Publication Bias