Medical College of Wisconsin
CTSICores SearchResearch InformaticsREDCap

Resampling-based stepwise multiple testing procedures with applications to clinical trial data. Pharm Stat 2021 Mar;20(2):297-313



Pubmed ID




Scopus ID

2-s2.0-85093927042 (requires institutional sign-in at Scopus site)   2 Citations


Endpoints in clinical trials are often highly correlated. However, the commonly used multiple testing procedures in clinical trials either do not take into consideration the correlations among test statistics or can only exploit known correlations. Westfall and Young constructed a resampling-based stepdown method that implicitly utilizes the correlation structure of test statistics in situations with unknown correlations. However, their method requires a "subset pivotality" assumption. Romano and Wolf proposed a more general stepdown method, which does not require such an assumption. There is at present little experience with the application of such methods in analyzing clinical trial data. We advocate the application of resampling-based multiple testing procedures to clinical trials data when appropriate. We have conjectured that the resampling-based stepdown methods can be extended to a stepup procedure under appropriate assumptions and examined the performance of both stepdown and stepup methods under a variety of correlation structures and distribution types. Results from our simulation studies support the use of the resampling-based methods under various scenarios, including binary data and small samples, with strong control of Family wise type I error rate (FWER). Under positive dependence and for binary data even under independence, the resampling-based methods are more powerful than the Holm and Hochberg methods. Last, we illustrate the advantage of the resampling-based stepwise methods with two clinical trial data examples: a cardiovascular outcome trial and an oncology trial.

Author List

He J, Li F, Gao Y, Rothmann M


Yan Gao PhD Assistant Professor in the Institute for Health and Equity department at Medical College of Wisconsin

MESH terms used to index this publication - Major topics in bold

Clinical Trials as Topic
Computer Simulation
Research Design