Simultaneous estimation of cluster number and feature sparsity in high-dimensional cluster analysis. Biometrics 2022 Jun;78(2):574-585
Date
02/24/2021Pubmed ID
33621349DOI
10.1111/biom.13449Scopus ID
2-s2.0-85102645202 (requires institutional sign-in at Scopus site) 3 CitationsAbstract
Estimating the number of clusters (K) is a critical and often difficult task in cluster analysis. Many methods have been proposed to estimate K, including some top performers using resampling approach. When performing cluster analysis in high-dimensional data, simultaneous clustering and feature selection is needed for improved interpretation and performance. To our knowledge, little has been studied for simultaneous estimation of K and feature sparsity parameter in a high-dimensional exploratory cluster analysis. In this paper, we propose a resampling method to bridge this gap and evaluate its performance under the sparse K-means clustering framework. The proposed target function balances between sensitivity and specificity of clustering evaluation of pairwise subjects from clustering of full and subsampled data. Through extensive simulations, the method performs among the best over classical methods in estimating K in low-dimensional data. For high-dimensional simulation data, it also shows superior performance to simultaneously estimate K and feature sparsity parameter. Finally, we evaluated the methods in four microarray, two RNA-seq, one SNP, and two nonomics datasets. The proposed method achieves better clustering accuracy with fewer selected predictive genes in almost all real applications.
Author List
Li Y, Zeng X, Lin CW, Tseng GCAuthor
Chien-Wei Lin PhD Associate Professor in the Data Science Institute department at Medical College of WisconsinMESH terms used to index this publication - Major topics in bold
AlgorithmsCluster Analysis
Computer Simulation
Humans
Sensitivity and Specificity
