Medical College of Wisconsin
CTSICores SearchResearch InformaticsREDCap

A human-computer collaborative approach to identifying common data elements in clinical trial eligibility criteria. J Biomed Inform 2013 Feb;46(1):33-9



Pubmed ID


Pubmed Central ID




Scopus ID

2-s2.0-84873106410   27 Citations


OBJECTIVE: To identify Common Data Elements (CDEs) in eligibility criteria of multiple clinical trials studying the same disease using a human-computer collaborative approach.

DESIGN: A set of free-text eligibility criteria from clinical trials on two representative diseases, breast cancer and cardiovascular diseases, was sampled to identify disease-specific eligibility criteria CDEs. In this proposed approach, a semantic annotator is used to recognize Unified Medical Language Systems (UMLSs) terms within the eligibility criteria text. The Apriori algorithm is applied to mine frequent disease-specific UMLS terms, which are then filtered by a list of preferred UMLS semantic types, grouped by similarity based on the Dice coefficient, and, finally, manually reviewed.

MEASUREMENTS: Standard precision, recall, and F-score of the CDEs recommended by the proposed approach were measured with respect to manually identified CDEs.

RESULTS: Average precision and recall of the recommended CDEs for the two diseases were 0.823 and 0.797, respectively, leading to an average F-score of 0.810. In addition, the machine-powered CDEs covered 80% of the cardiovascular CDEs published by The American Heart Association and assigned by human experts.

CONCLUSION: It is feasible and effort saving to use a human-computer collaborative approach to augment domain experts for identifying disease-specific CDEs from free-text clinical trial eligibility criteria.

Author List

Luo Z, Miotto R, Weng C


Jake Luo Ph.D. Associate Professor; Director, Center for Biomedical Data and Language Processing (BioDLP) in the Health Informatics & Administration department at University of Wisconsin - Milwaukee

MESH terms used to index this publication - Major topics in bold

Clinical Trials as Topic
Cooperative Behavior
Information Storage and Retrieval
Man-Machine Systems
Patient Selection
Unified Medical Language System