Medical College of Wisconsin
CTSICores SearchResearch InformaticsREDCap

Extracting temporal constraints from clinical research eligibility criteria using conditional random fields. AMIA Annu Symp Proc 2011;2011:843-52



Pubmed ID


Pubmed Central ID


Scopus ID

2-s2.0-84874216479   24 Citations


Temporal constraints are present in 38% of clinical research eligibility criteria and are crucial for screening patients. However, eligibility criteria are often written as free text, which is not amenable for computer processing. In this paper, we present an ontology-based approach to extracting temporal information from clinical research eligibility criteria. We generated temporal labels using a frame-based temporal ontology. We manually annotated 150 free-text eligibility criteria using the temporal labels and trained a parser using Conditional Random Fields (CRFs) to automatically extract temporal expressions from eligibility criteria. An evaluation of an additional 60 randomly selected eligibility criteria using manual review achieved an overall precision of 83%, a recall of 79%, and an F-score of 80%. We illustrate the application of temporal extraction with the use cases of question answering and free-text criteria querying.

Author List

Luo Z, Johnson SB, Lai AM, Weng C


Jake Luo Ph.D. Associate Professor; Director, Center for Biomedical Data and Language Processing (BioDLP) in the Health Informatics & Administration department at University of Wisconsin - Milwaukee

MESH terms used to index this publication - Major topics in bold

Biomedical Research
Clinical Trials as Topic
Databases, Factual
Eligibility Determination
Information Storage and Retrieval
Natural Language Processing
Vocabulary, Controlled