Valx: A System for Extracting and Structuring Numeric Lab Test Comparison Statements from Text

Abstract

Objectives To develop an automated method for extracting and structuring numeric lab test comparison statements from text and evaluate the method using clinical trial eligibility criteria text. Methods Leveraging semantic knowledge from the Unified Medical Language System (UMLS) and domain knowledge acquired from the Internet, Valx takes 7 steps to extract and normalize numeric lab test expressions: 1) text preprocessing, 2) numeric, unit, and comparison operator extraction, 3) variable identification using hybrid knowledge, 4) variable - numeric association, 5) context-based association filtering, 6) measurement unit normalization, and 7) heuristic rule-based comparison statements verification. Our reference standard was the consensus-based annotation among three raters for all comparison statements for two variables, i.e., HbA1c and glucose, identified from all of Type 1 and Type 2 diabetes trials in ClinicalTrials.gov. Results The precision, recall, and F-measure for structuring HbA1c comparison statements were 99.6%, 98.1%, 98.8% for Type 1 diabetes trials, and 98.8%, 96.9%, 97.8% for Type 2 Diabetes trials, respectively. The precision, recall, and F-measure for structuring glucose comparison statements were 97.3%, 94.8%, 96.1% for Type 1 diabetes trials, and 92.3%, 92.3%, 92.3% for Type 2 diabetes trials, respectively. Conclusions Valx is effective at extracting and structuring free-text lab test comparison statements in clinical trial summaries. Future studies are warranted to test its generalizability beyond eligibility criteria text. The open-source Valx enables its further evaluation and continued improvement among the collaborative scientific community.

GIST 2.0: A scalable multi-trait metric for quantifying population representativeness of individual clinical studies

“…The extraction of eligibility criteria from clinical trial summaries is a complex process that uses natural language processing (NLP) techniques. Those details are beyond the scope of this paper but can be found in [27] or substituted by other related NLP methods. With an approximation of the real-world patient population and eligibility criteria for a clinical trial we are in a position to define GIST 2.0.…”

Multivariate analysis of the population representativeness of related clinical studies

“…We have created a web-based visualization tool VITTA (http://is.gd/VITTA) to show how studies vary in their study populations with respect to study traits, one at a time [20]. Different trait names and measurement units were normalized by the Valx system [24] developed in house. Valx leveraged the Unified Medical Language System (UMLS) [25] and domain knowledge from Web resources such as MedLinePlus [26] to parse numeric expressions into a structured format.…”

Visual aggregate analysis of eligibility features of clinical trials

“…From each downloaded XML file, we extracted study characteristics and structured the free-text eligibility criteria text using previously published methods [28, 48]. We created a table in COMPACT and saved the extracted metadata of studies: National Clinical Trial (NCT) number (a unique ID assigned by ClinicalTrials.gov), study type (e.g., interventional, observational), intervention type, study design (i.e., allocation and intervention model for interventional studies; time perspective for observational studies), phase, sponsor agency type (e.g., NIH, industry), enrollment status, start date, gender, minimum age, maximum age, and enrollment.…”

“…We used a numeric expression extraction tool called Valx [48] to structure numeric expressions from free-text eligibility criteria. To identify the quantitative features and extract numeric expressions, Valx pre-compiled a list of unique quantitative features using domain knowledge (from online resources such as WebMD [51], WHO website [52], and web communities for various conditions) and frequent UMLS (Unified Medical Language System) [53] concepts with their synonyms, which cover selected semantic types relevant to quantifiable attributes in clinical studies (e.g., Clinical Attribute, Laboratory or Test Result, and Quantitative Concept ).…”