Identifying the status of genetic lesions in cancer clinical trial documents using machine learning

Abstract

BackgroundMany cancer clinical trials now specify the particular status of a genetic lesion in a patient's tumor in the inclusion or exclusion criteria for trial enrollment. To facilitate search and identification of gene-associated clinical trials by potential participants and clinicians, it is important to develop automated methods to identify genetic information from narrative trial documents.MethodsWe developed a two-stage classification method to identify genes and genetic lesion statuses in clinical trial documents extracted from the National Cancer Institute's (NCI's) Physician Data Query (PDQ) cancer clinical trial database. The method consists of two steps: 1) to distinguish gene entities from non-gene entities such as English words; and 2) to determine whether and which genetic lesion status is associated with an identified gene entity. We developed and evaluated the performance of the method using a manually annotated data set containing 1,143 instances of the eight most frequently mentioned genes in cancer clinical trials. In addition, we applied the classifier to a real-world task of cancer trial annotation and evaluated its performance using a larger sample size (4,013 instances from 249 distinct human gene symbols detected from 250 trials).ResultsOur evaluation using a manually annotated data set showed that the two-stage classifier outperformed the single-stage classifier and achieved the best average accuracy of 83.7% for the eight most frequently mentioned genes when optimized feature sets were used. It also showed better generalizability when we applied the two-stage classifier trained on one set of genes to another independent gene. When a gene-neutral, two-stage classifier was applied to the real-world task of cancer trial annotation, it achieved a highest accuracy of 89.8%, demonstrating the feasibility of developing a gene-neutral classifier for this task.ConclusionsWe presented a machine learning-based approach to detect gene entities and the genetic lesion statuses from clinical trial documents and demonstrated its use in cancer trial annotation. Such methods would be valuable for building information retrieval tools targeting gene-associated clinical trials.

Extracting genetic alteration information for personalized cancer therapy from ClinicalTrials.gov

“…In this project, we adopted a word sense disambiguation system developed in our previous work to determine ambiguous gene mentions in clinical trial documents. 7 We leveraged the previous training samples and modified the existing system to classify candidate gene mentions into 3 categories: "Gene-related" (eg, PTEN gene mutation), "Drug" (eg, no prior EGFR-inhibitor therapy), and "Others" (eg, patient met the criteria).…”

“…In our previous studies, we developed machine learning-based methods to detect genetic status from cancer trials by working with MyCancerGenome.org and IPCT data. 7,8 However, our previous studies were relatively small pilot projects that focused on methodology development and evaluation for intermediate tasks such as word sense disambiguation. In this study, we developed an end-to-end system that takes ClinicalTrials.gov documents as inputs and generates annotations of genetic alterations in all cancer trials.…”

“…It consists of 2 main components: (1) a new cancer treatment trial identification system, and (2) a revised genetic alteration extraction system built on previous methods. 7 By combining the automated approaches and manual review into a single framework, we built a knowledge base of cancer trials labeled with the relevant genetic alteration information. This knowledge base can facilitate cancer trial enrollment and clinical decisions (eg, selecting trials of targeted therapies for which a specific patient may be eligible).…”

Genomics in 2012: challenges and opportunities in the next generation sequencing era

“…GLAD4U ensures computational efficiency through the effective use of existing NCBI resources, which also made it one of the winning applications in the National Library of Medicine (NLM)'s 2011 Software Development Challenge on the Innovative Uses of NLM Information. Wu et al, [26] developed a two-stage classification method for identifying genes and genetic lesion statuses in clinical trial documents. Their system was initially developed and tested on individually annotated genes and later was expanded to all genes in cancer trial documents.…”

The My Cancer Genome clinical trial data model and trial curation workflow

“…Our prior approaches to clinical trial annotation explored both key word search 14 and natural language processing (NLP)-based approaches 10 for automated extraction of biomarker eligibility criteria from clinical trial documents. However, we and others with similar efforts concluded that the level of precision and recall achieved by these methods is relatively low.…”