Abstract:Objectives (1) To develop an automated eligibility screening (ES) approach for clinical trials in an urban tertiary care pediatric emergency department (ED); (2) to assess the effectiveness of natural language processing (NLP), information extraction (IE), and machine learning (ML) techniques on real-world clinical data and trials.Data and methods We collected eligibility criteria for 13 randomly selected, disease-specific clinical trials actively enrolling patients between January 1, 2010 and August 31, 2012.… Show more
“…poor structure, abundant shorthand, domain-specific vocabularies) that make the application of NLP challenging. Current NLP systems have proven to be useful for certain activities and have, for example, reduced the time required for screening candidates for clinical trial eligibility [1] and identified potential adverse drug reactions [2]. There are, however, other challenges in the field, such as identification of temporal associations, evaluation of context-dependent text, and concept normalization to particular terminologies, that remain open [3–7].…”
We followed a systematic approach based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses to identify existing clinical natural language processing (NLP) systems that generate structured information from unstructured free text. Seven literature databases were searched with a query combining the concepts of natural language processing and structured data capture. Two reviewers screened all records for relevance during two screening phases, and information about clinical NLP systems was collected from the final set of papers. A total of 7149 records (after removing duplicates) were retrieved and screened, and 86 were determined to fit the review criteria. These papers contained information about 71 different clinical NLP systems, which were then analyzed. The NLP systems address a wide variety of important clinical and research tasks. Certain tasks are well addressed by the existing systems, while others remain as open challenges that only a small number of systems attempt, such as extraction of temporal information or normalization of concepts to standard terminologies. This review has identified many NLP systems capable of processing clinical free text and generating structured output, and the information collected and evaluated here will be important for prioritizing development of new approaches for clinical NLP.
“…poor structure, abundant shorthand, domain-specific vocabularies) that make the application of NLP challenging. Current NLP systems have proven to be useful for certain activities and have, for example, reduced the time required for screening candidates for clinical trial eligibility [1] and identified potential adverse drug reactions [2]. There are, however, other challenges in the field, such as identification of temporal associations, evaluation of context-dependent text, and concept normalization to particular terminologies, that remain open [3–7].…”
We followed a systematic approach based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses to identify existing clinical natural language processing (NLP) systems that generate structured information from unstructured free text. Seven literature databases were searched with a query combining the concepts of natural language processing and structured data capture. Two reviewers screened all records for relevance during two screening phases, and information about clinical NLP systems was collected from the final set of papers. A total of 7149 records (after removing duplicates) were retrieved and screened, and 86 were determined to fit the review criteria. These papers contained information about 71 different clinical NLP systems, which were then analyzed. The NLP systems address a wide variety of important clinical and research tasks. Certain tasks are well addressed by the existing systems, while others remain as open challenges that only a small number of systems attempt, such as extraction of temporal information or normalization of concepts to standard terminologies. This review has identified many NLP systems capable of processing clinical free text and generating structured output, and the information collected and evaluated here will be important for prioritizing development of new approaches for clinical NLP.
“…In box 1, we preset various ways in which NLP has been used in biomedical research and arguably to promote EBM practice. Applications of natural language processing that may promote evidence-based medicine practice▸ Building an automated problem list in electronic medical records5▸ Automated clinical trial eligibility prescreening6▸ Identifying certain patients, conditions or interventions to be studied in observational studies (cohort or case–control studies)2▸ De-identification of protected health information in clinical narratives7▸ Extracting knowledge or information from published systematic reviews8▸ Facilitation of conducting systematic reviews by automating screening of abstracts or extracting data9▸ Facilitation of updating of systematic reviews10▸ Facilitation of computerised clinical decision support to aid decision-making of healthcare providers at the point of care11…”
Studies using natural language processing (NLP) techniques are increasingly being published. Evidence-based medicine (EBM) users need to learn the basics of NLP to be able to appraise these types of studies. We propose a set of criteria to evaluate the quality of studies that have used NLP, focusing on the methods of sample selection, coding, the gold standard, algorithm training, algorithm testing and measures of accuracy (such as recall and precision). NLP has proven critical for conducting biomedical research and has the potential to improve healthcare practice and facilitate EBM. Stakeholders (healthcare providers and policymakers) interested in using evidence derived from studies that used NLP need to know the basics of NLP and need to be able to appraise this type of study.
“…Unfortunately this dataset is no longer available [21]. Ni et al [22] evaluate a system working on real-world trials with a goal similar to the TREC challenge and retrieve patient encounters relevant to trial. Their system uses a combination of NLP, information extraction and machine learning methods.…”
Clinical trials are essential for determining whether new interventions are effective. In order to determine the eligibility of patients to enroll into these trials, clinical trial coordinators often perform a manual review of clinical notes in the electronic health record of patients. This is a very time-consuming and exhausting task. Efforts in this process can be expedited if these coordinators are directed towards specific parts of the text that are relevant for eligibility determination. In this study, we describe the creation of a dataset that can be used to evaluate automated methods capable of identifying sentences in a note that are relevant for screening a patient's eligibility in clinical trials. Using this dataset, we also present results for four simple methods in natural language processing that can be used to automate this task. We found that this is a challenging task (maximum F-Score=26.25), but it is a promising direction for further research.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
