Using natural language processing to provide personalized learning opportunities from trainee clinical notes

Denny, Joshua C.; Spickard, Anderson; Speltz, Peter; Porier, Renee; Rosenstiel, Donna; Powers, James S.

doi:10.1016/j.jbi.2015.06.004

Cited by 25 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It includes a set of rules that take into account the Bayesian scores of each detected section, as well as other kinds of inputs like the typical section length, according to the corpus. This proposal demonstrated very good results for section identification in history and physical examination texts and has been re-used in other studies [11, 12, 34].…”

Section: Resultsmentioning

confidence: 81%

See 1 more Smart Citation

Current approaches to identify sections within clinical narratives from electronic health records: a systematic review

Pomares-Quimbaya

Kreuzthaler

Schulz

2019

BMC Med Res Methodol

View full text Add to dashboard Cite

Background The identification of sections in narrative content of Electronic Health Records (EHR) has demonstrated to improve the performance of clinical extraction tasks; however, there is not yet a shared understanding of the concept and its existing methods. The objective is to report the results of a systematic review concerning approaches aimed at identifying sections in narrative content of EHR, using both automatic or semi-automatic methods. Methods This review includes articles from the databases: SCOPUS, Web of Science and PubMed (from January 1994 to September 2018). The selection of studies was done using predefined eligibility criteria and applying the PRISMA recommendations. Search criteria were elaborated by using an iterative and collaborative keyword enrichment. Results Following the eligibility criteria, 39 studies were selected for analysis. The section identification approaches proposed by these studies vary greatly depending on the kind of narrative, the type of section, and the application. We observed that 57% of them proposed formal methods for identifying sections and 43% adapted a previously created method. Seventy-eight percent were intended for English texts and 41% for discharge summaries. Studies that are able to identify explicit (with headings) and implicit sections correspond to 46%. Regarding the level of granularity, 54% of the studies are able to identify sections, but not subsections. From the technical point of view, the methods can be classified into rule-based methods (59%), machine learning methods (22%) and a combination of both (19%). Hybrid methods showed better results than those relying on pure machine learning approaches, but lower than rule-based methods; however, their scope was more ambitious than the latter ones. Despite all the promising performance results, very few studies reported tests under a formal setup. Almost all the studies relied on custom dictionaries; however, they used them in conjunction with a controlled terminology, most commonly the UMLSⓇ metathesaurus. Conclusions Identification of sections in EHR narratives is gaining popularity for improving clinical extraction projects. This study enabled the community working on clinical NLP to gain a formal analysis of this task, including the most successful ways to perform it.

show abstract

Section: Resultsmentioning

confidence: 81%

“…Finally, section identification had been used for education and quality assurance. Denny et al [11] e.g., used it in an automated education advisor system that analyzed students’ notes for relevance, regarding two geriatric competencies. The system identified UMLSⓇ concepts in sections of student’s notes.…”

Section: Resultsmentioning

confidence: 99%

Current approaches to identify sections within clinical narratives from electronic health records: a systematic review

Pomares-Quimbaya

Kreuzthaler

Schulz

2019

BMC Med Res Methodol

View full text Add to dashboard Cite

show abstract

“…Our abbreviated text string search is a basic form of natural language processing, which has been previously used to rapidly identify ACP in medical student notes with a high positive predictive value, but only moderate sensitivity. 20 Our abbreviated text string search can form the basis for future natural language processing interventions with improved sensitivity for retrieving ACP documentation.…”

Section: Figmentioning

confidence: 99%

Comparison of Methods To Identify Advance Care Planning in Patients with Severe Chronic Obstructive Pulmonary Disease Exacerbation

Stephens

Wiener

Ieong

2018

Journal of Palliative Medicine

View full text Add to dashboard Cite

show abstract

“…; for example, Denny at al. describe the use of NLP to identify competencies from students' clinical notes [7] and Spickard et al extracted and cataloged concepts from students' clinical notes to track their progress [8].…”

Section: Introductionmentioning

confidence: 99%

Automatic analysis of summary statements in virtual patients - a pilot study evaluating a machine learning approach

Hege

Kiesewetter

Adler³

2020

BMC Med Educ

View full text Add to dashboard Cite

Background The ability to compose a concise summary statement about a patient is a good indicator for the clinical reasoning abilities of healthcare students. To assess such summary statements manually a rubric based on five categories - use of semantic qualifiers, narrowing, transformation, accuracy, and global rating has been published. Our aim was to explore whether computer-based methods can be applied to automatically assess summary statements composed by learners in virtual patient scenarios based on the available rubric in real-time to serve as a basis for immediate feedback to learners. Methods We randomly selected 125 summary statements in German and English composed by learners in five different virtual patient scenarios. Then we manually rated these statements based on the rubric plus an additional category for the use of the virtual patients’ name. We implemented a natural language processing approach in combination with our own algorithm to automatically assess 125 randomly selected summary statements and compared the results of the manual and automatic rating in each category. Results We found a moderate agreement of the manual and automatic rating in most of the categories. However, some further analysis and development is needed, especially for a more reliable assessment of the factual accuracy and the identification of patient names in the German statements. Conclusions Despite some areas of improvement we believe that our results justify a careful display of the computer-calculated assessment scores as feedback to the learners. It will be important to emphasize that the rating is an approximation and give learners the possibility to complain about supposedly incorrect assessments, which will also help us to further improve the rating algorithms.

show abstract

Using natural language processing to provide personalized learning opportunities from trainee clinical notes

Cited by 25 publications

References 41 publications

Current approaches to identify sections within clinical narratives from electronic health records: a systematic review

Current approaches to identify sections within clinical narratives from electronic health records: a systematic review

Comparison of Methods To Identify Advance Care Planning in Patients with Severe Chronic Obstructive Pulmonary Disease Exacerbation

Automatic analysis of summary statements in virtual patients - a pilot study evaluating a machine learning approach

Contact Info

Product

Resources

About