Text mining occupations from the mental health electronic health record: a natural language processing approach using records from the Clinical Record Interactive Search (CRIS) platform in south London, UK

Abstract

ObjectivesWe set out to develop, evaluate and implement a novel application using natural language processing to text mine occupations from the free-text of psychiatric clinical notes.DesignDevelopment and validation of a natural language processing application using General Architecture for Text Engineering software to extract occupations from de-identified clinical records.Setting and participantsElectronic health records from a large secondary mental healthcare provider in south London, accessed through the Clinical Record Interactive Search platform. The text mining application was run over the free-text fields in the electronic health records of 341 720 patients (all aged ≥16 years).OutcomesPrecision and recall estimates of the application performance; occupation retrieval using the application compared with structured fields; most common patient occupations; and analysis of key sociodemographic and clinical indicators for occupation recording.ResultsUsing the structured fields alone, only 14% of patients had occupation recorded. By implementing the text mining application in addition to the structured fields, occupations were identified in 57% of patients. The application performed on gold-standard human-annotated clinical text at a precision level of 0.79 and recall level of 0.77. The most common patient occupations recorded were ‘student’ and ‘unemployed’. Patients with more service contact were more likely to have an occupation recorded, as were patients of a male gender, older age and those living in areas of lower deprivation.ConclusionThis is the first time a natural language processing application has been used to successfully derive patient-level occupations from the free-text of electronic mental health records, performing with good levels of precision and recall, and applied at scale. This may be used to inform clinical studies relating to the broader social determinants of health using electronic health records.

Machine Learning vs. Rule-Based Methods for Document Classification of Electronic Health Records within Psychiatry - A Systematic Literature Review

“…• The task -we distinguish between approaches that detect things and approaches that make predictions. We found the following detection tasks in this review: recognizing symptoms ( (Hammond et al, 2015), identify clinically relevant new information (Zhang et al, 2017), occupations (Chilman et al, 2021), drug information (Kadra et al, 2015;Hayes et al, 2015) or smoking detection (Wu et al, 2013). Additionally, we found the following prediction tasks in this review: predicting treatment outcome (Perlis et al, 2012;Rumshisky et al, 2016;Colling et al, 2020), patient readmission (Alvarez-Mellado et al, 2019) or patient violence (Menger et al, 2018(Menger et al, , 2019Mosteiro et al, 2021).…”

“…This could mean that the false positives are often minimized. This is supported by various authors' explicitly indicating to optimize precision (Wu et al, 2013;Kadra et al, 2015;Iqbal et al, 2015;Colling et al, 2017;Ohno-Machado and Séroussi, 2019;Chandran et al, 2019;Haerian et al, 2012;Chilman et al, 2021), whereas only a few authors favour recall (Carson et al, 2019;Zhang et al, 2017;Viani et al, 2021). In some cases, the preference for Table 2: An overview of the machine learning methods with a reported F1-, precision-or recall score, sorted by F1-score.…”

Occupation Recognition and Exploitation in Rheumatology Clinical Notes: Employing Deep Learning Models for Named Entity Recognition and Knowledge Discovery in Electronic Health Records

“…The approaches used in these seven articles consisted of rule-based algorithms. The authors in [47] presented a 10-step method for developing and validating an application to text-mining occupations from psychiatric clinical notes. A noteworthy finding was that the percentage of patients with an occupation recorded increased from 14% to 57% when considering unstructured fields.…”

Robust Speech Recognition in Sports Competition Review based on Natural Language Processing

“…No system requires key technology, and the open platform for natural language processing capabilities has a certain system architecture, and various subsystems and functional modules need to be designed [9]. This article needs to study and solve the problems faced by these two aspects [10]. In the literature, we develop the architecture levels in the design scheme layer by layer, study the problems that the system is most likely to encounter, and optimize various aspects of performance according to the problems, so as to design their own solutions to problems [11].…”