<i>Can you hear me now</i>? Clinical applications of audio recordings

Kumar, Anish; Jaquenoud, Theo; Becker, Jacqueline; Cho, Dayeon; Mindt, Monica Rivera; Federman, Alex D.; Pandey, Gaurav

doi:10.1101/2022.02.07.22270598

Cited by 3 publications

(6 citation statements)

References 92 publications

(147 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we plan to analyze voice and audio characteristics and extract-related features (such as pause rates, pitch, loudness, acoustic and spectral features, and multiple speakers, such as parent and child) [ 51 , 52 ]. Voice analytics will add a new dimension to PGHD analytics by investigating vocal and environmental audio features (markers) with patient notes and building a multimodal pipeline, such as improving transcription quality, improving sentiment analysis, identifying the environmental factors [ 53 , 54 ], and guiding future data collection protocols.…”

Section: Discussionmentioning

confidence: 99%

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

Sezgın¹,

Hussain²,

Rust³

et al. 2023

JMIR Form Res

View full text Add to dashboard Cite

Background Patient-generated health data (PGHD) captured via smart devices or digital health technologies can reflect an individual health journey. PGHD enables tracking and monitoring of personal health conditions, symptoms, and medications out of the clinic, which is crucial for self-care and shared clinical decisions. In addition to self-reported measures and structured PGHD (eg, self-screening, sensor-based biometric data), free-text and unstructured PGHD (eg, patient care note, medical diary) can provide a broader view of a patient’s journey and health condition. Natural language processing (NLP) is used to process and analyze unstructured data to create meaningful summaries and insights, showing promise to improve the utilization of PGHD. Objective Our aim is to understand and demonstrate the feasibility of an NLP pipeline to extract medication and symptom information from real-world patient and caregiver data. Methods We report a secondary data analysis, using a data set collected from 24 parents of children with special health care needs (CSHCN) who were recruited via a nonrandom sampling approach. Participants used a voice-interactive app for 2 weeks, generating free-text patient notes (audio transcription or text entry). We built an NLP pipeline using a zero-shot approach (adaptive to low-resource settings). We used named entity recognition (NER) and medical ontologies (RXNorm and SNOMED CT [Systematized Nomenclature of Medicine Clinical Terms]) to identify medication and symptoms. Sentence-level dependency parse trees and part-of-speech tags were used to extract additional entity information using the syntactic properties of a note. We assessed the data; evaluated the pipeline with the patient notes; and reported the precision, recall, and F1 scores. Results In total, 87 patient notes are included (audio transcriptions n=78 and text entries n=9) from 24 parents who have at least one CSHCN. The participants were between the ages of 26 and 59 years. The majority were White (n=22, 92%), had more than one child (n=16, 67%), lived in Ohio (n=22, 92%), had mid- or upper-mid household income (n=15, 62.5%), and had higher level education (n=24, 58%). Out of 87 notes, 30 were drug and medication related, and 46 were symptom related. We captured medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (precision >0.65, recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. Conclusions The proposed NLP pipeline was found to be feasible for use with real-world unstructured PGHD to accomplish medication and symptom extraction. Unstructured PGHD can be leveraged to inform clinical decision-making, remote monitoring, and self-care including medical adherence and chronic disease management. With customizable information extraction methods using NER and medical ontologies, NLP models can feasibly extract a broad range of clinical information from unstructured PGHD in low-resource settings (eg, a limited number of patient notes or training data).

show abstract

Section: Discussionmentioning

confidence: 99%

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

Sezgın¹,

Hussain²,

Rust³

et al. 2023

JMIR Form Res

View full text Add to dashboard Cite

show abstract

“…It would be beneficial to comprehensively quantify the sensitivity of our and other such algorithms by establishing a "gold standard" of recordings labeled in terms of speakers, e.g., patients, clinicians and other speakers. Our algorithm also leveraged the placement of the lavalier microphone on the patient, as is commonplace in clinical research settings [8], [17]. Thus, it's performance may not translate for other recording settings, which may need their own specialized algorithms.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…To the best of our knowledge, this is one of the largest clinical audio datasets collected for research purposes [8].…”

Section: A Audio Datasetmentioning

confidence: 99%

“…Our methods relied on the fact that the target speaker, i.e., the patient, was wearing the lavalier microphone that recorded the audio, and was thus expected to sound louder than the clinician and other speakers. While not universal, such recordings are commonplace in clinical research settings [8], [17].…”

Section: Isolation Of Patient Speechmentioning

confidence: 99%

“…Most commonly, the content and cadence of speech can help experts assess patients' mental and cognitive health [3]- [5]. With recent advances in machine learning (ML) algorithms [6], [7], it is becoming possible to automatically identify disorders or disease biomarkers from digitally recorded patient speech [8]. An important step in achieving this goal is to identify and isolate the patient's speech from other speech or noise that are captured in a recording of their conversation with a clinician.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Effective Automated Algorithm to Isolate Patient Speech from Conversations with Clinicians

Jaquenoud

Keene

Shlayan

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

A growing number of algorithms are being developed to automatically identify disorders or disease biomarkers from digitally recorded audio of patient speech. An important step in these analyses is to identify and isolate the patient's speech from that of other speakers or noise that are captured in a recording. However, current algorithms, such as diarization, only label the identified speech segments in terms of non-specific speakers, and do not identify the specific speaker of each segment, e.g., clinician and patient. In this paper, we present a novel algorithm that not only performs diarization on clinical audio, but also identifies the patient among the speakers in the recording and returns an audio file containing only the patient's speech. Our algorithm first uses pretrained diarization algorithms to separate the input audio into different tracks according to non-specific speaker labels. Next, in a novel step not conducted in other diarization tools, the algorithm uses the average loudness (quantified as power) of each audio track to identify the patient, and return the audio track containing only their speech. Using a practical expert-based evaluation methodology and a large dataset of clinical audio recordings, we found that the best implementation of our algorithm achieved near-perfect accuracy on two validation sets. Thus, our algorithm can be used for effectively identifying and isolating patient speech, which can be used in downstream expert and/or data-driven analyses.

show abstract

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data (Preprint)

Sezgın¹,

Hussain²,

Rust³

et al. 2022

Preprint

View full text Add to dashboard Cite

BACKGROUND Patient generated health data (PGHD) is important to understand a patient's health condition out of the clinic and communicate timely. It plays a supplementary role in preventive medicine, self-care, remote patient monitoring and patient-reported outcomes. In addition to standard measures and structured data (sensors, biometric data), unstructured PGHD (free-text data) can provide a broader view of a patient's journey and health condition. OBJECTIVE Our aim to evaluate feasibility of an NLP pipeline with real-world patient and caregiver data. METHODS Using a zero-shot approach which is adaptive to low-resource settings, the NLP pipeline is built upon named entity recognition (NER) to identify medication and symptoms using the standard ontologies (RXNorm and SNOMED CT). Sentence level dependency parse trees and part-of-speech tags were included to extract additional entity information using the syntactic properties of a note. We tested the model with the patient notes (text-based or transcribed audio notes) collected from 24 parents of children with special healthcare needs during a 2-weeks use of a voice-interactive app. In total, 87 patient notes were used. RESULTS In total, 87 patient notes are included (voice entry transcriptions (n=78) and text entries (n=9)). 30 of the notes are drug and medication-related, and 57 of the notes are symptom-related. We are able to capture medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (Precision >0.65, Recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. CONCLUSIONS Unstructured PGHD provides a new and untapped layer in patient health records which can inform decision making and support remote monitoring and self-care. In this paper, we share the new research findings and preliminary results for a customizable information extraction (IE) NLP model focused on extracting a broad-range of clinical information from unstructured PGHD in low-resource settings, especially as it relates to chronic disease management.

show abstract

Can you hear me now? Clinical applications of audio recordings

Cited by 3 publications

References 92 publications

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

An Effective Automated Algorithm to Isolate Patient Speech from Conversations with Clinicians

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data (Preprint)

Contact Info

Product

Resources

About