Automatic Detection of Everyday Social Behaviours and Environments from Verbatim Transcripts of Daily Conversations

Yordanova, Kristina; Demiray, Burcu; Mehl, Matthias R.; Martin, Mike

doi:10.1109/percom.2019.8767403

Cited by 15 publications

(36 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we mentioned at the outset, it is our hope that, as researchers construct their experiences with the method, the collective knowledge around its use will broaden and evolve and will ultimately render what we document here incomplete or outdated. Further, it is likely that computational advances in behavioral signal processing (Narayanan & Georgiou, 2013) will, in the future, render the coding process more efficient (e.g., through automatic identification of sound files that are silent or contain speech) and that some aspects of EAR coding may ultimately become automated (Dubey, Mehl, & Mankodiya, 2016;Yordanova, Demiray, Mehl & Martin, 2019). Until then, we hope that the documentation of what we have learned over two A PRACTICAL GUIDE TO CODING AND PROCESSING EAR DATA 33 decades or EAR research will help save researchers time and frustration and make a highly labor intensive method more accessible.…”

Section: Resultsmentioning

confidence: 99%

Best practices for Electronically Activated Recorder (EAR) research: A practical guide to coding and processing EAR data

et al. 2020

Self Cite

View full text Add to dashboard Cite

Since its introduction in 2001, the Electronically Activated Recorder (EAR) method has become an established and broadly used tool for the naturalistic observation of daily social behavior in clinical, health, personality, and social science research. Previous treatments of the method have focused primarily on its measurement approach (relative to other ecological assessment methods), research design considerations (e.g., sampling schemes, privacy considerations), and the properties of its data (i.e. reliability, validity, added measurement value). However, the evolved procedures and practices around arguably one of the most critical parts of EAR research-the coding process that converts the sampled raw ambient sounds into quantitative behavioral data for statistical analysis-have so far been largely communicated informally between EAR researchers. This article documents the "best practices" for processing EAR data that have been tested and refined in our research over the years. Our aim is to provide practical information on important topics such as the development of a coding system, the training and supervision of EAR coders, EAR data preparation and database optimization, the troubleshooting of common coding challenges, and coding considerations specific to diverse populations.

show abstract

Section: Resultsmentioning

confidence: 99%

Best practices for Electronically Activated Recorder (EAR) research: A practical guide to coding and processing EAR data

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are recent sources of unstructured data that are gaining importance in health care applications and may benefit dementia prediction. For example, transcripts of daily conversation are used in [26] to identify social behavior (e.g., giving advice, receiving advice, conversation). Another study [27] uses speech samples and MRI data from 32 semantic dementia patients and 10 HC.…”

Section: Data Sourcesmentioning

confidence: 99%

“…A preliminary step in processing free text consists of syntactic transformations such as removing punctuation and stop words [26]. This is followed by a process which identifies features that are representative of the text.…”

Section: Feature Engineeringmentioning

confidence: 99%

“…A mapping technique that can further reduce the feature space and enhance the entropy of each feature consists of grouping a set of related keywords under a single concept. For example, using latent Dirichlet allocation (LDA), keywords are grouped by concept in [11] and [26]. Word2vec [28] and autoencoders have also been used for keywordto-concept mapping in several health care applications [5].…”

Section: Feature Engineeringmentioning

confidence: 99%

See 1 more Smart Citation

Predicting dementia with routine care EMR data

Miled

Haas

Black

et al. 2020

Artificial Intelligence in Medicine

View full text Add to dashboard Cite

Our aim is to develop a machine learning (ML) model that can predict dementia in a general patient population from multiple health care institutions one year and three years prior to the onset of the disease without any additional monitoring or screening. The purpose of the model is to automate the cost-effective, non-invasive, digital pre-screening of patients at risk for dementia. Towards this purpose, routine care data, which is widely available through Electronic Medical Record (EMR) systems is used as a data source. These data embody a rich knowledge and make related medical applications easy to deploy at scale in a cost-effective manner. Specifically, the model is trained by using structured and unstructured data from three EMR data sets: diagnosis,

show abstract

“…Yordanova et al [ 24 ] were the first to investigate the applicability of NLP and machine learning methodologies on data from a naturalistic observation study by Demiray et al [ 6 ]; they introduced an NLP pipeline and machine learning routines to automatically code the social behaviors and interactions (eg, talking to a partner or daughter/son, giving advice, receive support, etc) in the transcripts of recorded conversations. As coding is a manual process that involves much effort and time, their results showed that the use of NLP and machine learning automation on transcripts of recorded conversations enabled reliable coding of social behaviors and interactions, reducing effort and time.…”

Section: Introductionmentioning

confidence: 99%

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

Ferrario¹,

Demiray²,

Yordanova³

et al. 2020

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

Background Reminiscence is the act of thinking or talking about personal experiences that occurred in the past. It is a central task of old age that is essential for healthy aging, and it serves multiple functions, such as decision-making and introspection, transmitting life lessons, and bonding with others. The study of social reminiscence behavior in everyday life can be used to generate data and detect reminiscence from general conversations. Objective The aims of this original paper are to (1) preprocess coded transcripts of conversations in German of older adults with natural language processing (NLP), and (2) implement and evaluate learning strategies using different NLP features and machine learning algorithms to detect reminiscence in a corpus of transcripts. Methods The methods in this study comprise (1) collecting and coding of transcripts of older adults’ conversations in German, (2) preprocessing transcripts to generate NLP features (bag-of-words models, part-of-speech tags, pretrained German word embeddings), and (3) training machine learning models to detect reminiscence using random forests, support vector machines, and adaptive and extreme gradient boosting algorithms. The data set comprises 2214 transcripts, including 109 transcripts with reminiscence. Due to class imbalance in the data, we introduced three learning strategies: (1) class-weighted learning, (2) a meta-classifier consisting of a voting ensemble, and (3) data augmentation with the Synthetic Minority Oversampling Technique (SMOTE) algorithm. For each learning strategy, we performed cross-validation on a random sample of the training data set of transcripts. We computed the area under the curve (AUC), the average precision (AP), precision, recall, as well as F1 score and specificity measures on the test data, for all combinations of NLP features, algorithms, and learning strategies. Results Class-weighted support vector machines on bag-of-words features outperformed all other classifiers (AUC=0.91, AP=0.56, precision=0.5, recall=0.45, F1=0.48, specificity=0.98), followed by support vector machines on SMOTE-augmented data and word embeddings features (AUC=0.89, AP=0.54, precision=0.35, recall=0.59, F1=0.44, specificity=0.94). For the meta-classifier strategy, adaptive and extreme gradient boosting algorithms trained on word embeddings and bag-of-words outperformed all other classifiers and NLP features; however, the performance of the meta-classifier learning strategy was lower compared to other strategies, with highly imbalanced precision-recall trade-offs. Conclusions This study provides evidence of the applicability of NLP and machine learning pipelines for the automated detection of reminiscence in older adults’ everyday conversations in German. The methods and findings of this study could be relevant for designing unobtrusive computer systems for the real-time detection of social reminiscence in the everyday life of older adults and classifying their functions. With further improvements, these systems could be deployed in health interventions aimed at improving older adults’ well-being by promoting self-reflection and suggesting coping strategies to be used in the case of dysfunctional reminiscence cases, which can undermine physical and mental health.

show abstract

Automatic Detection of Everyday Social Behaviours and Environments from Verbatim Transcripts of Daily Conversations

Cited by 15 publications

References 29 publications

Best practices for Electronically Activated Recorder (EAR) research: A practical guide to coding and processing EAR data

Best practices for Electronically Activated Recorder (EAR) research: A practical guide to coding and processing EAR data

Predicting dementia with routine care EMR data

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

Contact Info

Product

Resources

About