Prof-Life-Log: Analysis and classification of activities in daily audio streams

Abstract

A new method to analyze and classify daily activities in per sonal audio recordings (PARs) is presented. The method em ploys speech activity detection (SAD) and speaker diarization systems to provide high level semantic segmentation of the audio file. Subsequently. a number of audio. speech and lex ical features are computed in order to characterize events in daily audio streams. The features are selected to capture the statistical properties of conversations, topics and turn-taking behavior, which creates a classification space that allows us to capture the differences in interactions. The proposed sys tem is evaluated on 9 days of data from Prof-Life-Log cor pus, which contains naturalistic long duration audio record ings (each file is collected continuously and lasts between 8-to-16 hours). Our experimental results show that the pro posed system achieves good classification accuracy on a dif ficult real-world dataset.

Recognition of Activities of Daily Living Based on Environmental Analyses Using Audio Fingerprinting Techniques: A Systematic Review

“…The authors of [ 18 ] created a method to analyze and classify daily activities in personal audio recordings (PARs). The method applies: speech activity detection (SAD), speaker diarization systems, and computing the number of audio speech and lexical features [ 18 ]. It uses a TO-Combo-SAD (Threshold Optimized Combo SAD) algorithm for separating speech from noise [ 18 ].…”

“…The method applies: speech activity detection (SAD), speaker diarization systems, and computing the number of audio speech and lexical features [ 18 ]. It uses a TO-Combo-SAD (Threshold Optimized Combo SAD) algorithm for separating speech from noise [ 18 ]. The Principal Component Analysis (PCA) is first applied for dimensionality reduction, and then, the remaining features are supplied to a multi-class support vector machine (SVM) with radial basis function (RBF) kernel for model training and evaluation [ 18 ].…”

“…It uses a TO-Combo-SAD (Threshold Optimized Combo SAD) algorithm for separating speech from noise [ 18 ]. The Principal Component Analysis (PCA) is first applied for dimensionality reduction, and then, the remaining features are supplied to a multi-class support vector machine (SVM) with radial basis function (RBF) kernel for model training and evaluation [ 18 ]. The authors performed recognition of faculty meeting, research meeting, staff meeting, alone time, and conference call, reporting accuracies between 62.78% and 84.25% [ 18 ].…”

Automatic word count estimation from daylong child-centered recordings in various language environments using language-independent syllabification of speech

“…Automatic word count estimation (WCE) from audio recordings can be used to investigate vocal activity and social interaction as a function of recording time and location, such as in personal life logs derived from wearable sensors (Ziaei et al, 2015;Ziaei et al, 2016). WCE is also a highly useful tool in the scientific study of child language acquisition because it can help answer questions such as how much speech children hear in their daily lives in different contexts (e.g., Bergelson et al, 2018a), and how the language input maps to later developmental outcomes in the same children (Weisleder & Fernald, 2013;Ramírez-Esparza et al, 2014).…”

SylNet: An Adaptable End-to-End Syllable Count Estimator for Speech

“…For instance, syllable-based speaking rate estimation algorithms (such as [1], [2]) can be used to analyze prosodic patterns of speakers and speaking styles for linguistic research, or used as additional information for training text-to-speech (TTS) synthesis systems. Syllables are also used for automatic estimation of vocal activity and social interaction from long and noisy audio recordings captured by wearable microphones, as in the personal life log application of Ziaei et al [3] [4]. There is also a need for robust language-independent methods for quantifying the amount of speech in daylong child-centered audio recordings from various language environments [5], [6], as child language researchers use such data to understand language development in children in (e.g., [7], [8]).…”