Coupled HMM-based multimodal fusion for mood disorder detection through elicited audio–visual signals

Yang, Tian; Wu, Chung Hsien; Huang, Kun-Yi; Su, Ming Hsiang

doi:10.1007/s12652-016-0395-y

Cited by 30 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, bimodal emotion recognition can reach an accuracy of 86.85%, an increase of 5% compared with using a single modal of emotion recognition (Song et al 2015;Chuang and Wu 2004;Kessous et al 2010). Furthermore, previous studies have indicated that it is impossible to achieve satisfactory results by recognizing emotions based on a single model for either speech or facial expression (Ma et al 2019;Yang et al 2017). Accordingly, this study applied a bimodal emotion recognition system by using both facial expression recognition (55%) and speech recognition (45%).…”

Section: Methodsmentioning

confidence: 99%

“…The reliability study has accomplished by stratified random sampling 1067 children in Taiwan. Cronbach's a has reached 0.98 and the validity study on Kaiser-Meyer-Olkin (KMO) has reached 0.951 (Wei 2011). It is a 5-level scale comprised four subscales and each subscale is composed of ten emotional competencies.…”

Section: Ecsycmentioning

confidence: 99%

“…However, teachers are too busy to assess children's emotions. Wei (2011) developed a standardized ECSYC comprised of 40 emotional competencies including four subscales of understanding self, understanding others, emotional adjustment, and self-motivation. Can children's emotional competencies be detected by child-computer interaction?…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Development and evaluation of an emotional lexicon system for young children

Wei

2019

Microsyst Technol

View full text Add to dashboard Cite

Traditionally, children's emotion has been assessed by teachers according to observation. We should be able to detect children's emotions using algorithmic techniques. To achieve this goal, it is necessary to develop and evaluate an emotional lexicon based on the standardized test entitled Emotional Competencies Scale for Young Children (ECSYC). The purpose of this study was to establish the criterion-related validity. The methodology of this study was to firstly develop 40 scenarios based on ECSYC. Secondly, we developed the five-level criteria. Thirdly, this study implemented observer training and calculated inter-rater consistency reliability. Fourthly, observers categorized 200 children's replies into five levels. Fifthly, this study ranked the sequence of frequency of each level and completed the emotional lexicon. The findings showed that the Spearman's rho coefficient reached up to .406*. (p = .026), which is significant, indicating that Young Children Emotional Lexicon (YCEL) and ECSYC were significantly correlated. The accuracies of the emotion detection recognizer using a bimodal emotion recognition approach achieved 46.7%, 60.85% and 78.73% for facial expression recognition, speech recognition, and a bimodal emotion recognition, respectively. Findings confirmed that the YCEL is feasible for speech recognition. The bimodal emotion recognition accuracies increased 32.03% and 17.88% compared with using a single modal of facial expression recognition and speech recognition, respectively. It is feasible to automatically detect children's emotional development and bring the norm up to date.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Ecsycmentioning

confidence: 99%

See 1 more Smart Citation

Development and evaluation of an emotional lexicon system for young children

Wei

2019

Microsyst Technol

View full text Add to dashboard Cite

show abstract

“…At approximately the same time McIntyre et al [83] proposed an AU-based approach in the form of Region Units (RUs). Several studies have reported promising results on the application of AUs to automatic depression assessment [84], [85], [86], [87], [88], [89], [90], [91], [92], [93], [94], [95], [96]. Specific facial expressions, have also been examined for depression assessment, in terms of frequency of occurrence, variability, and intensity of a specific expression.…”

Section: Nonverbal Signs For Depression Assessmentmentioning

confidence: 99%

“…The selection criteria used depended greatly on the research question. DSM and HAM-D criteria were used for detection of depression [82], [90], [110], [111], [113], [132] or differentiation from Bipolar Disorder [95]. Others had more specific criteria, i.e., patients recovering from Deep Brain Stimulation of the Subcallosal Cingulate Cortex (DBS-SCC) [133], in order to monitor recovery progress.…”

Section: Reported Datasetsmentioning

confidence: 99%

Automatic Assessment of Depression Based on Visual Cues: A Systematic Review

Pampouchidou

Simos

Marias

et al. 2019

IEEE Trans. Affective Comput.

143

View full text Add to dashboard Cite

Automatic depression assessment based on visual cues is a rapidly growing research domain. The present exhaustive review of existing approaches as reported in over sixty publications during the last ten years focuses on image processing and machine learning algorithms. Visual manifestations of depression, various procedures used for data collection, and existing datasets are summarized. The review outlines methods and algorithms for visual feature extraction, dimensionality reduction, decision methods for classification and regression approaches, as well as different fusion strategies. A quantitative meta-analysis of reported results, relying on performance metrics robust to chance, is included, identifying general trends and key unresolved issues to be considered in future studies of automatic depression assessment utilizing visual cues alone or in combination with vocal or verbal cues.

show abstract

Automated assessment of psychiatric disorders using speech: A systematic review

Low

Bentley

Ghosh

2020

Laryngoscope Investig Oto

290

268

View full text Add to dashboard Cite

Objective There are many barriers to accessing mental health assessments including cost and stigma. Even when individuals receive professional care, assessments are intermittent and may be limited partly due to the episodic nature of psychiatric symptoms. Therefore, machine‐learning technology using speech samples obtained in the clinic or remotely could one day be a biomarker to improve diagnosis and treatment. To date, reviews have only focused on using acoustic features from speech to detect depression and schizophrenia. Here, we present the first systematic review of studies using speech for automated assessments across a broader range of psychiatric disorders. Methods We followed the Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA) guidelines. We included studies from the last 10 years using speech to identify the presence or severity of disorders within the Diagnostic and Statistical Manual of Mental Disorders (DSM‐5). For each study, we describe sample size, clinical evaluation method, speech‐eliciting tasks, machine learning methodology, performance, and other relevant findings. Results 1395 studies were screened of which 127 studies met the inclusion criteria. The majority of studies were on depression, schizophrenia, and bipolar disorder, and the remaining on post‐traumatic stress disorder, anxiety disorders, and eating disorders. 63% of studies built machine learning predictive models, and the remaining 37% performed null‐hypothesis testing only. We provide an online database with our search results and synthesize how acoustic features appear in each disorder. Conclusion Speech processing technology could aid mental health assessments, but there are many obstacles to overcome, especially the need for comprehensive transdiagnostic and longitudinal studies. Given the diverse types of data sets, feature extraction, computational methodologies, and evaluation criteria, we provide guidelines for both acquiring data and building machine learning models with a focus on testing hypotheses, open science, reproducibility, and generalizability. Level of Evidence 3a

show abstract

Coupled HMM-based multimodal fusion for mood disorder detection through elicited audio–visual signals

Cited by 30 publications

References 39 publications

Development and evaluation of an emotional lexicon system for young children

Development and evaluation of an emotional lexicon system for young children

Automatic Assessment of Depression Based on Visual Cues: A Systematic Review

Automated assessment of psychiatric disorders using speech: A systematic review

Contact Info

Product

Resources

About