Smartphone-based Mental State Estimation: A Survey from a Machine Learning Perspective

Fukazawa, Yusuke; Yamamoto, Naoki; Hamatani, Takashi; Ochiai, Keiichi; Uchiyama, Akira; Ohta, K.

doi:10.2197/ipsjjip.28.16

Cited by 9 publications

(4 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Relations between data collected from smartphones and the depressive and manic affective symptoms confirmed in this study demonstrate promising potential for both early detection of affective states and the prediction of phase changes. This is commonly formulated as either classification or regression task [49]. Recent papers [34,50] show that statistical and machine learning approaches can be complemented by process monitoring with control charts.…”

Section: Principal Findingsmentioning

confidence: 99%

Behavioral and Self-reported Data Collected From Smartphones for the Assessment of Depressive and Manic Symptoms in Patients With Bipolar Disorder: Prospective Observational Study

Dominiak¹,

Kaczmarek-Majer²,

Antosik-Wójcińska³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

Background Smartphones allow for real-time monitoring of patients’ behavioral activities in a naturalistic setting. These data are suggested as markers for the mental state of patients with bipolar disorder (BD). Objective We assessed the relations between data collected from smartphones and the clinically rated depressive and manic symptoms together with the corresponding affective states in patients with BD. Methods BDmon, a dedicated mobile app, was developed and installed on patients’ smartphones to automatically collect the statistics about their phone calls and text messages as well as their self-assessments of sleep and mood. The final sample for the numerical analyses consisted of 51 eligible patients who participated in at least two psychiatric assessments and used the BDmon app (mean participation time, 208 [SD 132] days). In total, 196 psychiatric assessments were performed using the Hamilton Depression Rating Scale and the Young Mania Rating Scale. Generalized linear mixed-effects models were applied to quantify the strength of the relation between the daily statistics on the behavioral data collected automatically from smartphones and the affective symptoms and mood states in patients with BD. Results Objective behavioral data collected from smartphones were found to be related with the BD states as follows: (1) depressed patients tended to make phone calls less frequently than euthymic patients (β=−.064, P=.01); (2) the number of incoming answered calls during depression was lower than that during euthymia (β=−.15, P=.01) and, concurrently, missed incoming calls were more frequent and increased as depressive symptoms intensified (β=4.431, P<.001; β=4.861, P<.001, respectively); (3) the fraction of outgoing calls was higher in manic states (β=2.73, P=.03); (4) the fraction of missed calls was higher in manic/mixed states as compared to that in the euthymic state (β=3.53, P=.01) and positively correlated to the severity of symptoms (β=2.991, P=.02); (5) the variability of the duration of the outgoing calls was higher in manic/mixed states (β=.0012, P=.045) and positively correlated to the severity of symptoms (β=.0017, P=.02); and (6) the number and length of the sent text messages was higher in manic/mixed states as compared to that in the euthymic state (β=.031, P=.01; β=.015, P=.01; respectively) and positively correlated to the severity of manic symptoms (β=.116, P<.001; β=.022, P<.001; respectively). We also observed that self-assessment of mood was lower in depressive (β=−1.452, P<.001) and higher in manic states (β=.509, P<.001). Conclusions Smartphone-based behavioral parameters are valid markers for assessing the severity of affective symptoms and discriminating between mood states in patients with BD. This technology opens a way toward early detection of worsening of the mental state and thereby increases the patient’s chance of improving in the course of the illness.

show abstract

Section: Principal Findingsmentioning

confidence: 99%

Behavioral and Self-reported Data Collected From Smartphones for the Assessment of Depressive and Manic Symptoms in Patients With Bipolar Disorder: Prospective Observational Study

Dominiak¹,

Kaczmarek-Majer²,

Antosik-Wójcińska³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…The reasons for using activity and biometric data for this are that other studies have examined the relationship between physical ability and QoL and physical activity levels and many studies have been conducted using biometric information, which can be acquired by wearable devices, to detect stress [ 15 , 16 , 17 , 18 , 19 ]. The physical approach is examining passive QoL and health measures from a machine learning perspective using “physical activity data” such as smartphone acceleration and tilt [ 20 , 21 ]. There is also a positive correlation between physical activity and QoL [ 15 ].…”

Section: Introductionmentioning

confidence: 99%

Stress Estimation Using Biometric and Activity Indicators to Improve QoL of the Elderly

Matsumoto

Matsui

Suwa

et al. 2023

Sensors

View full text Add to dashboard Cite

It is essential to estimate the stress state of the elderly to improve their QoL. Stress states change every day and hour, depending on the activities performed and the duration/intensity. However, most existing studies estimate stress states using only biometric information or specific activities (e.g., sleep duration, exercise duration/amount, etc.) as explanatory variables and do not consider all daily living activities. It is necessary to link various daily living activities and biometric information in order to estimate the stress state more accurately. Specifically, we construct a stress estimation model using machine learning with the answers to a stress status questionnaire obtained every morning and evening as the ground truth and the biometric data during each of the performed activities and the new proposed indicator including biological and activity perspectives as the features. We used the following methods: Baseline Method 1, in which the RRI variance and Lorenz plot area for 4 h after waking and 24 h before the questionnaire were used as features; Baseline Method 2, in which sleep time was added as a feature to Baseline Method 1; the proposed method, in which the Lorenz plot area per activity and total time per activity were added. We compared the results with the proposed method, which added the new indicators as the features. The results of the evaluation experiments using the one-month data collected from five elderly households showed that the proposed method had an average estimation accuracy of 59%, 7% better than Baseline Method 1 (52%) and 4% better than Baseline Method 2 (55%).

show abstract

“…The majority of the approaches rely on various characteristics like visual manifestations, acoustic and linguistic communication, smartphone usage activities, social media content, physiological cues, etc. [ 14 , 21 , 24 , 34 , 45 , 47 , 48 , 56 ]. Few approaches combine different modalities like visual with speech [ 38 , 42 ].…”

Section: Introductionmentioning

confidence: 99%

A novel multi-modal depression detection approach based on mobile crowd sensing and task-based mechanisms

Thati

Dhadwal

Kumar

et al. 2022

Multimed Tools Appl

View full text Add to dashboard Cite

Depression has become a global concern, and COVID-19 also has caused a big surge in its incidence. Broadly, there are two primary methods of detecting depression: Task-based and Mobile Crowd Sensing (MCS) based methods. These two approaches, when integrated, can complement each other. This paper proposes a novel approach for depression detection that combines real-time MCS and task-based mechanisms. We aim to design an end-to-end machine learning pipeline, which involves multimodal data collection, feature extraction, feature selection, fusion, and classification to distinguish between depressed and non-depressed subjects. For this purpose, we created a real-world dataset of depressed and non-depressed subjects. We experimented with: various features from multi-modalities, feature selection techniques, fused features, and machine learning classifiers such as Logistic Regression, Support Vector Machines (SVM), etc. for classification. Our findings suggest that combining features from multiple modalities perform better than any single data modality, and the best classification accuracy is achieved when features from all three data modalities are fused. Feature selection method based on Pearson’s correlation coefficients improved the accuracy in comparison with other methods. Also, SVM yielded the best accuracy of 86%. Our proposed approach was also applied on benchmarking dataset, and results demonstrated that the multimodal approach is advantageous in performance with state-of-the-art depression recognition techniques.

show abstract

Smartphone-based Mental State Estimation: A Survey from a Machine Learning Perspective

Cited by 9 publications

References 80 publications

Behavioral and Self-reported Data Collected From Smartphones for the Assessment of Depressive and Manic Symptoms in Patients With Bipolar Disorder: Prospective Observational Study

Behavioral and Self-reported Data Collected From Smartphones for the Assessment of Depressive and Manic Symptoms in Patients With Bipolar Disorder: Prospective Observational Study

Stress Estimation Using Biometric and Activity Indicators to Improve QoL of the Elderly

A novel multi-modal depression detection approach based on mobile crowd sensing and task-based mechanisms

Contact Info

Product

Resources

About