The Effectiveness of Facial Expression Recognition in Detecting Emotional Responses to Sound Interventions in Older Adults With Dementia

Liu, Ying; Wang, Zixuan; Yu, Gao

doi:10.3389/fpsyg.2021.707809

Cited by 8 publications

(8 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The emotional responses revealed by the objective facial expression analysis showed the same trend as the participants’ subjective assessment, with happy emotions having the largest values in both assessments. This result suggests that the facial expression analysis system can successfully detect the expressions that emerge in people with dementia, which is consistent with a previous study [ 40 ].…”

Section: Discussionsupporting

confidence: 92%

“…Second, the generalizability of our findings may be limited, as most of the participants in this study were female. Previous studies report that there are gender differences in emotional responses to some types of sounds [ 56 ], emotion expression [ 57 ], and emotion regulation [ 58 ], although gender and facial expression have been reported to have no significant correlation [ 40 ]. Further research with male participants will be needed to reveal possible gender differences in responses toward social robots.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study

Otaka,

Osawa,

Kato

et al. 2024

JMIR Aging

View full text Add to dashboard Cite

Background Interventions and care that can evoke positive emotions and reduce apathy or agitation are important for people with dementia. In recent years, socially assistive robots used for better dementia care have been found to be feasible. However, the immediate responses of people with dementia when they are given multiple sensory modalities from socially assistive robots have not yet been sufficiently elucidated. Objective This study aimed to quantitatively examine the immediate emotional responses of people with dementia to stimuli presented by socially assistive robots using facial expression analysis in order to determine whether they elicited positive emotions. Methods This pilot study adopted a single-arm interventional design. Socially assistive robots were presented to nursing home residents in a three-step procedure: (1) the robot was placed in front of participants (visual stimulus), (2) the robot was manipulated to produce sound (visual and auditory stimuli), and (3) participants held the robot in their hands (visual, auditory, and tactile stimuli). Expression intensity values for “happy,” “sad,” “angry,” “surprised,” “scared,” and “disgusted” were calculated continuously using facial expression analysis with FaceReader. Additionally, self-reported feelings were assessed using a 5-point Likert scale. In addition to the comparison between the subjective and objective emotional assessments, expression intensity values were compared across the aforementioned 3 stimuli patterns within each session. Finally, the expression intensity value for “happy” was compared between the different types of robots. Results A total of 29 participants (mean age 88.7, SD 6.2 years; n=27 female; Japanese version of Mini-Mental State Examination mean score 18.2, SD 5.1) were recruited. The expression intensity value for “happy” was the largest in both the subjective and objective assessments and increased significantly when all sensory modalities (visual, auditory, and tactile) were presented (median expression intensity 0.21, IQR 0.09-0.35) compared to the other 2 patterns (visual alone: median expression intensity 0.10, IQR 0.03-0.22; P<.001; visual and auditory: median expression intensity 0.10, IQR 0.04-0.23; P<.001). The comparison of different types of robots revealed a significant increase when all stimuli were presented by doll-type and animal-type robots, but not humanoid-type robots. Conclusions By quantifying the emotional responses of people with dementia, this study highlighted that socially assistive robots may be more effective in eliciting positive emotions when multiple sensory stimuli, including tactile stimuli, are involved. More studies, including randomized controlled trials, are required to further explore the effectiveness of using socially assistive robots in dementia care.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Discussionmentioning

confidence: 99%

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study

Otaka,

Osawa,

Kato

et al. 2024

JMIR Aging

View full text Add to dashboard Cite

show abstract

“…However, the facial expressions were recorded by nearly 30 different caregivers, so individual bias was not likely. In recent years, novel artificial-intelligence-based emotional analysis tools that capture facial expressions have been validated in older people [ 35 ], and such tools could be applied in the future to provide a more quantitative analysis of psychological effects in a large scale study.…”

Section: Discussionmentioning

confidence: 99%

A rise-assisting robot extends life space and improves facial expressions of nursing home residents

Kato

Yoshimi

Aimoto

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

Background The introduction of nursing care-support devices using robotic technology is expected to reduce the task burden in long-term care facilities. Objective To investigate the use of the rise-assisting robot, Resyone, in extending and improving the life space of nursing home residents with severe care needs. Methods We performed a feasibility study in which Resyone was used to facilitate visits to additional sites in and around the nursing home as part of the care package of three residents. Two weeks before and four weeks after implementation of the new arrangements, the 30 caregivers involved were asked to record transfer times and destinations, while also checking the residents’ facial expressions. Results Before implementation, participants had limited life spaces, but afterwards they regularly visited additional destinations including the garden, home entrance and corridors, which previously they had not visited frequently. The residents' facial expressions became more positive and less negative. This study demonstrates that Resyone can enrich care activities in severely disabled individuals. Conclusion These findings suggest that the sustainable use of Resyone would improve the quality of care at care facilities. Moreover, the extension of otherwise limited life space has the potential to improve care receivers’ quality of life. Trial registration UMIN Clinical Trials Registry No. UMIN000039204 (20/01/2020); retrospectively registered; interventional study; parallel, non-randomized, single blinded. URL of trial registry records: https://upload.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000044709.

show abstract

“…They affirmed that FaceReader can correspondingly recognise different emotions changing related to different sound stimuli. They found music to be the most effective sound that can induce much more emotion than other sounds, i.e., birdsong and streams [26]. Busso et al explored the interrelation between facial gestures and speech, highlighting that this relationship varies with the linguistic and emotional content of the message.…”

Section: Correlation Between Sound and Emotionsmentioning

confidence: 99%

Facial Expression Recognition for Measuring Jurors’ Attention in Acoustic Jury Tests

Jamali,

Generosi,

Villafan

et al. 2024

Sensors

View full text Add to dashboard Cite

The perception of sound greatly impacts users’ emotional states, expectations, affective relationships with products, and purchase decisions. Consequently, assessing the perceived quality of sounds through jury testing is crucial in product design. However, the subjective nature of jurors’ responses may limit the accuracy and reliability of jury test outcomes. This research explores the utility of facial expression analysis in jury testing to enhance response reliability and mitigate subjectivity. Some quantitative indicators allow the research hypothesis to be validated, such as the correlation between jurors’ emotional responses and valence values, the accuracy of jury tests, and the disparities between jurors’ questionnaire responses and the emotions measured by FER (facial expression recognition). Specifically, analysis of attention levels during different statuses reveals a discernible decrease in attention levels, with 70 percent of jurors exhibiting reduced attention levels in the ‘distracted’ state and 62 percent in the ‘heavy-eyed’ state. On the other hand, regression analysis shows that the correlation between jurors’ valence and their choices in the jury test increases when considering the data where the jurors are attentive. The correlation highlights the potential of facial expression analysis as a reliable tool for assessing juror engagement. The findings suggest that integrating facial expression recognition can enhance the accuracy of jury testing in product design by providing a more dependable assessment of user responses and deeper insights into participants’ reactions to auditory stimuli.

show abstract

The Effectiveness of Facial Expression Recognition in Detecting Emotional Responses to Sound Interventions in Older Adults With Dementia

Cited by 8 publications

References 59 publications

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study

A rise-assisting robot extends life space and improves facial expressions of nursing home residents

Facial Expression Recognition for Measuring Jurors’ Attention in Acoustic Jury Tests

Contact Info

Product

Resources

About