Can a humanoid social robot stimulate the interactivity of cognitively impaired elderly? A thorough study based on computer vision methods

Tulsulkar, Gauri; Mishra, Nidhi; Thalmann, Nadia Magnenat; Lim, Hwee Er; Lee, Mei Ping; Cheng, Siok Khoong

doi:10.1007/s00371-021-02242-y

Cited by 29 publications

(51 citation statements)

References 93 publications

(84 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The overall findings were mixed; some studies suggested therapeutic effects of the robots, whereas others were neutral or inconclusive. The most commonly reported benefits of the social robots were improved mood and emotional states [ 33 , 41 , 42 , 44 , 49 , 52 , 55 , 72 , 74 , 76 , 78 , 80 , 81 , 84 , 85 , 87 , 92 - 94 ] and an increase in social interaction between the residents and other human interactors [ 31 , 36 , 41 , 42 , 52 , 55 , 58 , 59 , 63 , 69 , 72 , 75 , 81 , 82 , 95 - 98 ], including caregivers [ 69 ] and preschoolers [ 59 ]. Other commonly reported benefits of the robots were reduced loneliness [ 36 , 92 , 99 ], evoked positive memories of pets [ 59 , 65 , 85 , 96 , 100 ], improved quality of life [ 53 , 79 ] and well-being [ 65 ], reduced pain and pain medication use [ 85 , 86 ], cognitive stimulation [ 31 , 59 , 78 ], and improved behavior [ 49 , 55 , 80 ].…”

Section: Resultsmentioning

confidence: 99%

“…One study illuminated safety concerns regarding relatives responding to emergencies via a telepresence robot instead of caregivers [ 64 ]. Barriers to use included technical difficulties [ 38 , 45 , 51 , 54 , 57 , 64 , 77 , 84 , 95 ], difficulty hearing [ 64 , 85 , 88 , 102 ] and seeing [ 48 ] the robots, and physical limitations [ 48 , 51 , 85 , 96 , 100 ]. However, other studies found the robots easy to use despite cognitive impairments and technological illiteracy [ 45 , 57 , 67 , 68 , 102 ].…”

Section: Resultsmentioning

confidence: 99%

“…The results were also mixed with respect to how participants’ age and level of cognitive decline affected robot-based interventions’ efficacy. Some studies found better results with younger participants [ 90 ] and those with milder cognitive decline [ 88 , 90 ], whereas others found better results with older participants [ 53 ] and those with more advanced cognitive decline [ 53 , 70 , 79 , 84 ]. Another potential factor that could have influenced the results of the studies was whether the robots spoke the participants’ native languages.…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Robots in Assisted Living Facilities: Scoping Review

Trainum¹,

Tunis²,

Xie³

et al. 2023

JMIR Aging

View full text Add to dashboard Cite

Background Various technological interventions have been proposed and studied to address the growing demand for care of residents in assisted living facilities, in which a preexisting shortage of professional caregivers has been exacerbated by the COVID-19 pandemic. Care robots are one such intervention with the potential to improve both the care of older adults and the work life of their professional caregivers. However, concerns about efficacy, ethics, and best practices in the applications of robotic technologies in care settings remain. Objective This scoping review aimed to examine the literature on robots used in assisted living facilities and identify gaps in the literature to guide future research. Methods On February 12, 2022, following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) protocol, we searched PubMed, CINAHL Plus with Full Text, PsycINFO, IEEE Xplore digital library, and ACM Digital Library using predetermined search terms. Publications were included if they were written in English and focused on the use of robotics in assisted living facilities. Publications were excluded if they did not provide peer-reviewed empirical data, focused on user needs, or developed an instrument to study human-robot interaction. The study findings were then summarized, coded, and analyzed using the Patterns, Advances, Gaps, Evidence for practice, and Research recommendations framework. Results The final sample included 73 publications from 69 unique studies on the use of robots in assisted living facilities. The findings of studies on older adults were mixed, with some studies suggesting positive impacts of robots, some expressing concerns about robots and barriers to their use, and others being inconclusive. Although many therapeutic benefits of care robots have been identified, methodological limitations have weakened the internal and external validity of the findings of these studies. Few studies (18/69, 26%) considered the context of care: most studies (48/69, 70%) collected data only on recipients of care, 15 studies collected data on staff, and 3 studies collected data on relatives or visitors. Theory-driven, longitudinal, and large sample size study designs were rare. Across the authors’ disciplines, a lack of consistency in methodological quality and reporting makes it difficult to synthesize and assess research on care robotics. Conclusions The findings of this study call for more systematic research on the feasibility and efficacy of robots in assisted living facilities. In particular, there is a dearth of research on how robots may change geriatric care and the work environment within assisted living facilities. To maximize the benefits and minimize the consequences for older adults and caregivers, future research will require interdisciplinary collaboration among health sciences, computer science, and engineering as well as agreement on methodological standards.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Robots in Assisted Living Facilities: Scoping Review

Trainum¹,

Tunis²,

Xie³

et al. 2023

JMIR Aging

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge, there is no publicly available labeled dataset for learning ERI, not to mention that in a multiparty scenario. In our work, we created a labeled one by utilizing a dataset called BHEH dataset 53,54 . The BHEH dataset is video recordings of real ERI.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…In our work, we created a labeled one by utilizing a dataset called BHEH dataset. 53,54 The BHEH dataset is video recordings of real ERI. There was no constraint imposed on the participants.…”

Section: Bheh Datasetmentioning

confidence: 99%

Engagement estimation of the elderly from wild multiparty human–robot interaction

Zhang

Zheng

Thalmann

2022

Computer Animation & Virtual

View full text Add to dashboard Cite

The use of social robots in healthcare systems or nursing homes to assist the elderly and their caregivers will be becoming common, where robots' understanding of engagement of the elderly is important. Traditional engagement estimation (EE) often requires expert involvement in a controlled dyadic interaction environment. In this article, we propose a supervised machine learning method to estimate the engagement state of the elderly in a multiparty human-robot interaction (HRI) scenario from the real-world video recording as input. The method is built upon the basic concept of engagement in geriatric psychiatry and HRI video representations. It adapts pretrained models to extract behavior, affective, and visual signals to form the multi-modal features. These features are then fed into a neural network made of a self-attention mechanism and average pooling for individual learning, a graph attention network for group learning and a fully connected layer to estimate the engagement. We tested the proposed method using 43 wild multiparty elderly robot interaction (ERI) videos. The experimental results show that our method is capable of detecting the key participants and estimating the engagement state of the elderly effectively. Also our study demonstrates the signals from side-participants in the main interaction group considerably contribute to the EE of the elderly in the multiparty ERI.

show abstract