Toward Capturing Scientific Evidence in Elderly Care: Efficient Extraction of Changing Facial Feature Points

Abstract

To capture scientific evidence in elderly care, a user-defined facial expression sensing service was proposed in our previous study. Since the time-series data of feature values have been growing at a high rate as the measurement time increases, it may be difficult to find points of interest, especially for detecting changes from the elderly facial expression, such as many elderly people can only be shown in a micro facial expression due to facial wrinkles and aging. The purpose of this paper is to implement a method to efficiently find points of interest (PoI) from the facial feature time-series data of the elderly. In the proposed method, the concept of changing point detection into the analysis of feature values is incorporated by us, to automatically detect big fluctuations or changes in the trend in feature values and detect the moment when the subject’s facial expression changed significantly. Our key idea is to introduce the novel concept of composite feature value to achieve higher accuracy and apply change-point detection to it as well as to single feature values. Furthermore, the PoI finding results from the facial feature time-series data of young volunteers and the elderly are analyzed and evaluated. By the experiments, it is found that the proposed method is able to capture the moment of large facial movements even for people with micro facial expressions and obtain information that can be used as a clue to investigate their response to care.

Assisting Personalized Healthcare of Elderly People: Developing a Rule-Based Virtual Caregiver System Using Mobile Chatbot

“…In [ 62 ], by utilizing the MMDAgent, we have developed a system called virtual care giver (VCG), where the virtual agent named “Mei” provides personalized cares for each elderly person [ 63 ]. The VCG supports a wide range of care, such as reminders to check for forgotten items, warnings for forgetting to take a medicine, and playing music videos on YouTube as a value-added service for entertainment [ 37 , 38 ]. Delegating the communication care with the agent, we believe human caregivers can concentrate on human-centric tasks that cannot be achieved with the machine.…”

“…using various sensors [ 27 , 28 ], we principally focus on the inner expressions (i.e., thinking and feeling) of elderly people. In previous studies, to realize a smooth care nursing intervention, a speech-based interaction between virtual agents (i.e., MMDAgent [ 29 ]) and elderly people on a general computer has been proposed [ 30 , 31 , 32 ], such as confirming the daily state [ 33 ], answering the counseling [ 34 , 35 , 36 ], and extending to micro service such as watching favorite videos [ 37 , 38 ]. Our key concept is to regard a virtual agent as a caregiver, called a virtual caregiver [ 39 ].…”

Analytics and Applications of Audio and Image Sensing Techniques

“…K. Hirayama, S. Chen, S. Saiki, and M. Nakamura in [ 15 ] describe a system designed for detecting significant changes in facial expression that can be used in elderly care, based on the video signal from the built-in camera of the laptop. Facial feature data were extracted at 1 s intervals, thus creating a time series, for one of the authors and for five subjects receiving care.…”

事前学習済みモデルと音声対話エージェントを用いた顔識別システムの開発

“…It is difficult to determine the identity of the person with whom the spoken dialogue agent interacts. Various smart devices and services [4] using cloud computing and deep learning have been introduced to face recognition technology in line with the rapid development of artificial intelligence. However, the cost and computational resources required to build and apply recognition models remain an issue.…”