Toward Kinecting cognition by behaviour recognition-based deep learning and big data

Abstract

The majority of older people wish to live independently at home as long as possible despite having a range of age-related conditions including cognitive impairment. To facilitate this, there has been an extensive focus on exploring the capability of new technologies with limited success. This paper investigates whether MS Kinect (a motion-based sensing 3-D scanner device) within the MiiHome (My Intelligent Home) project in conjunction with other sensory data, machine learning and big data techniques can assist in the diagnosis and prognosis of cognitive impairment and hence prolong independent living. A pool of Kinect devices and various sensors powered by minicomputers providing internet connectivity are being installed in up to 200 homes. This enables continuous remote monitoring of elderly residents living alone. Passive and off-the-shelf sensor technologies were chosen to implement data acquisition specifically from sources that are part of the fabric of the homes, so that no extra effort is required from the participants. Various constraints including environmental, geometrical and big data were identified and appropriately dealt with. A visualization tool (MAGID) was developed for validation and verification of numerous behavioural activities. Then, a subset of data, from twelve pensioners aged over 65 with age-related cognitive decline and frailty, were collected over a period of 6 months. These data were subjected to several machine learning algorithms (multilayer perceptron neural network, neuro-fuzzy and deep learning) for classification and to extract routine behavioural patterns. These patterns were then analysed further to ascertain any health-related information and their attributes. For the first time, important routine behaviour related to Activities of Daily Living (ADL) of elderly people with cognitive and physical decline has been learnt by machine learning techniques from selected sample data obtained by MS Kinect. Medically important behaviour, e.g. eating, walking, sitting, was best learnt by deep learning with accuracy of 99.30% during training stage and average error rate of 1.83% with maximum of 12.98% during the implementation phase. Observations obtained from the application of the above learnt behaviours are presented as trends over a period of time. These trends, supplemented by other sensory signals, have provided a clearer picture of physical (in)activities (including falls) of the pensioners. The calculated behavioural attributes related to key indicators of health events can be used to model the trajectory of health status related to cognitive decline in a home setting. These results, based on a small number of elderly residents over a short period of time, imply that within the results obtained from the MiiHome project, it is possible to find indicators of cognitive decline. However, further studies are needed for full clinical validation of these indications in conjunction with assessment of cognitive decline of the participants.

Gait speeds classifications by supervised modulation based machine-learning using kinect camera

“…Finally, the classifier's accuracy, confusion matrix, and AUC evaluation metric showed that a better results with high confidence band and predictive capabilities obtained by our purposed methodologies for classification of modified leg velocity signals. Further works includes extending the application of this techniques toward Kinecting cognitive impairments [15] beside using amplitude modified signals and explore the expectation that AM might offer better capability to represent the spectral components of a low frequency signal than the FM technique when it comes to dealing with MS Kinect data that has medium/low data rate of 30 frames per second.…”

Application of Behavior Recognition Technology Based on Deep Learning in Elderly Care

“…Neurons and perceptron models: The concept of neurons comes from biology. A simple neuron usually consists of several dendrites, an axon, and several axon-ends (Soufian et al, 2022). Among them, dendrites are mainly used to receive input information, axon ends transmit information to the next neuron, and axons are used to connect dendrites and axon ends (Yu et al, 2022).…”