Ecological Approach for Object Relationship Extraction in Elderly Care Robot

Besari, Adnan Rachmat Anom; Chin, Wei Hong; Kubota, Naoyuki; Kurnianingsih, Kurnianingsih

doi:10.1109/rem49740.2020.9313877

Cited by 7 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These phases are object-centered coordinate transformation and validation of HOI recognition using the task-specific reach-to-grasp cycle. We accomplished a coordinate transformation centered on the object to simplify the validation procedure and get fewer data [31]. The goal of centering the item is to relocate the image coordinates' initials (0,0) to the middle of the object.…”

Section: Mesoscopic Level: Active Perception Abilitymentioning

confidence: 99%

Hand–object interaction recognition based on visual attention using multiscopic cyber-physical-social system

Besari¹,

Saputra

Chin

et al. 2023

Int. J. Adv. Intell. Informatics

View full text Add to dashboard Cite

Computer vision-based cyber-physical-social systems (CPSS) are predicted to be the future of independent hand rehabilitation. However, there is a link between hand function and cognition in the elderly that this technology has not adequately supported. To investigate this issue, this paper proposes a multiscopic CPSS framework by developing hand–object interaction (HOI) based on visual attention. First, we use egocentric vision to extract features from hand posture at the microscopic level. With 94.87% testing accuracy, we use three layers of graph neural network (GNN) based on hand skeletal features to categorize 16 grasp postures. Second, we use a mesoscopic active perception ability to validate the HOI with eye tracking in the task-specific reach-to-grasp cycle. With 90.75% testing accuracy, the distance between the fingertips and the center of an object is used as input to a multi-layer gated recurrent unit based on recurrent neural network architecture. Third, we incorporate visual attention into the cognitive ability for classifying multiple objects at the macroscopic level. In two scenarios with four activities, we use GNN with three convolutional layers to categorize some objects. The outcome demonstrates that the system can successfully separate objects based on related activities. Further research and development are expected to support the CPSS application in independent rehabilitation.

show abstract

Section: Mesoscopic Level: Active Perception Abilitymentioning

confidence: 99%

Hand–object interaction recognition based on visual attention using multiscopic cyber-physical-social system

Besari¹,

Saputra

Chin

et al. 2023

Int. J. Adv. Intell. Informatics

View full text Add to dashboard Cite

show abstract

“…To determine an active perception ability, we design a task-dependent rule set using the finger state from feature extraction at the microscopic level. Task dependency is a relationship in which tasks depend on regulations to be performed in a specific order developed in our previous study [51]. This task relates to CMT activities where the fingers interact with the chopsticks.…”

Section: B Mesoscopic Level: Active Perceptionmentioning

confidence: 99%

“…This variable measures the development of a person's finger motor function during rehabilitation with CMT. We design symbolic cognition from the task domain knowledge at this level to support our previous study [55]. This task is related to CMT activities where the fingers interact with chopsticks.…”

Section: B Mesoscopic Level: Active Perceptionmentioning

confidence: 99%

Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test

et al. 2022

Self Cite

View full text Add to dashboard Cite

Most East Asian rehabilitation centers offer chopsticks manipulation tests (CMT). In addition to impaired hand function, approximately two-thirds of stroke survivors have visual impairment related to eye movement. This article investigates the significance of combining finger joint angle estimation and a visual attention measurement in CMT. We present a multiscopic framework that consists of microscopic, mesoscopic, and macroscopic levels. We develop a feature extraction technique to extract the kinematic finger model at the microscopic level. At the mesoscopic level, we propose an active perception ability to detect the position and geometry of the finger on the chopsticks. The proposed framework estimates the proximal interphalangeal (PIP) joint angle on the index finger during CMT using fully connected cascade neural networks (FCC-NN). At the macroscopic level, we implement a cognitive ability by measuring visual attention during CMT. We further evaluate the proposed framework with a conventional test that counts the number of peanuts (NP) which are moved from one bowl to another using chopsticks within a particular time frame. For the evaluation, we introduce three parameters, namely joint angle estimation movement (JAEM), chopstick attention movement (CAM), and chopstick tip movement (CTM), by detecting the local minima and maxima of the signal. According to the experiment results, the velocity of these three parameters could indicate improvement in hand and eye function during CMT. We expect this study to benefit therapists and researchers by providing valuable information that is not accessible in the clinic. Code and datasets are available online at https://github.com/anom-tmu/cmt-attention.INDEX TERMS hand-eye interaction, rehabilitation evaluation, first-person vision, eye-gaze, eye-tracking.

show abstract

Multiscopic CPSS for Independent Block-Design Test Based on Hand–Object Interaction Recognition With Visual Attention

et al. 2023

View full text Add to dashboard Cite

Ecological Approach for Object Relationship Extraction in Elderly Care Robot

Cited by 7 publications

References 23 publications

Hand–object interaction recognition based on visual attention using multiscopic cyber-physical-social system

Hand–object interaction recognition based on visual attention using multiscopic cyber-physical-social system

Finger Joint Angle Estimation With Visual Attention for Rehabilitation Support: A Case Study of the Chopsticks Manipulation Test

Multiscopic CPSS for Independent Block-Design Test Based on Hand–Object Interaction Recognition With Visual Attention

Contact Info

Product

Resources

About