Iterative path optimisation for personalised dressing assistance using vision and force information

Gao, Yixing; Chang, Hyung Jin; Demiris, Yiannis

doi:10.1109/iros.2016.7759647

Cited by 74 publications

(106 citation statements)

References 18 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…To compensate for the disadvantages of using vision information only, force information has been successfully used in [1] to infer if the user's forearm can successfully enter the sleeve. In [9] and our previous research [7], the force resistance between the robot gripper and the body has been used to adjust the robot motion with a pure position controller. Although these methods can update the robot motion based on the detected force, this is not achieved in real-time and the delay induced by robot position update may let the robot in an uncomfortable or even dangerous state for the user.…”

Section: Related Workmentioning

confidence: 99%

“…A common approach to solve this problem is to recognize the postures of the user in real time and to adjust the robot's trajectory accordingly [2][3][4][5][6]. However, severe occlusions occur when the robot arms, the clothes and the human body are in close contact [7], which lead to real-time human pose recognition failures during the dressing process. For instance, depth cameras, which are often used for human pose recognition, typically fail to extract the user's skeleton because of the occlusions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Personalized robot-assisted dressing using user modeling in latent spaces

Zhang¹,

Cully²,

Demiris³

2017

2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Abstract-Robots have the potential to provide tremendous support to disabled and elderly people in their everyday tasks, such as dressing. Many recent studies on robotic dressing assistance usually view dressing as a trajectory planning problem. However, the user movements during the dressing process are rarely taken into account, which often leads to the failures of the planned trajectory and may put the user at risk. The main difficulty of taking user movements into account is caused by severe occlusions created by the robot, the user, and the clothes during the dressing process, which prevent vision sensors from accurately detecting the postures of the user in real time. In this paper, we address this problem by introducing an approach that allows the robot to automatically adapt its motion according to the force applied on the robot's gripper caused by user movements. There are two main contributions introduced in this paper: 1) the use of a hierarchical multi-task control strategy to automatically adapt the robot motion and minimize the force applied between the user and the robot caused by user movements; 2) the online update of the dressing trajectory based on the user movement limitations modeled with the Gaussian Process Latent Variable Model in a latent space, and the density information extracted from such latent space. The combination of these two contributions leads to a personalized dressing assistance that can cope with unpredicted user movements during the dressing while constantly minimizing the force that the robot may apply on the user. The experimental results demonstrate that the proposed method allows the Baxter humanoid robot to provide personalized dressing assistance for human users with simulated upper-body impairments.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Personalized robot-assisted dressing using user modeling in latent spaces

Zhang¹,

Cully²,

Demiris³

2017

2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent studies included additional modalities such as haptics to improve the interaction with the user [18], [19], [20], [21], [22]. The evaluation of such systems focused on robot performance without considering the direct user input for robot personalization, hence limiting the scalability of such systems in applications with people.…”

Section: A Relevant Workmentioning

confidence: 99%

Personalized Robot Assistant for Support in Dressing

Jevtić

Valle

Alenyà

et al. 2019

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-Robot-assisted dressing is performed in close physical interaction with users who may have a wide range of physical characteristics and abilities. Design of user adaptive and personalized robots in this context is still indicating limited, or no consideration, of specific user-related issues. This paper describes the development of a multi-modal robotic system for a specific dressing scenario -putting on a shoe, where users' personalized inputs contribute to a much improved task success rate. We have developed: 1) user tracking, gesture recognition and posture recognition algorithms relying on images provided by a depth camera; 2) a shoe recognition algorithm from RGB and depth images; 3) speech recognition and text-to-speech algorithms implemented to allow verbal interaction between the robot and user. The interaction is further enhanced by calibrated recognition of the users' pointing gestures and adjusted robot's shoe delivery position. A series of shoe fitting experiments have been performed on two groups of users, with and without previous robot personalization, to assess how it affects the interaction performance. Our results show that the shoe fitting task with the personalized robot is completed in shorter time, with a smaller number of user commands and reduced workload.

show abstract

“…Gao et al (2016) have used RGB-D data to estimate user pose and initial robot trajectory, which was then updated based on force feedback information for dressing a sleeveless jacket using the Baxter robot. Another group have made predictive models for dressing a hospital gown based on just the force modality, removing the vision input altogether, in a simple single arm experiment (Kapusta et al, 2016).…”

Section: Related Workmentioning

confidence: 99%