Prototyping and evaluation of interactive and customized interface and control algorithms for robotic assistive devices using Kinect and infrared sensor

Abstract

Robotic assistive devices are popular in research and medical fields for their potential to automate tasks or to improve the quality of life of disabled users. They may be used for physical therapy, as exoskeletons, teleoperation devices, or to assist users in tasks in their home. Common methods for controlling these devices use both broad and difficult-to-maintain gesturing or peripherals such as mouse pointers or joysticks which are not designed for the task required of them. In addition, these devices are often not adaptive to the user and can only be minimally customized. This article proposes a fusion of infrared camera data for stress detection with Kinect body tracking to develop a customizable control method for a robotic limb. Devices such as the Microsoft Kinect have seen use in physical therapy applications but only some use in teleoperation. In addition, studies have shown potential in using infrared imaging to detect human stress. The objectives of this study are to design and build an interactive interface and adaptive control system and to evaluate its performance. Stress detection using infrared was tested using a Compix 222 and neural networks to categorize emotional states. Kinect v2 accuracy and reliability was tested by comparing joint positions to detected angles and perceived output angles. Our work suggests that infrared imaging and the Kinect v2 show potential to make a real-time adaptive system in which a control program can adapt its output when it detects stress from its user.