Vision-based Recognition of Activities by a Humanoid Robot

Abstract

We present an autonomous assistive robotic system for human activity recognition from video sequences. Due to the large variability inherent to video capture from a nonfixed robot (as opposed to a fixed camera), as well as the robot's limited computing resources, implementation has been guided by robustness to this variability and by memory and computing speed efficiency. To accommodate motion speed variability across users, we encode motion using dense interest point trajectories. Our recognition model harnesses the dense interest point bag-of-words representation through an intersection kernel-based SVM that better accommodates the large intra-class variability stemming from a robot operating in different locations and conditions. To contextually assess the engine as implemented in the robot, we compare it with the most recent approaches of human action recognition performed on public datasets (non-robot-based), including a novel approach of our own that is based on a two-layer SVMhidden conditional random field sequential recognition model. The latter's performance is among the best within the recent state of the art. We show that our robot-based recognition engine, while less accurate than the sequential model, nonetheless shows good performances, especially given the adverse test conditions of the robot, relative to those of a fixed camera.

Classification of materials by acoustic signal processing in real time for NAO robots

“…Other potential applications include systems for cognitive personal assistance such as user support in complex environments and basic support for elderly or disabled people making domestic tasks and activities of daily life that would be increasingly challenging and easier for them. 3,34 As the approach presented in this article works remotely, meaning that there is no need to go where the robot is to start the classification, we can think of even more potential applications that will reduce the need for the user to move as most robots will be remotely controlled using a computer or other device. 34 …”

“…Human-robot interaction is becoming more common; therefore, robot perception of the world has evolved into many modalities such as vision, speech, touch, among others. One example for artificial vision perception is presented by El-Yacoubi et al, 3 an autonomous assistive robotic system for human activity recognition from video sequences that integrates machine vision algorithms into an NAO robot, allowing it to recognize activities of daily life (sitting down, falling down, opening a door, applauding) performed by a person in a smart home.…”

“…Since robot manipulation relies on contact between the robot and the world, perception is one of the most important challenges facing the field. Although most object categorization systems used for perception in robotics and artificial intelligence are image-based, in NAO robots, El-Yacoubi et al 3 highlight important disadvantages presented such as dependency in the amount of light which leads to constant calibration of cameras, manual tuning on color variations, high processing resources causing delays in displaying information, and limits in storage power for image and video. For example, using visual motion and shape contours, a robot can detect a specific object such as a bottle or a teacup.…”

A Framework for Service Robots in Smart Home: An Efficient Solution for Domestic Healthcare

“…'Cooking', 'Sleeping' and 'Falling' are possible activities. The robot is able to recognize what the user is doing from vision [13,14]. However, these image based techniques can be a bit inaccurate and require a lot of training.…”

“…But, as expressed in Section III, it encounters difficulties. We enhance the vision process described presented in [13,14]. In brief, it works following four steps that rely on several supervised learning algorithm:…”