Abstract:For reaching to and grasping of an object, visual information about the object must be transformed into motor or postural commands for the arm and hand. In this paper, we present a robot model for visually guided reaching and grasping. The model mimics two alternative processing pathways for grasping, which are also likely to coexist in the human brain. The first pathway directly uses the retinal activation to encode the target position. In the second pathway, a saccade controller makes the eyes (cameras) focu… Show more
“…First, from the mixture model, the Gaussian j was chosen for which the noisy patch had the smallest normalized Mahalanobis distance p j . Such a distance value was computed from the eigenvectors W j (a d × q matrix containing the eigenvectors in its columns), the eigenvalues Λ j (a diagonal matrix), and the residual variance per dimension σ 2 j as obtained from a spatially-localized probabilistic principal component analysis, which is part of the above EM-algorithm (Tipping and Bishop, 1999;Hoffmann et al, 2005). Second, the image patch was reconstructed based on the principal components of the chosen local model (Fig.…”
Several scientists suggested that certain perceptual qualities are based on sensorimotor anticipation: for example, the softness of a sponge is perceived by anticipating the sensations resulting from a grasping movement. For the perception of spatial arrangements, this article demonstrates that this concept can be realized in a mobile robot. The robot first learned to predict how its visual input changes under movement commands. With this ability, two perceptual tasks could be solved: judging the distance to an obstacle in front by 'mentally' simulating a movement toward the obstacle, and recognizing a dead end by simulating either an obstacle-avoidance algorithm or a recursive search for an exit. A simulated movement contained a series of prediction steps. In each step, a multi-layer perceptron anticipated the next image, which, however, became increasingly noisy. To denoise an image, it was split into patches, and each patch was projected onto a manifold obtained by modeling the density of the distribution of training patches with a mixture of Gaussian functions.
“…First, from the mixture model, the Gaussian j was chosen for which the noisy patch had the smallest normalized Mahalanobis distance p j . Such a distance value was computed from the eigenvectors W j (a d × q matrix containing the eigenvectors in its columns), the eigenvalues Λ j (a diagonal matrix), and the residual variance per dimension σ 2 j as obtained from a spatially-localized probabilistic principal component analysis, which is part of the above EM-algorithm (Tipping and Bishop, 1999;Hoffmann et al, 2005). Second, the image patch was reconstructed based on the principal components of the chosen local model (Fig.…”
Several scientists suggested that certain perceptual qualities are based on sensorimotor anticipation: for example, the softness of a sponge is perceived by anticipating the sensations resulting from a grasping movement. For the perception of spatial arrangements, this article demonstrates that this concept can be realized in a mobile robot. The robot first learned to predict how its visual input changes under movement commands. With this ability, two perceptual tasks could be solved: judging the distance to an obstacle in front by 'mentally' simulating a movement toward the obstacle, and recognizing a dead end by simulating either an obstacle-avoidance algorithm or a recursive search for an exit. A simulated movement contained a series of prediction steps. In each step, a multi-layer perceptron anticipated the next image, which, however, became increasingly noisy. To denoise an image, it was split into patches, and each patch was projected onto a manifold obtained by modeling the density of the distribution of training patches with a mixture of Gaussian functions.
“…The visual representation of space is maintained by a spherical-like coordinate system that is implicitly defined by the gaze direction. This representation is particularly suitable for autonomous learning and has been adopted in several recent works (Schenck et al, 2003;Hoffmann et al, 2005;Chinellato et al, 2011;Jamone et al, 2014).…”
Active eye movements can be exploited to build a visuomotor representation of the surrounding environment. Maintaining and improving such representation requires to update the internal model involved in the generation of eye movements. From this perspective, action and perception are thus tightly coupled and interdependent. In this work, we encoded the internal model for oculomotor control with an adaptive filter inspired by the functionality of the cerebellum. Recurrent loops between a feed-back controller and the internal model allow our system to perform accurate binocular saccades and create an implicit representation of the nearby space. Simulations results show that this recurrent architecture outperforms classical feedback-error-learning in terms of both accuracy and sensitivity to system parameters. The proposed approach was validated implementing the framework on an anthropomorphic robotic head.
“…Our approach uses human demonstrations, which provide a model to guide the dynamics of motion as in open-loop visuomotor transformation techniques (Hoffmann et al 2005;Natale et al 2005Natale et al , 2007Hulse et al 2009). A stable model of the high-dimensional visuomotor coordination can be learned by using only several human demonstrations, making it a very efficient, fast and intuitive way to estimate parameters of a robot visuomotor controller.…”
Section: Discussion On Controller Architecturementioning
confidence: 99%
“…Not being able to rapidly and synchronously react to perturbations can cause fatal consequences for both the robot and its environment. Solutions to robotic visual-based reaching follow either of two well-established approaches: techniques that learn visuomotor transformations (Hoffmann et al 2005;Natale et al 2005Natale et al , 2007Hulse et al 2009;Jamone et al 2012), which operate in an open-loop manner, or visual servoing techniques (Espiau et al 1992;Mansard et al 2006;Natale et al 2007;Chaumette and Hutchinson 2008;Jamone et al 2012), which are closed-loop methods. Techniques that learn the visuomotor maps are very appealing because of their simplicity and practical applications.…”
Section: Robotic Visually Aided Manipulation and Obstacle Avoidancementioning
We investigate the role of obstacle avoidance in visually guided reaching and grasping movements. We report on a human study in which subjects performed prehensile motion with obstacle avoidance where the position of the obstacle was systematically varied across trials. These experiments suggest that reaching with obstacle avoidance is organized in a sequential manner, where the obstacle acts as an intermediary target. Furthermore, we demonstrate that the notion of workspace travelled by the hand is embedded explicitly in a forward planning scheme, which is actively involved in detecting obstacles on the way when performing reaching. We find that the gaze proactively coordinates the pattern of eye-arm motion during obstacle avoidance. This study provides also a quantitative assessment of the coupling between the eye-arm-hand motion. We show that the coupling follows regular phase dependencies and is unaltered during obstacle avoidance. These observations provide a basis for the design of a computational model. Our controller extends the coupled dynamical systems framework and provides fast and synchronous control of the eyes, the arm and the hand within a single and compact framework, mimicking similar control system found in humans. We validate our model for visuomotor control of a humanoid robot.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.