Multimodal saliency-based bottom-up attention a framework for the humanoid robot iCub

Ruesch, Jonas; Lopes, Manuel; Bernardino, Alexandre; Hörnstein, Jonas; Santos-Victor, José; Pfeifer, Rolf

doi:10.1109/robot.2008.4543329

Cited by 112 publications

(111 citation statements)

References 21 publications

Supporting

Mentioning

109

Contrasting

Unclassified

Order By: Relevance

“…shifting the focus of attention for efficient scene exploration (e.g. [8], [10]) and analysis (e.g. [14], [20], [21]) -has attracted increasing interest during the last years.…”

Section: Related Workmentioning

confidence: 99%

“…For active scene exploration, saliency can be used to steer the sensors towards salientthus potentially relevant -regions to detect objects of interest (e.g. [8], [10], [12]). Combining these methods, [14] utilized bottom-up attention, stereo vision and SIFT to perform robust and efficient scene analysis on a mobile robot.…”

Section: Related Workmentioning

confidence: 99%

“…[8]) or audio-visually (e.g. [10], [12]) salient regions is a natural and efficient method to detect and focus interacting persons.…”

Section: Realizationmentioning

confidence: 99%

See 2 more Smart Citations

Saliency-based identification and recognition of pointed-at objects

Schauerte

Richarz

Fink

2010

2010 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-When persons interact, non-verbal cues are used to direct the attention of persons towards objects of interest. Achieving joint attention this way is an important aspect of natural communication. Most importantly, it allows to couple verbal descriptions with the visual appearance of objects, if the referred-to object is non-verbally indicated. In this contribution, we present a system that utilizes bottom-up saliency and pointing gestures to efficiently identify pointed-at objects. Furthermore, the system focuses the visual attention by steering a pan-tilt-zoom camera towards the object of interest and thus provides a suitable model-view for SIFT-based recognition and learning. We demonstrate the practical applicability of the proposed system through experimental evaluation in different environments with multiple pointers and objects.

show abstract

“…shifting the focus of attention for efficient scene exploration (e.g. [8], [10]) and analysis (e.g. [14], [20], [21]) -has attracted increasing interest during the last years.…”

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Saliency-based identification and recognition of pointed-at objects

Schauerte

Richarz

Fink

2010

2010 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

show abstract

“…9). Whereas different methodologies can be used to define the target of the binocular fixation on the left and right image planes (Samarawickrama and Sabatini 2007; Ruesch et al 2008;Mishra et al 2009;Rea et al 2014;Antonelli et al 2014;Beuth and Hamker 2015), for the sake of simplicity we used a color segmentation based on the Water Shed algorithm (OpenCV implementation Bradski and Kaehler 2008). The algorithm computes a binary mask corresponding to a selected color, i.e.…”

Section: Integrated Vergence and Version Controlmentioning

confidence: 99%

A Portable Bio-Inspired Architecture for Efficient Robotic Vergence Control

et al. 2016

View full text Add to dashboard Cite

In stereoscopic vision, the ability of perceiving the three-dimensional structure of the surrounding environment is subordinated to a precise and effective motor control for the binocular coordination of the eyes/cameras. If, on the one side, the binocular coordination of camera movements is a complicating factor, on the other side, a proper vergence control, acting on the binocular disparity, facilitates the binocular fusion and the subsequent stereoscopic perception process. In real-world situations, an effective vergence control requires further features other than real time capabilities: real robot systems are indeed characterized by mechanical and geometrical imprecision that affect the binocular vision, and the illumination conditions are changeable and unpredictable. Moreover, in order to allow an effective visual exploration of the peripersonal space, it is necessary to cope with different gaze directions and provide a large working space. The proposed control strategy resorts to a neuromimetic approach that provides a distributed representation of disparity information. trol, which directly interacts with saccadic control. Before saccade, the open-loop component is computed in correspondence of the saccade target region, to obtain a vergence correction to be applied simultaneously with the saccade. At fixation, the closed-loop component drives the binocular disparity to zero in a foveal region. The obtained vergence servos are able to actively drive both the horizontal and the vertical alignment of the optical axes on the object of interest, thus ensuring a correct vergence posture. Experimental tests were purposely designed to measure the performance of the control in the peripersonal space, and were performed on three different robot platforms. The results demonstrated that the proposed approach yields real-time and effective vergence camera movements on a visual stimulus in a wide working range, regardless of the illumination in the environment and the geometry of the system.

show abstract

“…the act of directing the sensors towards salient stimuli, and scene exploration for robotic applications has been addressed by several authors in recent years (see, e.g., [3], [5], [7], [9], [27]- [29]). The main difference between (covert) attention mechanisms that operate on still images, i.e.…”

Section: Related Workmentioning

confidence: 99%

Multimodal saliency-based attention for object-based scene analysis

Schauerte

Kühn

Kroschel

et al. 2011

2011 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-Multimodal attention is a key requirement for humanoid robots in order to navigate in complex environments and act as social, cognitive human partners. To this end, robots have to incorporate attention mechanisms that focus the processing on the potentially most relevant stimuli while controlling the sensor orientation to improve the perception of these stimuli. In this paper, we present our implementation of audio-visual saliency-based attention that we integrated in a system for knowledge-driven audio-visual scene analysis and object-based world modeling. For this purpose, we introduce a novel isophote-based method for proto-object segmentation of saliency maps, a surprise-based auditory saliency definition, and a parametric 3-D model for multimodal saliency fusion. The applicability of the proposed system is demonstrated in a series of experiments.Index Terms-audio-visual saliency, auditory surprise, isophote-based visual proto-objects, parametric 3-D saliency model, object-based inhibition of return, multimodal attention, scene exploration, hierarchical object analysis, overt attention, active perception

show abstract

Multimodal saliency-based bottom-up attention a framework for the humanoid robot iCub

Cited by 112 publications

References 21 publications

Saliency-based identification and recognition of pointed-at objects

Saliency-based identification and recognition of pointed-at objects

A Portable Bio-Inspired Architecture for Efficient Robotic Vergence Control

Multimodal saliency-based attention for object-based scene analysis

Contact Info

Product

Resources

About