A geometric framework for nonlinear visual coding

Barth, Erhardt; Watson, Andrew B.

doi:10.1364/oe.7.000155

Cited by 21 publications

(13 citation statements)

References 30 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…have Gaussian curvature. In case of n = 3 (here movie surfaces), we still have mean and Gaussian curvature but here the "true" curvature is measured by the Riemann tensor -see Barth and Watson (2000).…”

Section: Curvature and Uniqueness Of 2d Regionsmentioning

confidence: 99%

Efficient visual coding and the predictability of eye movements on natural movies

2009

Self Cite

View full text Add to dashboard Cite

We deal with the analysis of eye movements made on natural movies in free-viewing conditions. Saccades are detected and used to label two classes of movie patches as attended and non-attended. Machine learning techniques are then used to determine how well the two classes can be separated, i.e. how predictable saccade targets are. Although very simple saliency measures are used and then averaged to obtain just one average value per scale, the two classes can be separated with an ROC score of around 0.7, which is higher than previously reported results. Moreover, predictability is analysed for different representations to obtain indirect evidence for the likelihood of a particular representation. It is shown that the predictability correlates with the local intrinsic dimension in a movie.

show abstract

Section: Curvature and Uniqueness Of 2d Regionsmentioning

confidence: 99%

Efficient visual coding and the predictability of eye movements on natural movies

2009

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the straightforward hypothesis that during visual processing signals with lower intrinsic dimension are suppressed renders also our model biologically plausible as well. Indeed, previous work has shown that this simple hypothesis can already explain the occurrence of lateral inhibition (i0D signals are suppressed), end-stopping (i1D signals are suppressed) [35], and motion selectivity [50].…”

Section: Discussionmentioning

confidence: 88%

Intrinsic Dimensionality Predicts the Saliency of Natural Dynamic Scenes

Vig

Dörr

Martinetz

et al. 2012

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-Since visual attention-based computer vision applications have gained popularity, ever more complex, biologicallyinspired models seem to be needed to predict salient locations (or interest points) in naturalistic scenes. In this paper, we explore how far one can go in predicting eye movements by using only basic signal processing, such as image representations derived from efficient coding principles, and machine learning. To this end, we gradually increase the complexity of a model from simple single-scale saliency maps computed on grayscale videos to spatio-temporal multiscale and multispectral representations. Using a large collection of eye movements on high-resolution videos, supervised learning techniques fine-tune the free parameters whose addition is inevitable with increasing complexity. The proposed model, although very simple, demonstrates significant improvement in predicting salient locations in naturalistic videos over four selected baseline models and two distinct data labelling scenarios.Index Terms-Computational models of vision, video analysis, computer vision, spatio-temporal saliency, eye movement prediction, intrinsic dimension, visual attention, interest point detection.

show abstract

“…they fully specify the image [8]. The evaluation of the intrinsic dimension is possible within a geometric approach that is plausible for biological vision [9] and is implemented here by using the structure tensor J, which is well known in the computer-vision literature [10]. The structure tensor is defined in terms of the spatio-temporal gradient ∇f of the image intensity function…”

Section: Eye Movement Predictionsmentioning

confidence: 99%

Guiding Eye Movements for Better Communication and Augmented Vision

Barth

Dörr

Böhme

et al. 2006

Perception and Interactive Technologies

Self Cite

View full text Add to dashboard Cite

Abstract. This paper briefly summarises our results on gaze guidance such as to complement the demonstrations that we plan to present at the workshop. Our goal is to integrate gaze into visual communication systems by measuring and guiding eye movements. Our strategy is to predict a set of about ten salient locations and then change the probability for one of these candidates to be attended: for one candidate the probability is increased, for the others it is decreased. To increase saliency, in our current implementation, we show a natural-scene movie and overlay red dots very briefly such that they are hardly perceived consciously. To decrease the probability, for example, we locally reduce the temporal frequency content of the movie. We here present preliminary results, which show that the three steps of our above strategy are feasible. The long-term goal is to find the optimal real-time video transformation that minimises the difference between the actual and the desired eye movements without being obtrusive. Applications are in the area of vision-based communication, augmented vision, and learning.

show abstract

A geometric framework for nonlinear visual coding

Cited by 21 publications

References 30 publications

Efficient visual coding and the predictability of eye movements on natural movies

Efficient visual coding and the predictability of eye movements on natural movies

Intrinsic Dimensionality Predicts the Saliency of Natural Dynamic Scenes

Guiding Eye Movements for Better Communication and Augmented Vision

Contact Info

Product

Resources

About