Linear spatio-temporal scale-space

Lindeberg, Tony

doi:10.1007/3-540-63167-4_44

Cited by 28 publications

(34 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative way of handling spatio-temporal scenes with dominant relative motions between the camera and the environment, in contrast to this use of space-time separable receptive fields for only image velocity v = 0, is by exploiting the full structure of the spatio-temporal receptive field model (1), by considering spatio-temporal receptive fields with nonzero image velocities v = 0, which can be locally adapted to the local motion direction corresponding to velocity adaptation [50,51,61] or alternatively performing local, regional or global image stabilization. Then, the image operations can be made truly covariant under local, regional or global Galilean image transformations [67,71] and allow for a more explicit separation of spatio-temporal receptive field responses that correspond to more complex spatio-temporal image structures than local Galilean motions.…”

Section: Resultsmentioning

confidence: 99%

“…For this domain, we can consider two basic use cases: For offline analysis of pre-recorded video, one may take the liberty of accessing the virtual future in relation to any pre-recorded time moment and make use of symmetric filtering over the temporal domain based on the non-causal Gaussian spatio-temporal scale-space theory [61,67,70]. For online analysis of real-time video streams on the other hand, the future cannot be accessed and we will base the analysis on a fully time-causal and time-recursive spatiotemporal scale-space concept for real-time image streams that only requires access to information from the present moment and a very compact buffer of what has occurred in the past [75] and which constitutes an extension of previous temporal scale-space and multi-scale models [23,27,45,81,110].…”

Section: Fig 4 the First-and Second-order Temporal Derivatives Of Thmentioning

confidence: 99%

See 1 more Smart Citation

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

Self Cite

View full text Add to dashboard Cite

This work presents a theory and methodology for simultaneous detection of local spatial and temporal scales in video data. The underlying idea is that if we process video data by spatio-temporal receptive fields at multiple spatial and temporal scales, we would like to generate hypotheses about the spatial extent and the temporal duration of the underlying spatio-temporal image structures that gave rise to the feature responses. For two types of spatio-temporal scale-space representations, (i) a non-causal Gaussian spatio-temporal scale space for offline analysis of pre-recorded video sequences and (ii) a time-causal and timerecursive spatio-temporal scale space for online analysis of real-time video streams, we express sufficient conditions for spatio-temporal feature detectors in terms of spatio-temporal receptive fields to deliver scale-covariant and scale-invariant feature responses. We present an in-depth theoretical analysis of the scale selection properties of eight types of spatio-temporal interest point detectors in terms of either: (i)-(ii) the spatial Laplacian applied to the first-and secondorder temporal derivatives, (iii)-(iv) the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives, (v) the determinant of the spatio-temporal Hessian matrix, (vi) the spatio-temporal Laplacian and (vii)-(viii) the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix. It is shown that seven of these spatio-temporal feature detectors allow for provable scale covariance and scale invariance. Then, we describe a time-causal and time-recursive algorithm for detecting sparse spatio-temporal interest points from video streams and show that it leads to intuitively reasonable results. An experimental quantification of the accuracy of the spatio-temporal scale estimates and the amount of temporal delay obtained from these spatio-temporal interest point detectors is given, showing that: (i) the spatial and temporal scale selection properties predicted by the continuous theory are well preserved in the discrete implementation and (ii) the spatial Laplacian or the determinant of the spatial Hessian applied to the first-and second-order temporal derivatives leads to much shorter temporal delays in a timecausal implementation compared to the determinant of the spatio-temporal Hessian or the first-and second-order temporal derivatives of the determinant of the spatial Hessian matrix.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Fig 4 the First-and Second-order Temporal Derivatives Of Thmentioning

confidence: 99%

Spatio-Temporal Scale Selection in Video Data

Lindeberg

2017

J Math Imaging Vis

Self Cite

View full text Add to dashboard Cite

show abstract

“…A few articles analyze spatio-temporal data from a scale-space standpoint [24][25][26][27]. These papers define multiscale representation of images from various axioms such as non-enhancement of local extrema.…”

Section: Theoretical Scale-space Studiesmentioning

confidence: 99%

“…Therefore, although continuous modeling can be useful for image-space analysis [23], we prefer a discrete approach that lets us work at a distance from the domain boundaries. Such a choice has also been done in scale-space studies [26,27].…”

Section: Time Axis In Video Streamsmentioning

confidence: 99%

See 1 more Smart Citation

Edge-Preserving Smoothing and Mean-Shift Segmentation of Video Streams

Paris

2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Video streams are ubiquitous in applications such as surveillance, games, and live broadcast. Processing and analyzing these data is challenging because algorithms have to be efficient in order to process the data on the fly. From a theoretical standpoint, video streams have their own specificities -they mix spatial and temporal dimensions, and compared to standard video sequences, half of the information is missing, i.e. the future is unknown. The theoretical part of our work is motivated by the ubiquitous use of the Gaussian kernel in tools such as bilateral filtering and mean-shift segmentation. We formally derive its equivalent for video streams as well as a dedicated expression of isotropic diffusion. Building upon this theoretical ground, we adapt a number of classical powerful algorithms to video streams: bilateral filtering, mean-shift segmentation, and anisotropic diffusion.

show abstract

A Markov Random Field Approach to Multi-scale Shape Analysis

Pizer

Joshi

2003

Scale Space Methods in Computer Vision

View full text Add to dashboard Cite

Characterizing the geometric conformation of object complexes requires the description of certain geometric features. These features are most intuitive and provide locality if each is at a restricted range of scale. Thus a complete representation of a geometric entity, such as an object, includes descriptions at multiple scale levels, each describing a residue from the information provided at the larger scales. We have been developing a methodology using medial representations for 3D geometric entities. In this framework, we represent these entities at the following natural discrete scale levels: a whole object complex, individual objects, various object parts and sections, and fine boundary details. This has been proven to be a robust representation of the geometry scale space. It is particularly useful for probabilistic characterization that describes both the common geometry of a population of object complexes and the variation of instances within the population. Using our medial representation, we build Markov random field (MRF) models on the geometry scale space based on the statistics of shape residue across scales and between neighboring geometric entities at the level of locality given by its scale. In this paper, we present how to design MRF models on two scale levels, namely boundary displacement and object sections. This approach can be applied to various applications in medical image analysis such as image segmentation and object discrimination into classes.

show abstract

Linear spatio-temporal scale-space

Cited by 28 publications

References 34 publications

Spatio-Temporal Scale Selection in Video Data

Spatio-Temporal Scale Selection in Video Data

Edge-Preserving Smoothing and Mean-Shift Segmentation of Video Streams

A Markov Random Field Approach to Multi-scale Shape Analysis

Contact Info

Product

Resources

About