Extraction of Motion Data from Image Sequences to AssistAnimators

Ritter

2003

SPIE Proceedings

The movement of objects in video sequences comprises a type of spatiotemporal redundancy that can be decreased mathematically to facilitate video compression. This observation holds particularly in the case of periodic motion, for example, bipedal or quadrupedal locomotion or repetitive gestures. Previously-published motion detection techniques were based on optical flow, interframe differences represented in terms of transform coefficient perturbations, or changes in eigenvalues between frames in a video sequence. However, such methods have deficits that include sensitivity to noise, burdensome computational requirements (e.g., floating point operations), and prohibitive instability in the presence of spatial or temporal interframe discontinuities.In this paper, we discuss several techniques of motion detection in two-dimensional images of three-dimensional scenespointwise tracking of constant-intensity pixels, region-based vector field characterization of apparent motion, and correlationbased detection. In the latter category is a technique called Interframe Similarity Matrices (ISMs). ISMs were developed and successfully applied by Yacoob, Black, and Davis [1,2] to address the challenging problem of detecting human and animal motion in surveillance video sequences. In particular, given an N-frame video sequence, an NxN-element interframe correlation matrix can be constructed and Fourier-transformed to obtain an N/2-element power spectrum of interframe periodicities. Different actions (e.g., walking vs. running) and various actors (e.g., quadruped versus human) tend to be characterized by distinct spatiotemporal spectra, and can often be distinguished from one another.Since each spectrum can be computed from a sequence of small image regions, it is possible to represent interframe motion by a pixel tagging technique, thus implementing detection, segmentation, and representation. If there are K objects with M pixels per frame having B bits per pixel (bpp) in N frames of a compressed video sequence, and each object is segmented into a region represented by a P bit tag, then increased compression results if NKMB < NKP, i.e., MB < P. Implementational discussion concerns efficient algorithms for tagging of motion-containing regions, to decrease the representational overhead to several bits per region. We also discuss motion encoding in a compressed format for purposes of efficient extraction of motion parameters from a compressed image, which can support efficient object recognition in highresolution compressed image sequences.

Section: Implications For Video Compressionmentioning

confidence: 93%

Detection and Characterization of Motion in Video Compression

Ritter

2003

SPIE Proceedings

“…Several researchers have implemented object-based compression. Of particular interest to this study is the work of Campbell et al at University of Bristol, by which neural nets have been developed that can segment and recognize image regions in still or video imagery according to spectral (color) and textural (variance) criteria [18]. Such segmentation can be carried out a multiple resolution levels, and can be guided by information from a preprocessing compression stage such as JPEG, MPEG, or JPEG-2000 [5][6][7], where compressed blocks contain color, variance, or motion information.…”

Section: Principles Of Object-based Compressionmentioning

confidence: 99%

“…In practice, each region's compressed boundary and contents representations are appended to a list of such regions, to produce the compressed image frame. This process can be implemented in three dimensions for regions exhibiting motion, which are extracted from video imagery [18]. If each segmented and compressed region is considered as an object, then we have object-based compression (OBC).…”

mentioning

confidence: 99%

<title>Techniques for region coding in object-based image compression</title>

2004

SPIE Proceedings

Object-based compression (OBC) is an emerging technology that combines region segmentation and coding to produce a compact representation of a digital image or video sequence. Previous research has focused on a variety of segmentation and representation techniques for regions that comprise an image. The author has previously suggested [1] partitioning of the OBC problem into three steps: (1) region segmentation, (2) region boundary extraction and compression, and (3) region contents compression. A companion paper [2] surveys implementationally feasible techniques for boundary compression.In this paper, we analyze several strategies for region contents compression, including lossless compression, lossy VPIC, EPIC, and EBLAST compression, wavelet-based coding (e.g., JPEG-2000), as well as texture matching approaches. This paper is part of a larger study that seeks to develop highly efficient compression algorithms for still and video imagery, which would eventually support automated object recognition (AOR) and semantic lookup of images in large databases or highvolume OBC-format datastreams. Example applications include querying journalistic archives, scientific or medical imaging, surveillance image processing and target tracking, as well as compression of video for transmission over the Internet. Analysis emphasizes time and space complexity, as well as sources of reconstruction error in decompressed imagery.

“…The work of thresholding and component labelling is required after Equation (7) or (8) has been computed, as discussed in Section 2.2.1. Thus far, we have applied OBC to a neighborhood N X of size one pixel (i.e., |N| = 1).…”

Section: Multivalued Pixel Selectionmentioning

confidence: 99%

“…Pixel-level approaches are analyzed in Section 2, while region-or block-oriented techniques that exploit information derived from wavelet transformation are considered in Section 3. Section 4 contains an implementational analysis of Campbell et al's object-based segmentation approach, which applies techniques of OBC to replace textured regions or entire objects with exemplars stored in a codebook [8]. Conclusions and suggestions for future work are given in Section 5.…”

Section: Introductionmentioning

confidence: 99%

<title>Computational complexity of object-based image compression</title>

Mathematics of Data/Image Coding, Compression, and Encryption VIII, With Applications

Ritter

2005

Image compression via transform coding applied to small rectangular regions or encoding blocks appears to be approaching asymptotic rate-distortion performance. However, an emerging compression technology, called object-based compression (OBC) promises significantly improved performance via compression ratios ranging from 200:1 to as high as 2,500:1. OBC involves segmentation of image regions, followed by efficient encoding of each region's content and boundary. During decompression, such regions can be approximated by objects from a codebook, yielding a reconstructed image that is semantically equivalent to the corresponding source image, but has pixel-and featural-level differences.Semantic equivalence between the source and decompressed image facilitates fast decompression through efficient substitutions, albeit at the cost of codebook search in the compression step. Given small codebooks, OBC holds promise for information-push technologies where approximate context is sufficient, for example, transmission of surveillance images that provide the gist of a scene. However, OBC is not necessarily useful for applications requiring high accuracy, such as medical image processing, because substitution of source content can be inaccurate at small spatial scales.The cost of segmentation is a significant disadvantage in current OBC implementations. Several innovative techniques have been developed for region segmentation, as discussed in a previous paper [4]. Additionally, tradeoffs between representational fidelity, computational cost, and storage requirement occur, as with the vast majority of lossy compression algorithms. This paper analyzes the computational (time) and storage (space) complexities of several recent OBC algorithms applied to single-frame imagery. A time complexity model is proposed, which can be associated theoretically with a space complexity model that we have previously published [2]. The result, when combined with measurements of representational accuracy described in a companion paper [5], supports estimation of a time-space-error bandwidth product that could facilitate dynamic optimization of OBC algorithms. In practice, this would support efficient compression with visually acceptable reconstruction for a wide variety of military and domestic applications.