Approximating optical flow within the MPEG-2 compressed domain

Abstract

MPEG-2 compressed domain information, namely motion vectors and DCT coefficients, is filtered and manipulated to obtain a motion field using a two-dimensional (2-D) translational model. The results are compared to a popular optical flow method, more specifically the one presented by Lucas and Kanade, revealing very good results. Our method provides a very fast motion estimation tool that can be useful for applications where algorithmic cost is critical, such as surveillance systems. All methods are theoretically explained and their efficiency confirmed on real-world data.

Modeling Dominance in Group Conversations Using Nonverbal Activity Cues

“…In our data set, there is a camera taking a close-up shot of each participant, as shown in the bottom row of Figure 3. Each of these video streams has already been compressed by a MPEG-4 encoder with a group-of-picture (GOP) size of 250 frames and a GOP structure of I-P-P-..., where the first frame in the GOP is Intra-coded, and the rest of the frames are predicted frames [8]. Figure 4 summarizes the various compressed domain features which can be extracted cheaply from compressed video.…”

Using audio and video features to classify the most dominant person in a group meeting

“…This can be interpreted as crude approximations of the optical flow [6], where their magnitudes indicate the degree of translational motion at each block location. For our dataset, we used an MPEG-4 encoder with a group-of-picture (GOP) size of 250 frames and a {I-P-P-...} structure where the first frame (I) is Intra-coded, and the rest (P) are predicted frames.…”

High-Speed Action Recognition and Localization in Compressed Domain Videos

“…In addition, the Discrete Cosine Transform (DCT) coefficients can also be used to provide a confidence measure on the estimate. We follow the approach outlined by Coimbra and Davies [13] for computing a coarse estimate and a confidence map of the optical flow. To generate the optical flow estimate, we use the following rules [13]: 1) Motion vectors are normalized by the temporal distance of the predicted frame to the reference frame, and their directions are reversed if the motion vectors are forwardreferencing.…”

“…In particular, if the local neighborhood suffers from the aperture problem, then it is likely to have an unreliable optical flow estimate. By thresholding a confidence measure derived from the DCT coefficients that measures the amount of texture in the block [13], we can filter out optical flow estimates that are likely to be noisy. To compute the confidence measure for intra-coded macroblocks, we use [13]:…”

“…Coimbra and Davies have shown that F i (1, 0) and F i (0, 1) can be interpreted as a weighted average of spatial gradient in the x and y direction respectively [13].…”