Spatio-Temporal Semantic Object Segmentation using Probabilistic Sub-Object Regions

Abstract

The MPEG-4 standard generated a need for the extraction of Video Object Planes for usage in video retrieval and description. Two later standards MPEG-7 and MPEG-21 further the need for systems requiring minimal user interaction for the accurate extraction of semantic video objects. Many previous approaches have either relied too heavily on user interaction or made compromises in end accuracy to achieve a faster segmentation process. With the advancement of computer processing power we propose a higher quality segmentation process with end application requiring a hardware implementation. The focus of our approach is to achieve a reliable and high quality segmentation mask per frame using sophisticated offline techniques to minimise the user interaction within the process.

Video Object Segmentation Using Multiple Features

“…Regarding the relative contribution of each feature to the overall performance, we found out, as note in [7], that colour information alone is not very reliable. For this reason colour and spatial information must be appended into the same feature vector.…”

“…With the initial object and background classification, we learn the object and background GMM. This is similar in spirit to [4,7,6]. We describe object and background as a set of regions each one modelled with a Gaussian distribution.…”

“…The reasons for this problem are, on one hand the fact that GMM usually produce small errors that then are propagated to future frames. On the other hand, the difficulty to reliably update the spatial information [7]. To overcome these problems we propose: first to regularize the posterior probabilities of object and background using an isotropic probability diffusion algorithm [8].…”