Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution

Zhang, Hong-Jiang; Low, Chien Yong; Smolier, Stephen W.; Jianhua, WU

doi:10.1016/b978-155860651-7/50116-9

Cited by 98 publications

(119 citation statements)

References 9 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…The intersection histogram is a generalization of the intersection of two image histograms -intersection is a rephrasing of histogram difference in an L 1 norm [19]. Other algorithms advanced have used comparison to the first frame in a shot using colour and motion [23], accumulating frame differences [15], algorithms based on geometric metaphors, e.g. [3], and other approaches but here we mean to set out an algorithm that captures the salient colour features in a frame more effectively.…”

Section: Description Of Previous Methods and Motivation For New Methodsmentioning

confidence: 99%

Clustering of compressed illumination-invariant chromaticity signatures for efficient video summarization

Drew

2003

Image and Vision Computing

View full text Add to dashboard Cite

Motivated by colour constancy work in physics-based vision, we develop a new low-dimensional video frame feature that is effectively insensitive to lighting change and apply the feature to keyframe production using hierarchical clustering. The new image feature results from normalising colour channels for frames and then treating 2D histograms of chromaticity as images and compressing these. Because we effectively reduce any video to the same lighting conditions, we can precompute a universal basis on which to project video frame feature vectors. The new feature thus has the advantage of more expressively capturing essential colour information, and is useful for video indexing because it is very low-dimension -the feature vector is only of length 8. We carry out clustering efficiently by adapting the hierarchical clustering data structure to temporally-ordered clusters. Using a new multi-stage hierarchical clustering method, we merge clusters based on the ratio of cluster variance to variance of the parent node, merging only adjacent clusters, and then follow with a second round of clustering. The second stage merges clusters incorrectly split in the first round by the greedy hierarchical algorithm, and as well merges non-adjacent clusters to fuse near-repeat shots. The new summarization method produces a very succinct set of keyframes for videos and, compared to a previous well-known technique, results are excellent.

show abstract

Section: Description Of Previous Methods and Motivation For New Methodsmentioning

confidence: 99%

Clustering of compressed illumination-invariant chromaticity signatures for efficient video summarization

Drew

2003

Image and Vision Computing

View full text Add to dashboard Cite

show abstract

“…Amir et al (2005) indicated that a search can be defined as a global operation over an ent ire collection, while browsing, which usually follows a search operation, operates o n the results of the search to pinpoint the correct matches. Houten et al (2000) described five types of general browsing/ navigation models in general: Model navigation (e.g., storyboard video surrogates) presents all video information on one-page; Hierarchical navigation (Zhang et al, 1995) structures video surrogates in a hierarch ical tree; Relational navigation groups small video segments into clusters; Sequential navigation (or neighbor browsing) allows navigation in the temporal space; and Conceptual navigation usually supports metaphor to aid browsing.…”

Section: Related Researchmentioning

confidence: 99%

Semantic visual features in content‐based video retrieval

2006

Proc of Assoc for Info

View full text Add to dashboard Cite

A new semantic visual features (e.g., car, mountain, and fire) navigation technology is proposed to improve the effectiveness of video retrieval. Traditional temporal neighbor browsing technology allows users to navigate temporal neighbors of a selected sample frame to find additional matches, while semantic visual feature browsing enables users to navigate keyframes that have similar features to the selected sample frame. A pilot evaluation was conducted to compare the effectiveness of three video retrieval designs that support 1) temporal neighbor browsing; 2) semantic visual feature browsing; and 3) fused browsing which is a combination of both temporal neighbor and semantic visual feature browsing. Two types of searching tasks: visual centric and non-visual centric tasks were applied. Initial results indicated that the semantic visual feature browsing system was more efficient for non-visual centric tasks. IntroductionAccess to digital video from news sources such as CNN, MSNBC, or ABC has become commonplace. To make digital multimedia resource discovery and search more convenient, multimedia digital libraries are being developed for research and education.Increasingly, students or instructors are consulting video col lections in search of video shots within larger video "documents" to be used in their projects or lectures. Viewing all videos in full length to find the desired video shots may be feasible for a small collection, but can be very time intensive for a large collection. The ability to search within individual videos, much in the same way that full text searching allows users to search for content instead of their bibliographic surrogates, would g reatly increase access to video content. Recent research on content-based video retrieval indicated that initially performing a text-based query and subsequently proceeding with neighbor or visual similarity browsing proved to be an effective retrieval strategy (Wildemuth et al., 2003;Heesch et al., 2004;Mezaris et al., 2004 ; Amir et al., 2005). Human beings are usually good at pattern recognition through navigation. A retrieval system supporting navigation functions would provide users additional means for content rel ated searching tasks.In this paper we propose a new video content browsing techniqu e: semantic visual feature browsing. Our purpose is to evaluate its effectiveness as compared to traditional temporal neighbor browsing technique for two types of retrieval tasks: visual centric tasks and non-visual centric tasks. After the introduction of related research, a description of the semantic visual feature browsing algorithm will be given. The user interface of a prototype web-based video retrieval system that supports semantic visual feature browsing will be then illustrated. Finally, the methodology of a pilot user study and some initial results from the study will be presented, fol lowed by a brief discussion. Related ResearchVideo retrieval in the context of a digital library has only recently begun to be studied from a research perspective...

show abstract

“…In the last few decades, a number of commercial products and experimental prototype systems have been developed, including QBIC [23], SWIM [113], Photobook [72], Virage [26], VisualSEEK [85], Netra [47], MARS [61], and SIMPLIcity [103].…”

Section: Introductionmentioning

confidence: 99%

Exploring Mediatoil Imagery: A Content-based Approach

Saroop¹,

Viktor

McCurdy

et al. 2017

Proceedings of the 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management

View full text Add to dashboard Cite

The future of Alberta's bitumen sands, also known as "oil sands" or "tar sands," and their place in Canada's energy future has become a topic of much public debate. Within this debate, the print, television, and social media campaigns of those who both support and oppose developing the oil sands are particularly visible. As such, campaigns around the oil sands may be seen as influencing audience perceptions of the benefits and drawbacks of oil sands production. There is consequently a need to study the media materials of various tar sands stakeholders and explore how they differ. In this setting, it is essential to gather documents and identify content within images, which requires the use of an image retrieval technique such as a content-based image retrieval (CBIR) system. In a CBIR system, images are represented by low-level features (i.e. specific structures in the image such as points, edges, or objects), which are used to distinguish pictures from one another.The oil sands domain has to date not been mapped using CBIR systems. The research thus focuses on creating an image retrieval system, namely Mediatoil-IR, for exploring documents related to the oil sands. Our aim is to evaluate various low-level representations of the images within this context. To this end, our experimental framework employs LAB color histogram (LAB) and speeded up robust features (SURF) in order to typify the imagery. We further use machine learning techniques to improve the quality of retrieval (in terms of both accuracy and speed). To achieve this aim, the extracted features from each image are encoded in the form of vectors and used as a training set for learning classification models to organize pictures into different categories. Different algorithms were considered such as Linear SVM, Quadratic SVM, Weighted KNN, Decision Trees, Bagging, and Boosting on trees. It was shown that Quadratic SVM algorithm trained on SURF features is a good approach for building CBIR, and is used in building Mediatoil-IR.Finally, with the help of created CBIR, we were able to extract the similar documents and explore the different types of imagery used by different stakeholders. Our experimental evaluation shows that our Mediatoil-IR system is able to accurately explore the imagery used by different stakeholders.iii

show abstract

Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution

Cited by 98 publications

References 9 publications

Clustering of compressed illumination-invariant chromaticity signatures for efficient video summarization

Clustering of compressed illumination-invariant chromaticity signatures for efficient video summarization

Semantic visual features in content‐based video retrieval

Exploring Mediatoil Imagery: A Content-based Approach

Contact Info

Product

Resources

About