SEVA

Abstract

In this paper, we study how a sensor-rich world can be exploited by digital recording devices such as cameras and camcorders to improve a user's ability to search through a large repository of image and video files. We design and implement a digital recording system that records identities and locations of objects (as advertised by their sensors) along with visual images (as recorded by a camera). The process, which we refer to as sensor-enhanced video annotation (SEVA), combines a series of correlation, interpolation, and extrapolation techniques. It produces a tagged stream that later can be used to efficiently search for videos or frames containing particular objects or people. We present detailed experiments with a prototype of our system using both stationary and mobile objects as well as GPS and ultrasound. Our experiments show that: (i) SEVA has zero error rates for static objects, except very close to the boundary of the viewable area; (ii) for moving objects or a moving camera, SEVA only misses objects leaving or entering the viewable area by 1-2 frames; (iii) SEVA can scale to 10 fast moving objects using current sensor technology; and (iv) SEVA runs online using relatively inexpensive hardware.

Movies tags extraction using deep learning

“…The video annotation technique presented in [14] utilizes the redundancy among YouTube videos to find connection among videos and propagating tags among similar videos. The techniques presented in [13] [7] utilize the contextual meta-data acquired from the sensors on smart phones to generate video tags. In [9], the proposed technique recognizes basic objects in images and videos of a digital camera and extracts the meta data including geographical and date/time information to generate tags.…”

Client-Side Relevance Feedback Approach for Image Retrieval in Mobile Environment

“…This is not consistent with the reality: a semantic concept can have very different representation (for example, side view of a sedan and front view of a sedan can be very different). Trying to bridge the gap, some mobile information management systems use tags generated at the point of picture was taken: In Davis et al (2004), Liu et al (2005), and Lahti et al (2005), location, temporal, and sometimes social contextual metadata are used as image features instead of the visual features. However, a picture is worth a thousand words.…”