Inverted Index Compression for Scalable Image Matching

Chen, David M.; Tsai, Shiou‐Chuan; Chandrasekhar, Vijay; Takacs, Gabriel; Vedantham, Ramakrishna; Grzeszczuk, Radek; Girod, Bernd

doi:10.1109/dcc.2010.53

Cited by 43 publications

(27 citation statements)

References 19 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, visual search systems have been developed for applications such as product recognition [1,2] and landmark recognition [3]. In these systems, local image features [4,5,6] are extracted from images taken with a camera-phone and are matched to a large database using visual word indexing techniques [7,8]. Although current visual search technologies have reached a certain level of maturity, they have largely ignored a class of informative features often observed in images: text.…”

Section: Introductionmentioning

confidence: 99%

Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions

Chen

Tsai

Schroth

et al. 2011

2011 18th IEEE International Conference on Image Processing

Self Cite

357

185

View full text Add to dashboard Cite

Detecting text in natural images is an important prerequisite. In this paper, we propose a novel text detection algorithm, which employs edge-enhanced Maximally Stable Extremal Regions as basic letter candidates. These candidates are then filtered using geometric and stroke width information to exclude non-text objects. Letters are paired to identify text lines, which are subsequently separated into words. We evaluate our system using the ICDAR competition dataset and our mobile document database. The experimental results demonstrate the excellent performance of the proposed method.

show abstract

Section: Introductionmentioning

confidence: 99%

Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions

Chen

Tsai

Schroth

et al. 2011

2011 18th IEEE International Conference on Image Processing

Self Cite

357

185

View full text Add to dashboard Cite

show abstract

“…It largely outperforms directly sending the compact local descriptors (more than 5 KB in reported works). Their successive work in Chen et al (2010) further compressed the inverted indexing structure of visual vocabulary (Nister and Stewenius 2006) with arithmetic coding to reduce the memory and storage cost to maintain the scalable visual search system in server(s).…”

Section: State-of-the-art Mobile Landmark Search Frameworkmentioning

confidence: 99%

Location Discriminative Vocabulary Coding for Mobile Landmark Search

et al. 2011

View full text Add to dashboard Cite

With the popularization of mobile devices, recent years have witnessed an emerging potential for mobile landmark search. In this scenario, the user experience heavily depends on the efficiency of query transmission over a wireless link. As sending a query photo is time consuming, recent works have proposed to extract compact visual descriptors directly on the mobile end towards low bit rate transmission. Typically, these descriptors are extracted based solely on the visual content of a query, and the location cues from the mobile end are rarely exploited. In this paper, we present a Location Discriminative Vocabulary Coding (LDVC) scheme, which achieves extremely low bit rate query transmission, discriminative landmark description, as well as scalable descriptor delivery in a unified framework. Our first contribution is a compact and location discriminative visual landmark descriptor, which is offline learnt in two-step: First, we adopt spectral clustering to segment a city map into distinct geographical regions, where both visual and geographical similarities are fused to optimize the partition of cityscale geo-tagged photos. Second, we propose to learn LDVC in each region with two schemes: (1) a Ranking Sensitive PCA and (2) a Ranking Sensitive Vocabulary Boosting. Both schemes embed location cues to learn a compact descriptor, which minimizes the retrieval ranking loss by replacing the original high-dimensional signatures. Our second contribution is a location aware online vocabulary adaption: We store a single vocabulary in the mobile end, which is efficiently adapted for a region specific LDVC coding once a mobile device enters a given region. The learnt LDVC landmark descriptor is extremely compact (typically 10-50 bits with arithmetical coding) and performs superior over stateof-the-art descriptors. We implemented the framework in a real-world mobile landmark search prototype, which is validated in a million-scale landmark database covering typical areas e.g. Beijing, New York City, Lhasa, Singapore, and Florence.

show abstract

“…8(b) further shows the SS-0.5Million Evaluation in the 0.5 million Flickr data set. Similarly, there is a large margin from the learning codebook [16] [baseline (3)] and our final approach to the VT [1] [baseline (1)], adaptive vocabulary [16] [baseline (3)], and our alternative approaches [baseline (7)]. Note that baseline (3) outperforms our alternative approach [baseline (7)] with only visual discriminability embedding.…”

Section: ) Nd Image Retrieval In Ukbenchmentioning

confidence: 99%

“…Such a high dimensionality usually introduces obvious computational cost in both processing time (to match visual descriptors online) and storage space (to maintain the search model in the memory). This is extremely crucial for the state-ofthe-art mobile visual search scenario [6], [7], where mobile devices directly extract the BoW histogram and transmit over the wireless link. On the contrary, since the number of local features extracted from each image is limited (typically hundreds), each image is represented as a sparse histogram of nonzero codewords.…”

mentioning

confidence: 99%

Task-Dependent Visual-Codebook Compression

Yao

Liu

et al. 2012

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Abstract-A visual codebook serves as a fundamental component in many state-of-the-art computer vision systems. Most existing codebooks are built based on quantizing local feature descriptors extracted from training images. Subsequently, each image is represented as a high-dimensional bag-of-words histogram. Such highly redundant image description lacks efficiency in both storage and retrieval, in which only a few bins are nonzero and distributed sparsely. Furthermore, most existing codebooks are built based solely on the visual statistics of local descriptors, without considering the supervise labels coming from the subsequent recognition or classification tasks. In this paper, we propose a task-dependent codebook compression framework to handle the above two problems. First, we propose to learn a compression function to map an originally high-dimensional codebook into a compact codebook while maintaining its visual discriminability. This is achieved by a codeword sparse coding scheme with Lasso regression, which minimizes the descriptor distortions of training images after codebook compression. Second, we propose to adapt our codebook compression to the subsequent recognition or classification tasks. This is achieved by introducing a label constraint kernel (LCK) into our compression loss function. In particular, our LCK can model heterogeneous kinds of supervision, i.e., (partial) category labels, correlative semantic annotations, and image query logs. We validated our codebook compression in three computer vision tasks: 1) object recognition in PASCAL Visual Object Class 07; 2) near-duplicate image retrieval in UKBench; and 3) web image search in a collection of 0.5 million Flickr photographs. Our compressed codebook has shown superior performances over several state-of-the-art supervised and unsupervised codebooks.

show abstract

Inverted Index Compression for Scalable Image Matching

Abstract: Abstract

Cited by 43 publications

References 19 publications

Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions

Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions

Location Discriminative Vocabulary Coding for Mobile Landmark Search

Task-Dependent Visual-Codebook Compression

Contact Info

Product

Resources

About