Sketch-based Image Retrieval via Shape Words

Xiao, Changcheng; Wang, Changhu; Zhang, Liqing; Zhang, Lei

doi:10.1145/2671188.2749360

Cited by 24 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the images differ by the addition of a watermark, and by being reversed. right) A screen capture from [36] showing a query-by-sketching system. While Google image search is able to find slightly corrupted version of images, it cannot find semantically similar images.…”

Section: Why Natural Language For Time Series?mentioning

confidence: 99%

“…One could imagine a more sophisticated query-by-sketching system. Figure 3.right shows one of the most highly cited examples [36]. However, it is not clear that this system would retrieve either the second image in Figure 2 (occluded wheel) or the third image in Figure 2 (the bike is viewed from the rear).…”

Section: Why Natural Language For Time Series?mentioning

confidence: 99%

“…However, it is not clear that this system would retrieve either the second image in Figure 2 (occluded wheel) or the third image in Figure 2 (the bike is viewed from the rear). Moreover, while most people could draw a bicycle in its canonical side view, clearly drawing and indexing the concept "barefoot girl" is beyond the intended scope of [36], and beyond the artistic ability of most people.…”

Section: Why Natural Language For Time Series?mentioning

confidence: 99%

See 2 more Smart Citations

Putting the Human in the Time Series Analytics Loop

Imani

Alaee

Keogh

2019

Companion Proceedings of the 2019 World Wide Web Conference

View full text Add to dashboard Cite

Time series are one of the most common data types in nature. Given this fact, there are dozens of query-by-sketching/ queryby-example/ query-algebra systems proposed to allow users to search large time series collections. However, none of these systems have seen widespread adoption. We argue that there are two reasons why this is so. The first reason is that these systems are often complex and unintuitive, requiring the user to understand complex syntax/interfaces to construct high-quality queries. The second reason is less well appreciated. The expressiveness of most query-by-content systems is surprisingly limited. There are well defined, simple queries that cannot be answered by any current query-by-content system, even if it uses a state-of-the-art distance measure such as Dynamic Time Warping. In this work, we propose a natural language search mechanism for searching time series. We show that our system is expressive, intuitive, and requires little space and time overhead. Because our system is text-based, it can leverage decades of research text retrieval, including ideas such as relevance feedback. Moreover, we show that our system subsumes both motif/discord discovery and most existing queryby-content systems in the literature. We demonstrate the utility of our system with case studies in domains as diverse as animal motion studies, medicine and industry.

show abstract

Section: Why Natural Language For Time Series?mentioning

confidence: 99%

Section: Why Natural Language For Time Series?mentioning

confidence: 99%

Section: Why Natural Language For Time Series?mentioning

confidence: 99%

See 1 more Smart Citation

Putting the Human in the Time Series Analytics Loop

Imani

Alaee

Keogh

2019

Companion Proceedings of the 2019 World Wide Web Conference

View full text Add to dashboard Cite

show abstract

“…Apart from images, sketches have been suc-cessfully used for 3D shape retrieval purpose as well [44,38]. Since sketch is more close to the semantic representation, it tends to help retrieving target results in the user mind from a semantic perspective [37]. Here, the main challenge is to bridge the domain gap between sketches and natural images.…”

Section: Related Workmentioning

confidence: 99%

Aligning Salient Objects to Queries: A Multi-modal and Multi-object Image Retrieval Framework

Dey

Dutta

Ghosh

et al. 2019

Computer Vision – ACCV 2018

View full text Add to dashboard Cite

In this paper we propose an approach for multi-modal image retrieval in multi-labelled images. A multi-modal deep network architecture is formulated to jointly model sketches and text as input query modalities into a common embedding space, which is then further aligned with the image feature space. Our architecture also relies on a salient object detection through a supervised LSTM-based visual attention model learned from convolutional features. Both the alignment between the queries and the image and the supervision of the attention on the images are obtained by generalizing the Hungarian Algorithm using different loss functions. This permits encoding the object-based features and its alignment with the query irrespective of the availability of the co-occurrence of different objects in the training set. We validate the performance of our approach on standard single/multi-object datasets, showing state-of-the art performance in every dataset.

show abstract

“…Furthermore, with the explosive growth of touch screens, sketch-based images can be retrieved easily, which are also a form of contour-based representation. Hand-drawn shapes are well studied by researchers for sketch parsing [26], image retrieval [27], face sketch synthesis [28] and photo matching [29]. Those researches proved that contour-based images can achieve an effective reconstructive efficiency and a superior performance in discrimination.…”

Section: Introductionmentioning

confidence: 99%

Using a Vertical-Stream Variational Auto-Encoder to Generate Segment-Based Images and Its Biological Plausibility for Modelling the Visual Pathways

2019

View full text Add to dashboard Cite

Human beings have a strong capability to identify objects in different viewpoints. Unlike computer vision that requires sufficient training samples in various scales and rotations, biological visual systems can efficiently recognize objects in diverse spatial states. To achieve this objective, images are processed into a segment-based representation and then a vertical stream variational auto-encoder (VSVAE) is utilized to generate images based on the preprocessed segments in this study. The novel structure of the two vertical streams can be also considered as a computational model for the interaction between the ventral pathway and the dorsal pathway in the visual cortex. The reconstructive capability of the VSVAE is testified by using a series of geometric information sets to enhance the segment-based representation. By visualizing the learnt features in the hidden layers of VSVAE, the biological plausibility of the model is discussed. In addition, the proposed methodology is able to facilitate the classification accuracy, especially when the images are severely transformed.

show abstract

Sketch-based Image Retrieval via Shape Words

Cited by 24 publications

References 8 publications

Putting the Human in the Time Series Analytics Loop

Putting the Human in the Time Series Analytics Loop

Aligning Salient Objects to Queries: A Multi-modal and Multi-object Image Retrieval Framework

Using a Vertical-Stream Variational Auto-Encoder to Generate Segment-Based Images and Its Biological Plausibility for Modelling the Visual Pathways

Contact Info

Product

Resources

About