SEVQER: Automatic Semantic Visual Query Builder to Support Intelligent Image Search in Traffic Images

Abstract

Image search is a challenging process in the field of Content Based Image Retrieval (CBIR). Image search-by-example, search-bykeyword and search-by-sketch methods seldom provide user interface that allows user to accurately formulate their search intent easily. To overcome such issue, a novel image search interface-Semantic Visual Query Builder (SeVQer) is proposed as a non-verbal interface which allows user to drag and drop from the image data provided to formulate user query. The drag and drop mechanism minimizes the difficulty of verbalizing query image into keywords or sketching a correct drawing of the query image. SeVQer was implemented and compared with 3 image search methods (search-by-example, search-by-keyword and search-by-sketch) in terms of task completion time and user satisfaction using traffic images. SeVQer achieved statistically significant lower task completion time with an average of 28 sec, a promising 50% reduction than search-by-sketch (average of 56 sec). The significance of this work is two-fold: the SeVQer user interface allows user to easily formulate intent specific query, while the novel architecture and methodology reduces the semantic gap in general.