Utilising Information Foraging Theory for User Interaction with Image Query Auto-Completion

Abstract

Query Auto-completion (QAC) is a prominently used feature in search engines, where user interaction with such explicit feature is facilitated by the possible automatic suggestion of queries based on a prefix typed by the user. Existing QAC models have pursued a little on user interaction and cannot capture a user's information need (IN) context. In this work, we devise a new task of QAC applied on an image for estimating patch (one of the key components of Information Foraging Theory) probabilities for query suggestion. Our work supports query completion by extending a user query prefix (one or two characters) to a complete query utilising a foraging-based probabilistic patch selection model. We present iBERT, to fine-tune the BERT (Bidirectional Encoder Representations from Transformers) model, which leverages combined textual-image queries for a solution to image QAC by computing probabilities of a large set of image patches. The reflected patch probabilities are used for selection while being agnostic to changing information need or contextual mechanisms. Experimental results show that query autocompletion using both natural language queries and images is more effective than using only language-level queries. Also, our fine-tuned iBERT model allows to efficiently rank patches in the image.

Information Foraging on Social Media Using Elephant Herding Optimization

“…At the end of each generation, the elephant with the worst fitness value will leave the clan. This behavior is implemented using formula (7).…”

“…Techniques and approaches such as game theory [2], deep learning [3], multi-agent systems [4], ontologies [5] and bio-inspired computing [6] were employed for this purpose. Besides, information foraging was applied to many domains like query auto-completion [7], recommender systems [8] and cyberattack prediction [9]. More recently, some works focused on tackling information foraging on social media [10].…”

TIPS: Time-aware Personalised Semantic-based query auto-completion

“…Also, Mitra [38] introduced a distributed representation of queries using CLSM. Jaiswal et al [39] proposed the first method for image QAC by extending the LSTM language model. Kannadasan and Aslanyan [40] represented a method for personalised QAC employing lightweight embeddings learned through fastText.…”

Information foraging with an oracle

“…From an information foraging perspective, studies A and B reflect the difference between one environment in which food sources are disposed randomly and another (more ecologically valid) environment in which they follow a structure, e.g., some areas are richer in nutrients than others, which foragers can learn through exploration [32][33][34][35][36][37]. Here, we add to this large body of literature the availability of an external information source-the oracle-that can be used in combination with or instead of standard exploration.…”