Specialisation dynamics in federated web search

Abstract

Organising large-scale Web information retrieval systems into hierarchies of topic-specific search resources can improve both the quality of results and the efficient use of computing resources. A promising way to build such systems involves federations of topicspecific search engines in decentralised search environments. Most of the previous research concentrated on various technical aspects of such environments (e.g. routing of search queries or merging of results from multiple sources). We focus on organisational dynamics: what happens to topical specialisation of search engines in the absence of centralised control, when each engine makes individual and self-interested decisions on its service parameters? We investigate this question in a computational economics framework, where search providers compete for user queries by choosing what topics to index. We provide a formalisation of the competition problem and then analyse theoretically and empirically the specialisation dynamics of such systems.

Searching multimedia federated content web collections

“…Becarevic and Roantree (2004) studied federated multimedia database systems and Martin et al (2002), discuss federated rights management. Recent studies have explored the design and development of federated or distributed Web search (Avrahami et al , 2006; Callan, 2000; Khoossainov and Kushmerick, 2004; Si and Callan, 2005; Xu and Callan, 1998). Powell and Fox (1998) describe a scalable system for searching heterogeneous multilingual collections on the Web.…”