Estimating probabilities for effective data fusion

The weighted Condorcet fusion in information retrieval

“…Data fusion (also known as meta-search) in information retrieval has been investigated by many researchers and quite a few data fusion algorithms such as CombSum (Fox, Koushik, Shaw, Modlin, & Rao, 1993;Fox & Shaw, 1994), CombMNZ (Fox et al, 1993;Fox & Shaw, 1994), the linear combination methods (Vogt & Cottrell, 1998, 1999Wu, 2012b), the Borda count (Aslam & Montague, 2001), the Bayesian fusion (Aslam & Montague, 2001), the Condorcet fusion (Montague & Aslam, 2002), the correlation methods (Wu & McClean, 2006a), the probabilistic fusion (Lillis, Toolan, Collier, & Dunnion, 2006), MAPFuse (Lillis et al, 2010) and others have been proposed and extensive experimentation has been conducted to evaluate these algorithms.…”

“…Then methods such as CombSum (Fox et al, 1993;Fox & Shaw, 1994), CombMNZ (Fox et al, 1993;Fox & Shaw, 1994), the linear combination method (Bartell, Cottrell, & Belew, 1994;Thompson, 1993;Vogt & Cottrell, 1998;Vogt & Cottrell, 1999;Wu, 2012b;Wu, Bi, Zeng, & Han, 2009), the correlation methods (Wu & McClean, 2006a) are score-based methods. Methods such as the Borda count (Aslam & Montague, 2001), the Condorcet fusion (Montague & Aslam, 2002), the probabilistic fusion (Lillis et al, 2006), MAPFuse (Lillis et al, 2010) are ranking-based methods. The Bayesian fusion (Aslam & Montague, 2001) is a little special and it can go either way because it may use scores or ranking information.…”

“…MAPFuse (Lillis et al, 2010) also converts ranking information into scores with the help of some training data. In MAPFuse, a document at rank t is assigned a score of v map /t, where v map is the MAP value of the system in question over a group of training queries using a collection of training documents.…”

Query sampling for learning data fusion

“…Supervised methods, for example, SegFuse [20], SlideFuse [14], PosFuse [13] and PLQA [23], get a better approximation of the score based on the given training data, which shows information about how well the models performed for some given queries before. With such training data, supervised methods can further estimate the quality of the evidences such as the accuracy of a rank position for a specific model [13] and the weight of a model [15], and then adjust the ranking score accordingly.…”

“…Previous work on supervised methods mostly used all of the training data to learn their fusion models [3,13,15]. They assumed ineffective training data was tolerable and could be remedied via data fusion in the process of document retrieval.…”