Measuring the likelihood property of scoring functions in general retrieval models

Abstract

Although retrieval systems based on probabilistic models will rank the objects (e.g. documents) being retrieved according to the probability of some matching criterion (e.g. relevance) they rarely yield an actual probability and the scoring function is interpreted to be purely ordinal within a given retrieval task. In this paper it is shown that some scoring functions possess the likelihood property, which means that the scoring function indicates the likelihood of matching when compared to other retrieval tasks which is potentially more useful than pure tanking although it cannot be interpreted as an actual probability. This property can be detected by using two modified effectiveness measure, entire precision and entire recall. Empirical evidence is offered to show the existence of this property both for traditional document retrieval and for analysis of crime data where suspects of an unsolved crime are ranked according to probability of culpability.