Recall and precision are often used to evaluate the effectiveness of information retrieval systems. They are easy to define if there is a single query and if the retrieval result generated for the query is a linear ordering. However, when the retrieval results are weakly ordered, in the sense that several documents have an identical retrieval status value with respect to a query, some probabilistic notion of precision has to be introduced. Relevance probability, expected precision, and so forth, are some alternatives mentioned in the literature for this purpose. Furthermore, when many queries are to be evaluated and the retrieval results averaged over these queries, some method of interpolation of precision values at certain preselected recall levels is needed. The currently popular approaches for handling both a weak ordering and interpolation are found to be inconsistent, and the results obtained are not easy to interpret. Moreover, in cases where some alternatives are available, no comparative analysis that would facilitate the selection of a particular strategy has been provided. In this paper, we systematically investigate the various problems and issues associated with the use of recall and precision as measures of retrieval system performance. Our motivation is to provide a comparative analysis of methods available for defining precision in a probabilistic sense and to promote a better understanding of the various issues involved in retrieval performance evaluation.

There are important theoretical and practical reasons to study belief structures. Similar to qualitative probability, qualitative belief can also be described in terms of a preference relation. One of the objectives in this study is to specify the precise conditions that a preference relation must satisfy such that it can be faithfully represented by a belief function. Two special classes of preference relations identified are weak and strict belief relations. It is shown that only strict belief relations are consistent (compatible) with belief functions. More importantly, the axiomatization of qualitative belief provides a foundation to develop a utility theory for decision making based on belief functions. The established relationship between qualitative probability and qualitative belief may also lead to a better understanding and useful applications of belief structures in approximate reasoning.

No abstract

Linear decision (retrieval)functions have been widely adopted in information retrieval systems such as in Boolean, vector, and probabilistic models. Based on measurement theory, adaptive linear retrieval models are proposed in this paper. A necessary and sufficient condition for the existence of a linear decision function is given. By ann inductive learning (feedback)process, techniques in linear integer programming can be directly applied to estimate parameters for automatic query formulation.

Over the last decade many software metrics have been introduced by researchers and many software tools have been developed using software metrics to measure the "quality" of programs. These metrics for measuring productivity, reliability, maintainability, and complexity, for example, are vital to software development planning and management. In this paper a new approach is presented to describe the properties of the software metrics and their scales using measurement theory. Methods are shown to describe a software complexity metric as an ordinal, an interval or a ratio scale. The use of this concept is shown by application to the Metric of McCabe. These results are very important for selecting appropriate software metrics for software measurement and for developing tools which use software metrics to evaluate the "quality" of software.