Methods and techniques for the evaluation of user-adaptive systems

Abstract

This article presents a comprehensive overview of methods and techniques used for the evaluation of user-adaptive systems. It describes the methodologies derived both from the evaluation of human–computer interaction systems and from information retrieval and information filtering systems by giving examples of the application of these methodologies in the user-adaptive systems. The state of the art and the main results in the evaluation of these systems are reported. In particular, empirical evaluation and layered approaches are discussed in detail. Finally, focus on less explored methodologies, such as qualitative approaches (e.g. Grounded Theory), is proposed.

The evaluation of a social adaptive website for cultural events

“…Data collected when a large number of real users do real tasks in a real context of use are very relevant to the evaluation of adaptive systems. Many authors have emphasized the importance of studies involving real users in the real context of usage as well as the lack of case studies reported in the literature [Chin, 2001, Weibelzahl, 2003, Gena, 2005. Moreover the analysis of the real behavior of the users under real conditions makes it possible to analyze the facts more in depth rather than under experimental conditions, wherein the ultimate goal is to explain the cause-effect relationships between variables and make generalizations on the basis of the obtained results.…”

User Profiling - A Short Review

“…In the explicit method, information regarding to the user's interest and preferences is provided explicitly by the user to the system. The downside of this method is that the explicit profiles have a static nature and are valid only until the user changes their interest and preferences parameters explicitly [6]. Explicit information gathering methods are used by the static profiling that analyzes the static and predictable characteristics of the user.…”

EVALUATION OF A USER-ADAPTED SPOKEN LANGUAGE DIALOGUE SYSTEM - Measuring the Relevance of the Contextual Information Sources

“…Several evaluation methodologies have been proposed for measuring the relevance of a user model in different research fields (Chin, 2001;Gena, 2005). However, it is difficult to find performance figures from real-world applications that can be extrapolated to other systems or be worldwide accepted, as all of them are directly related to an specific dialogue system.…”