User-Centered Predictive Model for Improving Cultural Heritage Augmented Reality Applications: An HMM-Based Approach for Eye-Tracking Data

Abstract

Today, museum visits are perceived as an opportunity for individuals to explore and make up their own minds. The increasing technical capabilities of Augmented Reality (AR) technology have raised audience expectations, advancing the use of mobile AR in cultural heritage (CH) settings. Hence, there is the need to define a criteria, based on users’ preference, able to drive developers and insiders toward a more conscious development of AR-based applications. Starting from previous research (performed to define a protocol for understanding the visual behaviour of subjects looking at paintings), this paper introduces a truly predictive model of the museum visitor’s visual behaviour, measured by an eye tracker. A Hidden Markov Model (HMM) approach is presented, able to predict users’ attention in front of a painting. Furthermore, this research compares users’ behaviour between adults and children, expanding the results to different kind of users, thus providing a reliable approach to eye trajectories. Tests have been conducted defining areas of interest (AOI) and observing the most visited ones, attempting the prediction of subsequent transitions between AOIs. The results demonstrate the effectiveness and suitability of our approach, with performance evaluation values that exceed 90%.

A Visual Attentive Model for Discovering Patterns in Eye-Tracking Data—A Proposal in Cultural Heritage

“…The approach presented in Reference [15] has been extended for the development of the proposed VAM. The framework is schematically depicted in Figure 1 and comprises three main components-the eye-tracking data acquisition, the Coordinates-Based View Transformation, the museums' visitors classification.…”

“…Given the above, it can be stated that some studies proposing physical markers have been attached to each area of interest (AOI) for identifying them, while others, for the detection of many objects, report large scale variations entailed in a lower detection rate. Referring to our experiments, we decided to collect data without AOI constrains, supported by our previous research [15].…”

“…This provides new answers to predict the museums' visitor characteristics. The main contributions of this paper, aside from extending the system and the analysis presented in References [14,15], are (i) the proposal of a novel VAM to help identify the type of user in front of paintings; (ii) the collection and analysis of a real eye-tracking CH dataset for deep learning purposes that is public to all researchers and also useful to art researchers and artists alike (http://vrai.dii.univpm.it/content/eye-tracking-dataset-ch); (iii) an in-depth analysis to investigate whether there is a positive attentional bias when looking at similar paintings, and (iv) a real environment test with results that prove the main goals of the paper mentioned above.…”

“…In this paper, we have proposed a novel Visual Attentive Model (VAM) that is learned from eye tracking data. As demonstrated by our previous work in this field [14,15], the exploitation of a user's gaze proved to be particularly useful for the development of Augmented Reality applications [16]. In fact, while the current practice for the implementation of such kinds of multimedia experience is to deliver applications from the domain expert's point of view, it has been proved that a user-centered design is more effective, with the contents tailored according to the user's preferences.…”

Considered Factors of Online News Based on Respondents’ Eye Activity Using Eye-Tracker Analysis

“…Because observers pay attention to the object by looking at it, it seems that the most natural choice from the users' point of view must be based on devices that detect their gaze. Many aspects are considered in tracking eye movements, and there are several ways to implement eye-trackers in several areas such as in Augmented Reality (AR) applications for cultural heritage [30], user support systems [31], gameplay methods in shooter games [32], and so forth.…”