MoBeTrack: A Toolkit to Analyze User Experience of Mobile Apps in the Wild

Generosi, Andrea; Altieri, Alex; Ceccacci, Silvia; Foresi, G.; Talipu, Abudukaiyoumu; Turri, Giacomo; Mengoni, Maura; Giraldi, Luca

doi:10.1109/icce.2019.8662020

Cited by 10 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the best of our knowledge, no studies so far have proposed the integrated use of gaze and emotion recognition systems, based on deep learning algorithms, to support the collection of relevant information useful for UX assessment of laptop and desktop web applications nor to power recommending systems. Only [11] proposed a system to support UX assessment based deep learning, but such a solution works only for mobile applications. While only [7] proposed to process data provided by a common pc webcam to enable users' gaze and emotion detection in order to manage an intelligent e-commerce recommendation system: it proposed respectively to use SVM algorithm to enable emotion recognition and a gradient-based method to perform gaze tracking.…”

Section: Research Backgroundmentioning

confidence: 99%

“…For the proposed toolkit a centralized architecture has been designed. This kind of architecture has been inherited from the system described in [11] and adapted to be used in synergy with web platforms instead of mobile applications. In Figure 1 are shown the main components: the Web Plugin and the Deep Learning Platform (DLP), respectively the frontend and the backend sides.…”

Section: System Architecturementioning

confidence: 99%

“…The use of cameras is so pervasive that we no longer worry about their presence, and we are not fully aware of how much information can be collected through them. For example, using appropriate systems, it is possible to process video images so that demographic (e.g., gender and age) and behavioral (e.g., emotions, eye movements) information can be easily extracted [10,11]. This opens up the possibility of using technologies that until now had been restricted to laboratories to analyze UX in the wild.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Toolkit for the Automatic Analysis of Human Behavior in HCI Applications in the Wild

Generosi¹,

Ceccacci²,

Faggiano³

et al. 2020

Adv. sci. technol. eng. syst. j.

View full text Add to dashboard Cite

Nowadays, smartphones and laptops equipped with cameras have become an integral part of our daily lives. The pervasive use of cameras enables the collection of an enormous amount of data, which can be easily extracted through video images processing. This opens up the possibility of using technologies that until now had been restricted to laboratories, such as eye-tracking and emotion analysis systems, to analyze users' behavior in the wild, during the interaction with websites. In this context, this paper introduces a toolkit that takes advantage of deep learning algorithms to monitor user's behavior and emotions, through the acquisition of facial expression and eye gaze from the video captured by the webcam of the device used to navigate the web, in compliance with the EU General data protection regulation (GDPR). Collected data are potentially useful to support user experience assessment of web-based applications in the wild and to improve the effectiveness of e-commerce recommendation systems.

show abstract

Section: Research Backgroundmentioning

confidence: 99%

Section: System Architecturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Toolkit for the Automatic Analysis of Human Behavior in HCI Applications in the Wild

Generosi¹,

Ceccacci²,

Faggiano³

et al. 2020

Adv. sci. technol. eng. syst. j.

View full text Add to dashboard Cite

show abstract

“…Most of these systems process data using AI, usually with CNNs [68,69] to predict primary emotions in Ekman and Fresner's framework (as shown in Section 2.1). In fact, most of the currently available facial expressions datasets are based on this emotion framework (e.g., EmotioNet [70]).…”

Section: Emotion Recognition Technologies In the Vehiclementioning

confidence: 99%

On Driver Behavior Recognition for Increased Safety: A Roadmap

Davoli

Martalò

Cilfone

et al. 2020

Safety

View full text Add to dashboard Cite

Advanced Driver-Assistance Systems (ADASs) are used for increasing safety in the automotive domain, yet current ADASs notably operate without taking into account drivers’ states, e.g., whether she/he is emotionally apt to drive. In this paper, we first review the state-of-the-art of emotional and cognitive analysis for ADAS: we consider psychological models, the sensors needed for capturing physiological signals, and the typical algorithms used for human emotion classification. Our investigation highlights a lack of advanced Driver Monitoring Systems (DMSs) for ADASs, which could increase driving quality and security for both drivers and passengers. We then provide our view on a novel perception architecture for driver monitoring, built around the concept of Driver Complex State (DCS). DCS relies on multiple non-obtrusive sensors and Artificial Intelligence (AI) for uncovering the driver state and uses it to implement innovative Human–Machine Interface (HMI) functionalities. This concept will be implemented and validated in the recently EU-funded NextPerception project, which is briefly introduced.

show abstract

“…For this reason, this research area is starting to focus on non-intrusive devices in the last year to automatically recognize human emotions, particularly for speech and facial coding analysis. Most of the facial expression recognition systems today make use of Deep Neural Networks (especially Convolutional Neural Networks), like the one presented by (Generosi et al, 2018;Generosi et al, 2019), taking pictures of human faces in input and providing a prediction of the relative Ekman's primary emotions (i.e., happiness, surprise, sadness, anger, disgust, and fear) (Ekman and Friesen, 1978), just like most of the state-of-the-art systems that involve this kind of technologies (Li et al, 2018). The literature proposes different emotion-aware car systems, some using voice analysis (Jones and Jonsson, 2007) others wearable devices (Nasoz et al, 2010;Katsis et al, 2008).…”

Section: Introductionmentioning

confidence: 99%

Designing in-car emotion-aware automation

Ceccacci¹

2021

European Transport/Trasporti Europei

View full text Add to dashboard Cite

Driver behaviour recognition is of paramount importance for in-car automation assistance. It is widely recognized that not only attentional states, but also emotional ones have an impact on the safety of the driving behaviour. This research work proposes an emotion-aware in-car architecture where it is possible to adapt driver’s emotions to the vehicle dynamics, investigating the correlations between negative emotional states and driving performances, and suggesting a system to regulate the driver’s engagement through a unique user experience (e.g. using music, LED lighting) in the car cabin. The relationship between altered emotional states induced through auditory stimuli and vehicle dynamics is investigated in a driving simulator. The results confirm the need for both types of information to improve the robustness of the driver state recognition function and open up the possibility that auditory stimuli can modify driving performance somehow.

show abstract

MoBeTrack: A Toolkit to Analyze User Experience of Mobile Apps in the Wild

Cited by 10 publications

References 8 publications

A Toolkit for the Automatic Analysis of Human Behavior in HCI Applications in the Wild

A Toolkit for the Automatic Analysis of Human Behavior in HCI Applications in the Wild

On Driver Behavior Recognition for Increased Safety: A Roadmap

Designing in-car emotion-aware automation

Contact Info

Product

Resources

About