Analysis of facial expressions in response to basic taste stimuli using artificial intelligence to predict perceived hedonic ratings

Abstract

Taste stimuli can induce a variety of physiological reactions depending on the quality and/or hedonics (overall pleasure) of tastants, for which objective methods have long been desired. In this study, we used artificial intelligence (AI) technology to analyze facial expressions with the aim of assessing its utility as an objective method for the evaluation of food and beverage hedonics compared with conventional subjective (perceived) evaluation methods. The face of each participant (10 females; age range, 21–22 years) was photographed using a smartphone camera a few seconds after drinking 10 different solutions containing five basic tastes with different hedonic tones. Each image was then uploaded to an AI application to achieve outcomes for eight emotions (surprise, happiness, fear, neutral, disgust, sadness, anger, and embarrassment), with scores ranging from 0 to 100. For perceived evaluations, each participant also rated the hedonics of each solution from –10 (extremely unpleasant) to +10 (extremely pleasant). Based on these, we then conducted a multiple linear regression analysis to obtain a formula to predict perceived hedonic ratings. The applicability of the formula was examined by combining the emotion scores with another 11 taste solutions obtained from another 12 participants of both genders (age range, 22–59 years). The predicted hedonic ratings showed good correlation and concordance with the perceived ratings. To our knowledge, this is the first study to demonstrate a model that enables the prediction of hedonic ratings based on emotional facial expressions to food and beverage stimuli.

Towards Foodservice Robotics: A Taxonomy of Actions of Foodservice Workers and a Critical Review of Supportive Technology

“…To conclude, it is worth noting the relevance (especially for the design of emotionally intelligent/responsive FS robots) of the past literature on Machine Learning [257] and Deep Learning [258] [259] strategies of facial expressions recognition. In addition, recent studies (under food emotion research) are of particular interest for FS robotics, for they have analyzed facial expressions (recognized with CV software applications like FaceReader™) and related these with food and beverage taste ratings [260] [261], identified ethnic differences in various facial expressions resulting from a same food tasting [262], and showed that facial expressions reflect the level of food-specific satiety [263]. These studies not only reveal the viability and usefulness of monitoring human facial expressions for robots that must interact with human FS workers/clients (e.g., to receive feedback on food prepared by the robot), but also give specific insights on how robots can interpretate facial expressions resulting from food stimuli.…”