A Bio-inspired Motivational Decision Making System for Social Robots Based on the Perception of the User

Abstract

Nowadays, many robotic applications require robots making their own decisions and adapting to different conditions and users. This work presents a biologically inspired decision making system, based on drives, motivations, wellbeing, and self-learning, that governs the behavior of the robot considering both internal and external circumstances. In this paper we state the biological foundations that drove the design of the system, as well as how it has been implemented in a real robot. Following a homeostatic approach, the ultimate goal of the robot is to keep its wellbeing as high as possible. In order to achieve this goal, our decision making system uses learning mechanisms to assess the best action to execute at any moment. Considering that the proposed system has been implemented in a real social robot, human-robot interaction is of paramount importance and the learned behaviors of the robot are oriented to foster the interactions with the user. The operation of the system is shown in a scenario where the robot Mini plays games with a user. In this context, we have included a robust user detection mechanism tailored for short distance interactions. After the learning phase, the robot has learned how to lead the user to interact with it in a natural way.

Reinforcement Learning Approaches in Social Robotics

“…Q-learning, along with its different variations, is the most commonly used RL method in social robotics. The studies using Q-learning are [ 3 , 13 , 34 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 ]. These comprise studies using standard Q-learning [ 3 , 54 , 55 , 58 , 60 , 62 ], studies modify Q-learning for dealing with delayed reward [ 52 ], tuning the parameters for Q-learning such as [ 13 , 34 , 52 ], dealing with decreasing human feedback over time [ 52 ], comparing their proposed algorithm with Q-learning [ 33 , 49 , 61 , 63 , 64 ], variation of Q-learning called Object Q-learning [ 64 , 65 , 66 ], combining Q-learning with fuzzy inference [ 67 ], SARSA [ 68 , 69 ], TD( ) [ 70 ], MAXQ [ 33 , 71 , 72 ], R-learning [ 32 ], and Deep Q-learning [ 35 , 36 , 73 , 74 ].…”

“…There are different intrinsic motivation models within the RL framework [ 20 ]. However, in social robotics, the idea of maintaining the internal needs of the robot (detailed in Section 5.2 ) has received much attention [ 13 , 34 , 63 , 64 , 65 , 66 , 108 ]. One exception is [ 74 ], in which prediction error of social event occurrences was used as intrinsic motivation.…”

“…In the same manner, the body maintains its internal equilibrium through a variety of voluntary and involuntary processes and behaviors. The homeostasis-based RL in social robotics is presented in [ 13 , 34 , 64 , 65 , 66 , 108 ]. These studies introduced a biologically inspired approach that depends on homeostasis.…”

Smart Environments and Social Robots for Age-Friendly Integrated Care Services

“…In the case of the care personnel the impact was on dimensions such as the working atmosphere, the professional development and the competences, while in the case of the older adults the impact was on the physical activities, the older adults' interaction and the sensory experiences. Authors of [108] consider a mini robot as support for a motivational decision-making system (DMS). The stimulating and the improvement of the interaction of the robot with the users is considered from various perspectives such as the performance of cognitive exercises and the performance of educational games.…”

Modelling Multimodal Dialogues for Social Robots Using Communicative Acts

“…The architecture of this robot works as shown in Figure 2 . At the top, a Decision Making System (DMS) [ 33 ] controls all high level decisions and selects which of the robot’s applications has to be activated at any time. These applications provide all of the robot’s functionalities.…”