The Emotionally Intelligent Robot: Improving Social Navigation in Crowded Environments

Bera, Aniket; Randhavane, Tanmay; Prinja, Rohan; Kapsaskis, Kyra; Wang, Austin; Gray, Kurt; Manocha, Dinesh

doi:10.48550/arxiv.1903.03217

Cited by 6 publications

(10 citation statements)

References 38 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This separation between the robot and people can be limited by the accessibility distance, the user’s comfort distance, and the user’s emotion. Based on these features and the ability of robots to recognise moods or emotional states of people, robots can plan the best routes to follow [ 15 , 43 , 44 ].…”

Section: Related Workmentioning

confidence: 99%

“…Developing systems with this perspective makes the robot able to adapt to social groups of humans [ 12 ]. The detection of groups of people improves the navigation of a social robot in indoor and outdoor environments, and the detection of group emotions allows the robot to improve HRI, exhibiting acceptable social behaviour [ 13 , 14 , 15 , 16 ], as well as associating the group emotion with the scene in which the group is participating. Nevertheless, most existing studies related to detecting group emotions are based on third-person cameras [ 17 , 18 , 19 , 20 , 21 ], but their complexity makes them unsuitable for social robots with egocentric vision due to their sensory capacity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Group Emotion Detection Based on Social Robot Perception

Quiroz

Patiño

Díaz-Amado

et al. 2022

Sensors

View full text Add to dashboard Cite

Social robotics is an emerging area that is becoming present in social spaces, by introducing autonomous social robots. Social robots offer services, perform tasks, and interact with people in such social environments, demanding more efficient and complex Human–Robot Interaction (HRI) designs. A strategy to improve HRI is to provide robots with the capacity of detecting the emotions of the people around them to plan a trajectory, modify their behaviour, and generate an appropriate interaction with people based on the analysed information. However, in social environments in which it is common to find a group of persons, new approaches are needed in order to make robots able to recognise groups of people and the emotion of the groups, which can be also associated with a scene in which the group is participating. Some existing studies are focused on detecting group cohesion and the recognition of group emotions; nevertheless, these works do not focus on performing the recognition tasks from a robocentric perspective, considering the sensory capacity of robots. In this context, a system to recognise scenes in terms of groups of people, to then detect global (prevailing) emotions in a scene, is presented. The approach proposed to visualise and recognise emotions in typical HRI is based on the face size of people recognised by the robot during its navigation (face sizes decrease when the robot moves away from a group of people). On each frame of the video stream of the visual sensor, individual emotions are recognised based on the Visual Geometry Group (VGG) neural network pre-trained to recognise faces (VGGFace); then, to detect the emotion of the frame, individual emotions are aggregated with a fusion method, and consequently, to detect global (prevalent) emotion in the scene (group of people), the emotions of its constituent frames are also aggregated. Additionally, this work proposes a strategy to create datasets with images/videos in order to validate the estimation of emotions in scenes and personal emotions. Both datasets are generated in a simulated environment based on the Robot Operating System (ROS) from videos captured by robots through their sensory capabilities. Tests are performed in two simulated environments in ROS/Gazebo: a museum and a cafeteria. Results show that the accuracy in the detection of individual emotions is 99.79% and the detection of group emotion (scene emotion) in each frame is 90.84% and 89.78% in the cafeteria and the museum scenarios, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Group Emotion Detection Based on Social Robot Perception

Quiroz

Patiño

Díaz-Amado

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Their algorithm, however, requires a large number of 3D skeletal key-points to detect emotions and is limited to single individual cases. Bera et al [33] classify emotions based on facial expressions along with a pedestrian trajectory obtained from overhead cameras. Although this technique achieves good accuracy in predicting emotions from trajectories and facial expressions, it explicitly requires overhead cameras in its pipeline and does not work on a mobile robot equipped with only onboards cameras.…”

Section: B Emotion Modeling and Classificationmentioning

confidence: 99%

ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation

Narayanan¹,

Manoghar²,

Dorbala³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…have been used to model social behavior, affective behavior, and personality traits [59]. These approaches have also been used for robot navigation among pedestrians [10,52].…”

Section: Intelligent Virtual Agent Modelingmentioning

confidence: 99%

FVA:Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics

Randhavane

Bera

Kapsaskis

et al. 2019

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

Figure 1: Friendly Virtual Agent (FVA) : We present an algorithm to model perceived friendliness of a virtual agent by varying its gaze (A), gait (B), and gestures corresponding to head nodding (C) and waving (D). These movement cues are generated using our novel data-driven friendliness model. We evaluate the benefits in terms of an improved sense of social presence using an AR validation study using a HoloLens. ABSTRACTWe present a new approach for improving the friendliness and warmth of a virtual agent in an AR environment by generating appropriate movement characteristics. Our algorithm is based on a novel data-driven friendliness model that is computed using a userstudy and psychological characteristics. We use our model to control the movements corresponding to the gaits, gestures, and gazing of friendly virtual agents (FVAs) as they interact with the user's avatar and other agents in the environment. We have integrated FVA agents with an AR environment using with a Microsoft HoloLens. Our algorithm can generate plausible movements at interactive rates to increase the social presence. We also investigate the perception of * a user in an AR setting and observe that an FVA has a statistically significant improvement in terms of the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling. We observe an increment of 5.71% in the mean responses to a friendliness measure and an improvement of 4.03% in the mean responses to a social presence measure.

show abstract

The Emotionally Intelligent Robot: Improving Social Navigation in Crowded Environments

Cited by 6 publications

References 38 publications

Group Emotion Detection Based on Social Robot Perception

Group Emotion Detection Based on Social Robot Perception

ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation

FVA:Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics

Contact Info

Product

Resources

About