Real-Time Robotic Mirrored Behavior of Facial Expressions and Head Motions Based on Lightweight Networks

Abstract

The ability of a humanoid robot to imitate facial expressions with simultaneous head motions is crucial to natural human-robot interaction. This mirrored behavior from human beings to humanoid robots has high demands of similarity and real-time performance. To fulfill these needs, this paper proposes a real-time robotic mirrored behavior of facial expressions and head motions based on lightweight networks. First, a humanoid robot that can change the state of its facial organs and neck through servo displacement is developed to achieve the mirrored behavior of facial expressions and head motions. Second, to overcome the high latency caused by deep learning models running in embedded devices, a lightweight deep learning network is constructed for detecting facial feature points, which can reduce model size and improve running speed without affecting the performance of the model. Finally, a mapping relationship of 68 facial feature points to optimal servo displacements is established to realize the mirrored behavior from human beings to humanoid robots. The experimental results show that the facial feature point recognition method based on lightweight model performs better than other state-of-the-art methods, and our head motion tracking method can maintain high accuracy compared with the gold standard optical motion capture system NOKOV. Overall, our method ensures the accurate and real-time generation of robot mirrored behavior and has a certain reference value for the efficient and natural interaction between humans and robots.

Principles and Methods of Servomotor Control: Comparative Analysis and Applications

“…Thanks to integration with Industry 4.0, new opportunities have emerged in predictive management and seamless communication of production processes and mechanisms. Based on [16,[73][74][75][76][77], it is possible to identify the key aspects in the construction of this principle and draw conclusions about its advantages and disadvantages.…”

“…Various technologies for servomotor control are currently employed, including digital signal processing systems, feedback systems, field-oriented control, and control systems integrated with Industry 4.0. All these types of servomotor control are used to achieve different results, each having its own merits and drawbacks, which are discussed in these works [15][16][17][18][19][20][21].…”

FastNet: A Lightweight Convolutional Neural Network for Tumors Fast Identification in Mobile-Computer-Assisted Devices

“…Depthwise separable convolutions [16], group convolution [17] and parallel one-dimensional depthwise convolutions [18] are usually used to instead of standard convolutions to reduce parameters. Some lightweight convolutional networks, combining cloud-edge computing with AI technology, are proposed to meet the real-time requirement [19], [20].…”

Human Impression of Humanoid Robots Mirroring Social Cues

“…In contrast, vision-based controlled mirroring uses external cameras and pose estimation algorithms to interpret an actor's head movements and mirror them through a robot [8]. Liu et al [17] show that the lightweight model surpasses the other state-of-the-art models on the same robot doing the head movement mirroring. Geminiani et al [9] find that the Microsoft Kinect-based controlled NAO robot outperforms the IMU-based controlled NAO robot regarding limb movement mirroring in the autism treatment.…”