Humanoid Intelligent Display Platform for Audiovisual Interaction and Sound Identification

Abstract

This study proposes a rational strategy for the design, fabrication and system integration of the humanoid intelligent display platform (HIDP) to meet the requirements of highly humanized mechanical properties and intelligence for human–machine interfaces. The platform's sandwich structure comprises a middle light-emitting layer and surface electrodes, which consists of silicon elastomer embedded with phosphor and silk fibroin ionoelastomer, respectively. Both materials are highly stretchable and resilient, endowing the HIDP with skin-like mechanical properties and applicability in various extreme environments and complex mechanical stimulations. Furthermore, by establishing the numerical correlation between the amplitude change of animal sounds and the brightness variation, the HIDP realizes audiovisual interaction and successful identification of animal species with the aid of Internet of Things (IoT) and machine learning techniques. The accuracy of species identification reaches about 100% for 200 rounds of random testing. Additionally, the HIDP can recognize animal species and their corresponding frequencies by analyzing sound characteristics, displaying real-time results with an accuracy of approximately 99% and 93%, respectively. In sum, this study offers a rational route to designing intelligent display devices for audiovisual interaction, which can expedite the application of smart display devices in human–machine interaction, soft robotics, wearable sound-vision system and medical devices for hearing-impaired patients.

Recent Advances in In-Memory Computing: Exploring Memristor and Memtransistor Arrays with 2D Materials

“…To enable these applications, the integration of memristive devices is crucial for the development of functional hardware systems. Neuron network, drawing inspiration from the human brain, exhibits wide applications such as sound identification [ 126 ], health monitoring [ 127 ], language transition [ 128 ]. Neural networks also serve as a common approach to integrating memristive devices for neuromorphic computing and artificial intelligence and demonstrate strong performance in many areas such as vision sensor [ 129 ], pressure sensor [ 130 ], flexible devices [ 131 ].…”

Micro robot as the feature of robotic in healthcare approach from design to application: the State of art and challenges

“…Nanorobots offer high precision and the potential for targeted therapy. However, their development is still in the early stages, and challenges related to manufacturing, control, and biocompatibility need to be addressed [54].…”

Advanced Design of Soft Robots with Artificial Intelligence

“…Immersive communications can be accomplished with the assistance of intelligent soft robotics. The rapid advancement of flexible sensors [227][228][229][230] and machine learning [231][232][233][234] have provided the operators with the rich experience. In an attempt to achieve this goal, a modular soft glove with multimodal sensing abilities was proposed [235], which was illustrated in Fig.…”