No abstract

No abstract

For the study of single-modal recognition, for example, the research on speech signals, ECG signals, facial expressions, body postures and other physiological signals have made some progress. However, the diversity of human brain information sources and the uncertainty of single-modal recognition determine that the accuracy of single-modal recognition is not high. Therefore, building a multimodal recognition framework in combination with multiple modalities has become an effective means of improving performance. With the rise of multi-modal machine learning, multi-modal information fusion has become a research hotspot, and audio-visual fusion is the most widely used direction. The audio-visual fusion method has been successfully applied to various problems, such as emotion recognition and multimedia event detection, biometric and speech recognition applications. This paper firstly introduces multimodal machine learning briefly, and then summarizes the development and current situation of audio-visual fusion technology in some major areas, and finally puts forward the prospect for the future.

Flowers are a common species in people’s lives, but people’s understanding of flowers is not comprehensive. The traditional UI display method lacks immersion and interactivity. This paper proposes a method based on the deep learning neural network flower recognition algorithm to recognize the flower target information and replace the traditional UI with the AR information to display the flower information. The tracking and positioning technology used in the floral AR display method is a very important underlying technology. This paper focuses on the tracking technology of flower target, and proposes a server-based flower recognition neural network algorithm recognition algorithm. After identifying the flower information and detection frame, it outputs the detection frame parameters of the flower target, and then combines the OpenCV KCF tracker pair. The method of performing AR display of the flower information has high tracking accuracy and stability.