For the study of single-modal recognition, for example, the research on speech signals, ECG signals, facial expressions, body postures and other physiological signals have made some progress. However, the diversity of human brain information sources and the uncertainty of single-modal recognition determine that the accuracy of single-modal recognition is not high. Therefore, building a multimodal recognition framework in combination with multiple modalities has become an effective means of improving performance. With the rise of multi-modal machine learning, multi-modal information fusion has become a research hotspot, and audio-visual fusion is the most widely used direction. The audio-visual fusion method has been successfully applied to various problems, such as emotion recognition and multimedia event detection, biometric and speech recognition applications. This paper firstly introduces multimodal machine learning briefly, and then summarizes the development and current situation of audio-visual fusion technology in some major areas, and finally puts forward the prospect for the future.
Flowers are a common species in people’s lives, but people’s understanding of flowers is not comprehensive. The traditional UI display method lacks immersion and interactivity. This paper proposes a method based on the deep learning neural network flower recognition algorithm to recognize the flower target information and replace the traditional UI with the AR information to display the flower information. The tracking and positioning technology used in the floral AR display method is a very important underlying technology. This paper focuses on the tracking technology of flower target, and proposes a server-based flower recognition neural network algorithm recognition algorithm. After identifying the flower information and detection frame, it outputs the detection frame parameters of the flower target, and then combines the OpenCV KCF tracker pair. The method of performing AR display of the flower information has high tracking accuracy and stability.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.