Research on Video Captioning Based on Multifeature Fusion

Abstract

Aiming at the problems that the existing video captioning models pay attention to incomplete information and the generation of expression text is not accurate enough, a video captioning model that integrates image, audio, and motion optical flow is proposed. A variety of large-scale dataset pretraining models are used to extract video frame features, motion information, audio features, and video sequence features. An embedded layer structure based on self-attention mechanism is designed to embed single-mode features and learn single-mode feature parameters. Then, two schemes of joint representation and cooperative representation are used to fuse the multimodal features of the feature vectors output by the embedded layer, so that the model can pay attention to different targets in the video and their interactive relationships, which effectively improves the performance of the video captioning model. The experiment is carried out on large datasets MSR-VTT and LSMDC. Under the metrics BLEU4, METEOR, ROUGEL, and CIDEr, the MSR-VTT benchmark dataset obtained scores of 0.443, 0.327, 0.619, and 0.521, respectively. The result shows that the proposed method can effectively improve the performance of the video captioning model, and the evaluation indexes are improved compared with comparison models.

Artificial Intelligence Technology and Its Application in Improving Thought-Politics Education

“…e proposed attention mechanism is inspired by the human visual mechanism [29], and the basic idea is to weaken irrelevant information and increase the attention of focused information during the operation. In this paper, the time-dimensional features Tand spatial-dimensional features F are fused by using the multiscale feature fusion attention mechanism as shown in Figure 3.…”

Research on Online Education Resources Recommendation Based on Deep Learning

“…e proposed attention mechanism is inspired by the human visual mechanism, and the basic idea is to weaken irrelevant information and increase the attention of focused information during the operation [27]. In this paper, learner features u i , attribute features s j of learning resources, and text features t j of learning resources are fused using the multiscale feature fusion attention mechanism shown in Figure 4.…”