Ensembling of text and images using Deep Convolutional Neural Networks for Intelligent Information Retrieval

Animated Graphics Interchange Format (GIF) images have become an important part of network information interaction, and are one of the main characteristics of analyzing social media emotions. At present, most of the research on GIF affection recognition fails to make full use of spatial-temporal characteristics of GIF images, which limits the performance of model recognition to a certain extent. A GIF emotion recognition algorithm based on ResNet-ConvGRU is proposed in this paper. First, GIF data is preprocessed, converting its image sequences to static image format for saving. Then, the spatial features of images and the temporal features of static image sequences are extracted with ResNet and ConvGRU networks, respectively. At last, the animated GIFs data features are synthesized and the seven emotional intensities of GIF data are calculated. The GIFGIF dataset is used to verify the experiment. From the experimental results, the proposed animated GIFs emotion recognition model based on ResNet-ConvGRU, compared with the classical emotion recognition algorithms such as VGGNet-ConvGRU, ResNet3D, CNN-LSTM, and C3D, has a stronger feature extraction ability, and sentiment classification performance. This method provides a finer-grained analysis for the study of public opinion trends and a new idea for affection recognition of GIF data in social media.

show abstract

“…(1) VGGNet [23]. It is deep network structure consisting of 3 × 3 small convolutional kernels, commonly used in VGG16 and VGG19.…”

Section: Experimental Analysis and Resultsmentioning

confidence: 99%

Research on Animated GIFs Emotion Recognition Based on ResNet-ConvGRU

Zhang

Qing-dao-er-ji

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…The template and retrieval-based image captions generate the possible description of the images. The advanced image captioning method [12,13] includes the encoder and decoder structure in identifying the description for the images. In addition, the description performance is improved using the attention mechanism [14] and by capturing the relationship information about the objects.…”

Section: Literature Reviewmentioning

confidence: 99%

Summarization of Text and Image Captioning in Information Retrieval Using Deep Learning Techniques

Mahalakshmi

Fatima

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…These layers are usually convolution, pooling, and fully connected layers. The convolution layer is the most fundamental element of a CNN architecture and is used for feature extraction and nonlinear processing [25]. Negative values resulting from convolution are eliminated by applying a non-linear process to the resulting image of the weighted sum.…”

Section: Lightweight Convolutional Neural Networkmentioning

confidence: 99%

Defect Classification of Railway Fasteners Using Image Preprocessing and a Lightweight Convolutional Neural Network

2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Railway fasteners are used to securely fix rails to sleeper blocks. Partial wear or complete loss of these components can lead to serious accidents and cause train derailments. To ensure the safety of railway transportation, computer vision and pattern recognition-based methods are increasingly used to inspect railway infrastructure. In particular, it has become an important task to detect defects in railway tracks. This is challenging since rail track images are acquired using a measuring train in varying environmental conditions, at different times of day and in poor lighting conditions, and the resulting images often have low contrast. In this study, a new method is proposed for the classification of defects on rail track fasteners. The proposed approach uses image enhancement to first filter the rail images and obtain a high contrast image. Then, the rail track and sleeper positions are determined from the high contrast image. The location of the fastener is determined by applying the Line Local Binary Pattern method and the defects of the fastener are classified using an improved lightweight convolutional neural network (LCNN) model. Features are extracted from two fully connected layers of the developed LCNN model and the feature vector is constructed by concatenating these layers. The concatenated features are processed using a number of machine learning methods and the optimum classifier is chosen. Experimental results show that Cubic SVM gives the best results with a detection accuracy rate of 99.7%.

show abstract

Ensembling of text and images using Deep Convolutional Neural Networks for Intelligent Information Retrieval

Cited by 15 publications

References 26 publications

Research on Animated GIFs Emotion Recognition Based on ResNet-ConvGRU

Research on Animated GIFs Emotion Recognition Based on ResNet-ConvGRU

Summarization of Text and Image Captioning in Information Retrieval Using Deep Learning Techniques

Defect Classification of Railway Fasteners Using Image Preprocessing and a Lightweight Convolutional Neural Network

Contact Info

Product

Resources

About