Answer selection is one of the key steps in many question answering (QA) applications. In this paper, a new deep model with two kinds of attention is proposed for answer selection: the double attention recurrent convolution neural network (DARCNN). Double attention means self-attention and cross-attention. The design inspiration of this model came from the transformer in the domain of machine translation. Self-attention can directly calculate dependencies between words regardless of the distance. However, self-attention ignores the distinction between its surrounding words and other words. Thus, we design a decay self-attention that prioritizes local words in a sentence. In addition, cross-attention is established to achieve interaction between question and candidate answer. With the outputs of self-attention and decay self-attention, we can get two kinds of interactive information via cross-attention. Finally, using the feature vectors of the question and answer, elementwise multiplication is used to combine with them and multilayer perceptron is used to predict the matching score. Experimental results on four QA datasets containing Chinese and English show that DARCNN performs better than other answer selection models, thereby demonstrating the effectiveness of self-attention, decay self-attention and cross-attention in answer selection tasks.
Reciprocating compressors are widely used in the petroleum industry and because of their complex and nonlinear signals, it is difficult to extract fault features. Recently, deep learning has been used in intelligent mechanical fault diagnosis and achieved great success. In the deep learning model, the recursive neural network (RNN) can capture global features, but it is difficult to parallelize and not good at dealing with long sequences. The convolutional neural network (CNN) can capture local features, but its receptive field is limited by the number of layers of the network and the size of the sliding window, resulting in the model not capturing sufficient features. In this paper, we propose a deep learning model without any RNN or CNN structures, called the group self-attention network (GSAN), for fault diagnosis of multisource signals in reciprocating compressors. The GSAN model mainly includes intra-group self-attention, inter-group self-attention and a fusion gate. Among them, intra-group self-attention is used to capture local features within a group, inter-group self-attention is used to capture global features between groups, and the fusion gate finally integrates these features. Experimental results show that compared with other models based on the RNN or the CNNs, the GSAN proposed in this paper not only has higher prediction accuracy, but also better anti-noise performance. In addition, the effectiveness of each part of the model is verified by ablation experiment.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.