Incorporating context-relevant concepts into convolutional neural networks for short text classification

Xu, Jingyun; Cai, Yi; Wu, Xin; Xue, Lei; Huang, Qingbao; Leung, Ho-fung; Li, Qing

doi:10.1016/j.neucom.2019.08.080

Cited by 61 publications

(25 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e task of multitext reading comprehension needs to screen out important content and answers from multiple documents, and providing answers is only part of the document. When it is necessary to extract the content and answers of multiple documents, the existence of multiple interference information will increase the difficulty of machine reading comprehension and screening answers to a certain extent [14][15][16]. In the face of massive documents, multitext machine reading comprehension needs to first retrieve and filter multiple documents and then conduct single text retrieval on the filtered documents to capture the content and answers in the documents.…”

Section: Model Framework Of English Multitext Readingmentioning

confidence: 99%

Analysis of English Multitext Reading Comprehension Model Based on Deep Belief Neural Network

Tang

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In order to solve the problems of low accuracy and low efficiency of answer prediction in machine reading comprehension, a multitext English reading comprehension model based on the deep belief neural network is proposed. Firstly, the paragraph selector in the multitext reading comprehension model is constructed. Secondly, the text reader is designed, and the deep belief neural network is introduced to predict the question answering probability. Finally, the popular English dataset of SQuAD is used for test analysis. The final results show that, after the comparative analysis of different learning methods, it is found that the English multitext reading comprehension model has a strong reading comprehension ability. In addition, two evaluation methods are used to score the overall performance of the model, which shows that the overall score of the English multitext reading comprehension model based on the deep confidence neural network is more than 90, and the efficiency will not be reduced because of the change of the number of documents in the dataset. The above results show that the use of the deep belief neural network to improve the probability generation performance of the model can well solve the task of English multitext reading comprehension, effectively reduce the difficulty of machine reading comprehension in multitask reading, and has a good guiding significance for promoting human convenient Internet knowledge acquisition.

show abstract

Section: Model Framework Of English Multitext Readingmentioning

confidence: 99%

Analysis of English Multitext Reading Comprehension Model Based on Deep Belief Neural Network

Tang

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Thus, the introduction of contextual information helps to provide supplementary knowledge for classification accuracy. The context-aware technique is widely used in natural language processing (NLP) tasks such as text classification (Xu et al, 2020 ), while few-shot learning is often used in image classification, and it has been a meaningful challenge in a modeling context in computer vision tasks (Bell et al, 2016 ; Kantorov et al, 2016 ). Research has been proposed to achieve object detection (Chen and Gupta, 2017 ; Wang et al, 2018 ) and image classification (Zhang et al, 2020 ) with contexts, and (Kamara et al, 2020 ) combines contextual information in time series classification.…”

Section: Related Workmentioning

confidence: 99%

Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification

et al. 2021

View full text Add to dashboard Cite

Deep transfer learning aims at dealing with challenges in new tasks with insufficient samples. However, when it comes to few-shot learning scenarios, due to the low diversity of several known training samples, they are prone to be dominated by specificity, thus leading to one-sidedness local features instead of the reliable global feature of the actual categories they belong to. To alleviate the difficulty, we propose a cross-modal few-shot contextual transfer method that leverages the contextual information as a supplement and learns context awareness transfer in few-shot image classification scenes, which fully utilizes the information in heterogeneous data. The similarity measure in the image classification task is reformulated via fusing textual semantic modal information and visual semantic modal information extracted from images. This performs as a supplement and helps to inhibit the sample specificity. Besides, to better extract local visual features and reorganize the recognition pattern, the deep transfer scheme is also used for reusing a powerful extractor from the pre-trained model. Simulation experiments show that the introduction of cross-modal and intra-modal contextual information can effectively suppress the deviation of defining category features with few samples and improve the accuracy of few-shot image classification tasks.

show abstract

“…Convolutional Neural Network Convolutional neural networks (CNNs) have demonstrated exceptional success in NLP tasks such as document classification, language modeling, or machine translation (Lin et al, 2018). As Xu et al (2020) described, CNN models can produce consistent performance when applied to the various text types such as short sequences. We evaluated a CNN architecture (Shen et al, 2018) with a convolutional layer, followed by batch normalization, ReLU, and a dropout layer, which was followed by a maxpooling layer.…”

Section: Model Architecture and Trainingmentioning

confidence: 99%

Handling Extreme Class Imbalance in Technical Logbook Datasets

Akhbardeh¹,

Alm²,

Zampieri³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Technical logbooks are a challenging and under-explored text type in automated event identification. These texts are typically short and written in non-standard yet technical language, posing challenges to off-the-shelf NLP pipelines. The granularity of issue types described in these datasets additionally leads to class imbalance, making it challenging for models to accurately predict which issue each logbook entry describes. In this paper we focus on the problem of technical issue classification by considering logbook datasets from the automotive, aviation, and facilities maintenance domains. We adapt a feedback strategy from computer vision for handling extreme class imbalance, which resamples the training data based on its error in the prediction process. Our experiments show that with statistical significance this feedback strategy provides the best results for four different neural network models trained across a suite of seven different technical logbook datasets from distinct technical domains. The feedback strategy is also generic and could be applied to any learning problem with substantial class imbalances.

show abstract

Incorporating context-relevant concepts into convolutional neural networks for short text classification

Cited by 61 publications

References 1 publication

Analysis of English Multitext Reading Comprehension Model Based on Deep Belief Neural Network

Analysis of English Multitext Reading Comprehension Model Based on Deep Belief Neural Network

Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification

Handling Extreme Class Imbalance in Technical Logbook Datasets

Contact Info

Product

Resources

About