Distant supervision has become the standard method for relation extraction. However, even though it is an efficient method, it does not come at no cost-The resulted distantly-supervised training samples are often very noisy. To combat the noise, most of the recent state-of-theart approaches focus on selecting onebest sentence or calculating soft attention weights over the set of the sentences of one specific entity pair. However, these methods are suboptimal, and the false positive problem is still a key stumbling bottleneck for the performance. We argue that those incorrectly-labeled candidate sentences must be treated with a hard decision, rather than being dealt with soft attention weights. To do this, our paper describes a radical solution-We explore a deep reinforcement learning strategy to generate the false-positive indicator, where we automatically recognize false positives for each relation type without any supervised information. Unlike the removal operation in the previous studies, we redistribute them into the negative examples. The experimental results show that the proposed strategy significantly improves the performance of distant supervision comparing to state-of-the-art systems.

Learning high-quality sentence representations benefits a wide range of natural language processing tasks. Though BERT-based pretrained language models achieve high performance on many downstream tasks, the native derived sentence representations are proved to be collapsed and thus produce a poor performance on the semantic textual similarity (STS) tasks. In this paper, we present ConSERT, a Contrastive Framework for Self-Supervised SEntence Representation Transfer, that adopts contrastive learning to fine-tune BERT in an unsupervised and effective way. By making use of unlabeled texts, ConSERT solves the collapse issue of BERT-derived sentence representations and make them more applicable for downstream tasks. Experiments on STS datasets demonstrate that ConSERT achieves an 8% relative improvement over the previous state-of-the-art, even comparable to the supervised SBERT-NLI. And when further incorporating NLI supervision, we achieve new stateof-the-art performance on STS tasks. Moreover, ConSERT obtains comparable results with only 1000 samples available, showing its robustness in data scarcity scenarios.

Learning high-quality sentence representations benefits a wide range of natural language processing tasks. Though BERT-based pretrained language models achieve high performance on many downstream tasks, the native derived sentence representations are proved to be collapsed and thus produce a poor performance on the semantic textual similarity (STS) tasks. In this paper, we present ConSERT, a Contrastive Framework for Self-Supervised SEntence Representation Transfer, that adopts contrastive learning to fine-tune BERT in an unsupervised and effective way. By making use of unlabeled texts, ConSERT solves the collapse issue of BERT-derived sentence representations and make them more applicable for downstream tasks. Experiments on STS datasets demonstrate that ConSERT achieves an 8% relative improvement over the previous state-of-the-art, even comparable to the supervised SBERT-NLI. And when further incorporating NLI supervision, we achieve new stateof-the-art performance on STS tasks. Moreover, ConSERT obtains comparable results with only 1000 samples available, showing its robustness in data scarcity scenarios.

No abstract

The tremendous success of deep learning in machine fault diagnosis is dependent on the hypothesis that training and test datasets are subordinated to the same distribution. This subordination is difficult to meet in practical scenarios of industrial applications. On the one hand, the working conditions of rotating machinery can change easily. On the other hand, vibration data and labels are difficult to obtain to train a specific model for each working condition. In this study, we solve these problems by constructing a novel deep transfer learning model called multi-scale deep intra-class adaptation network, which first uses the modified ResNet-50 to extract low-level features and then constructs a multiple scale feature learner to analyze these low-level features at multiple scales and obtain high-level features as input for the classifier. Pseudo labels are then computed to shorten the conditional distribution distance of vibration data collected under different working loads for intra-class adaptation. The proposed method is validated using two datasets to recognize the bearing normal state, the inner race, the ball and outer race faults, and their fault degrees under four different working loads. The high-precision diagnosis results of 24 transfer learning experiments reveal the reliability and generalizability of the constructed model.

Detecting out-of-domain (OOD) input intents is critical in the task-oriented dialog system. Different from most existing methods that rely heavily on manually labeled OOD samples, we focus on the unsupervised OOD detection scenario where there are no labeled OOD samples except for labeled in-domain data. In this paper, we propose a simple but strong generative distancebased classifier to detect OOD samples. We estimate the class-conditional distribution on feature spaces of DNNs via Gaussian discriminant analysis (GDA) to avoid over-confidence problems. And we use two distance functions, Euclidean and Mahalanobis distances, to measure the confidence score of whether a test sample belongs to OOD. Experiments on four benchmark datasets show that our method can consistently outperform the baselines.

No abstract

No abstract