Neural Unsupervised Domain Adaptation in NLP---A Survey

Ramponi, Alan; Plank, Barbara

doi:10.48550/arxiv.2006.00632

Cited by 22 publications

(29 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised means that the model is trained on normal log messages only. As not all possible anomalies in log data can be known and used for training [34], unsupervised approaches are well suited for log data scenarios, and thus are of high interest for industry and academia [11,3]. Therefore, the challenge is to develop a good understanding of normal log messages, e.g.…”

Section: General Frameworkmentioning

confidence: 99%

A2Log: Attentive Augmented Log Anomaly Detection

Wittkopp¹,

Acker²,

Nedelkoski³

et al. 2021

Preprint

View full text Add to dashboard Cite

Anomaly detection becomes increasingly important for the dependability and serviceability of IT services. As log lines record events during the execution of IT services, they are a primary source for diagnostics. Thereby, unsupervised methods provide a significant benefit since not all anomalies can be known at training time. Existing unsupervised methods need anomaly examples to obtain a suitable decision boundary required for the anomaly detection task. This requirement poses practical limitations. Therefore, we develop A2Log, which is an unsupervised anomaly detection method consisting of two steps: Anomaly scoring and anomaly decision. First, we utilize a self-attention neural network to perform the scoring for each log message. Second, we set the decision boundary based on data augmentation of the available normal training data. The method is evaluated on three publicly available datasets and one industry dataset. We show that our approach outperforms existing methods. Furthermore, we utilize available anomaly examples to set optimal decision boundaries to acquire strong baselines. We show that our approach, which determines decision boundaries without utilizing anomaly examples, can reach scores of the strong baselines.

show abstract

Section: General Frameworkmentioning

confidence: 99%

A2Log: Attentive Augmented Log Anomaly Detection

Wittkopp¹,

Acker²,

Nedelkoski³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…As an important case of transfer learning, unsupervised domain adaptation (UDA) has drawn significant attention from the computer vision and natural language processing communities. The UDA research can be mainly categorized into three streams [29]: model-centric, data-centric and hybrid methods that combines the previous two.…”

Section: Related Work a Unsupervised Domain Adaptationmentioning

confidence: 99%

Cross-Lingual Transfer Learning for Statistical Type Inference

Li¹,

Xie²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning-based techniques have been widely applied to the program analysis tasks, in fields such as type inference, fault localization, and code summarization.Hitherto deep learning-based software engineering systems rely thoroughly on supervised learning approaches, which require laborious manual effort to collect and label a prohibitively large amount of data. However, most Turing-complete imperative languages share similar control-and data-flow structures, which make it possible to transfer knowledge learned from one language to another. In this paper, we propose cross-lingual adaptation of program analysis, which allows us to leverage prior knowledge learned from the labeled dataset of one language and transfer it to the others. Specifically, we implemented a cross-lingual adaptation framework, PLATO, to transfer a deep learning-based type inference procedure across weakly typed languages, e.g., Python to JavaScript and vice versa. PLATO incorporates a novel joint graph kernelized attention based on abstract syntax tree and control flow graph, and applies anchor word augmentation across different languages.Besides, by leveraging data from strongly typed languages, PLATO improves the perplexity of the backbone crossprogramming-language model and the performance of downstream cross-lingual transfer for type inference. Experimental results illustrate that our framework significantly improves the transferability over the baseline method by a large margin.

show abstract

“…The domain adaptation problem refers to the situation where a statistical learning model trained on one labeled dataset needs to be generalized to the target dataset, or target domain, drawn from a different distribution and with insufficient labeled data (Daume III and Marcu, 2006). Learning from data collected in different domains is an active area of research in computer science and has been explored in various applications including natural language processing (Ramponi andPlank, 2020), visual classification (Wang andDeng, 2018), sentiment prediction (Glorot et al, 2011), and more recently in prediction problems in public health and clinical settings (Rehman et al, 2018;Mhasawade et al, 2020;Laparra et al, 2020).…”

Section: Domain Adaption In Cause-of-death Assignmentmentioning

confidence: 99%

Bayesian Nested Latent Class Models for Cause-of-Death Assignment using Verbal Autopsies Across Multiple Domains

Li¹,

Wu²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding cause-specific mortality rates is crucial for monitoring population health and designing public health interventions. Worldwide, two-thirds of deaths do not have a cause assigned. Verbal autopsy (VA) is a well-established tool to collect information describing deaths outside of hospitals by conducting surveys to caregivers of a deceased person. It is routinely implemented in many low-and middle-income countries. Statistical algorithms to assign cause of death using VAs are typically vulnerable to the distribution shift between the data used to train the model and the target population. This presents a major challenge for analyzing VAs as labeled data are usually unavailable in the target population. This article proposes a Latent Class model framework for VA data (LCVA) that jointly models VAs collected over multiple heterogeneous domains, assign cause of death for out-of-domain observations, and estimate cause-specific mortality fractions for a new domain. We introduce a parsimonious representation of the joint distribution of the collected symptoms using nested latent class models and develop an efficient algorithm for posterior inference. We demonstrate that LCVA outperforms existing methods in predictive performance and scalability. Supplementary materials for this article and the R package to implement the model are available online.

show abstract

Neural Unsupervised Domain Adaptation in NLP---A Survey

Cited by 22 publications

References 65 publications

A2Log: Attentive Augmented Log Anomaly Detection

A2Log: Attentive Augmented Log Anomaly Detection

Cross-Lingual Transfer Learning for Statistical Type Inference

Bayesian Nested Latent Class Models for Cause-of-Death Assignment using Verbal Autopsies Across Multiple Domains

Contact Info

Product

Resources

About