A survey on heterogeneous transfer learning

Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough labeled data. Second, unlike traditional machine learning, deep learning techniques automatically generate features, which saves feature engineering costs, but in return may require larger amounts of labeled data. Interestingly, recent research in data collection comes not only from the machine learning, natural language, and computer vision communities, but also from the data management community due to the importance of handling large amounts of data. In this survey, we perform a comprehensive study of data collection from a data management point of view. Data collection largely consists of data acquisition, data labeling, and improvement of existing data or models. We provide a research landscape of these operations, provide guidelines on which technique to use when, and identify interesting research challenges. The integration of machine learning and data management for data collection is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research. TaskApproach Techniques Data discovery Sharing Collaborative Analysis [9]-[11] Web [12]-[17]

show abstract

“…Improve Data Data Cleaning [158]- [166] Re-labeling [119] Improve Model Robust Against Noise [167]- [171] Transfer Learning [172]- [178]…”

Section: Task Techniquesmentioning

confidence: 99%

A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective

Roh

Heo

Whang

2021

IEEE Trans. Knowl. Data Eng.

597

270

View full text Add to dashboard Cite

show abstract

“…Before we cover the example of transfer learning, we must clarify that our definition of "transfer learning" differs from another definition present in current literature. In works such as [82], "transfer learning" refers to training a supervised machine learning algorithm with a labeled dataset, and then feeding the trained algorithm input values from a different dataset. This is not the technique we refer to as transfer learning here.…”

Section: Transfer Learningmentioning

confidence: 99%

Survey on categorical data for neural networks

2020

Self Cite

View full text Add to dashboard Cite

This survey investigates current techniques for representing qualitative data for use as input to neural networks. Techniques for using qualitative data in neural networks are well known. However, researchers continue to discover new variations or entirely new methods for working with categorical data in neural networks. Our primary contribution is to cover these representation techniques in a single work. Practitioners working with big data often have a need to encode categorical values in their datasets in order to leverage machine learning algorithms. Moreover, the size of data sets we consider as big data may cause one to reject some encoding techniques as impractical, due to their running time complexity. Neural networks take vectors of real numbers as inputs. One must use a technique to map qualitative values to numerical values before using them as input to a neural network. These techniques are known as embeddings, encodings, representations, or distributed representations. Another contribution this work makes is to provide references for the source code of various techniques, where we are able to verify the authenticity of the source code. We cover recent research in several domains where researchers use categorical data in neural networks. Some of these domains are natural language processing, fraud detection, and clinical document automation. This study provides a starting point for research in determining which techniques for preparing qualitative data for use with neural networks are best. It is our intention that the reader should use these implementations as a starting point to design experiments to evaluate various techniques for working with qualitative data in neural networks. The third contribution we make in this work is a new perspective on techniques for using categorical data in neural networks. We organize techniques for using categorical data in neural networks into three categories. We find three distinct patterns in techniques that identify a technique as determined, algorithmic, or automated. The fourth contribution we make is to identify several opportunities for future research. The form of the data that one uses as an input to a neural network is crucial for using neural networks effectively. This work is a tool for researchers to find the most effective technique for working with categorical data in neural networks, in big data settings. To the best of our knowledge this is the first in-depth look at techniques for working with categorical data in neural networks.

show abstract

“…Heterogeneous transfer learning [37] was proposed for the nonequivalent of feature spaces or label spaces. However, according to the transfer learning survey [5], few existing methods addressed the issue of differing label spaces. Furthermore, the existing methods, which can directly address the issue of differing label spaces, usually had additional restrictions.…”

Section: Related Workmentioning

confidence: 99%

Cross-domain Aspect Category Transfer and Detection via Traceable Heterogeneous Graph Representation Learning

Jiang

Wang

Zhao

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Aspect category detection is an essential task for sentiment analysis and opinion mining. However, the cost of categorical data labeling, e.g., label the review aspect information for a large number of product domains, can be inevitable but unaffordable. In this study, we propose a novel problem, cross-domain aspect category transfer and detection, which faces three challenges: various feature spaces, different data distributions, and diverse output spaces. To address these problems, we propose an innovative solution, Traceable Heterogeneous Graph Representation Learning (THGRL). Unlike prior text-based aspect detection works, THGRL explores latent domain aspect category connections via massive user behavior information on a heterogeneous graph. Moreover, an innovative latent variable "Walker Tracer" is introduced to characterize the global semantic/aspect dependencies and capture the informative vertexes on the random walk paths. By using THGRL, we project different domains' feature spaces into a common one, while allowing data distributions and output spaces stay differently. Experiment results show that the proposed method outperforms a series of state-of-the-art baseline models.

show abstract

A survey on heterogeneous transfer learning

Cited by 264 publications

References 91 publications

A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective

A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective

Survey on categorical data for neural networks

Cross-domain Aspect Category Transfer and Detection via Traceable Heterogeneous Graph Representation Learning

Contact Info

Product

Resources

About