Automatic data acquisition for deep learning

Abstract

Deep learning (DL) has widespread applications and has revolutionized many industries. Although automated machine learning (AutoML) can help us away from coding for DL models, the acquisition of lots of high-quality data for model training remains a main bottleneck for many DL projects, simply because it requires high human cost. Despite many works on weak supervision ( i.e. , adding weak labels to seen data) and data augmentation ( i.e. , generating more data based on seen data), automatically acquiring training data, via smartly searching a pool of training data collected from open ML benchmarks and data markets, is not explored. In this demonstration, we demonstrate a new system, automatic data acquisition (AutoData), which automatically searches training data from a heterogeneous data repository and interacts with AutoML. It faces two main challenges. (1) How to search high-quality data from a large repository for a given DL task? (2) How does AutoData interact with AutoML to guide the search? To address these challenges, we propose a reinforcement learning (RL)-based framework in AutoData to guide the iterative search process. AutoData encodes current training data and feedbacks of AutoML, learns a policy to search fresh data, and trains in iterations. We demonstrate with two real-life scenarios, image classification and relational data prediction, showing that AutoData can select high-quality data to improve the model.

Data augmentation: A comprehensive survey of modern approaches

“…These techniques have different strengths and limitations related to complexity, flexibility and predictive performance. While the crudest approaches to automating data processing tasks involve using hand-crafted routines, more advanced methods [51], [52], [53] utilize learned mechanisms to perform processing. The most advanced methods, however, employ automated machine learning (AutoML) techniques [12], [54], [55], [56] to automate data processing functions.…”

“…Data preprocessing functions consist of a set of basic operations that transform raw data into a form that is useful for the machine learning model. Important preprocessing subtasks include data cleaning [61], [62], labeling or relabeling [53], [63], [63], [64], categorical encoding [65], [66], [67] and imputation of missing data [68], [69], [70]. Figure 8 depicts the main categories of data preprocessing tasks and the set of common problems they commonly tackle.…”

“…Even though a variety of machine-guided automated labeling [12], [53], [119], [120], [121] techniques have been proposed, they are generally limited to simple labeling tasks targeting very narrow application settings. It is currently challenging to extend them to more complex or general tasks.…”

“…Some new approaches such as AutoData [53], DataAssist [125], BioAutoMATED [127], Atlantic [54] and DiffPrep [126] are designed as dedicated pre-processing plug-ins that can be integrated into standard AutoML frameworks. In this way, they can leverage the hyperparameter and pipeline optimization capabilities of large-scale AutoML models to optimize pre-processing functions.…”

“…These models are generally aimed at addressing the limitations of generic methods and tools by developing specialized modules that can be used to perform specific data preparation tasks within general-purpose AutoML pipelines. AutoData [53] (Figure 11), for instance, implements a dedicated data processing module that can be used within standard AutoML frameworks as an end-to-end mechanism for performing many data pre-processing and and augmentation tasks, including data acquisition, labeling, cleaning and augmentation. It uses reinforcement learning method to search for suitable data from diverse external sources.…”