AutoDAL: Distributed Active Learning with Automatic Hyperparameter Selection

Abstract

Automated machine learning (AutoML) strives to establish an appropriate machine learning model for any dataset automatically with minimal human intervention. Although extensive research has been conducted on AutoML, most of it has focused on supervised learning. Research of automated semi-supervised learning and active learning algorithms is still limited. Implementation becomes more challenging when the algorithm is designed for a distributed computing environment. With this as motivation, we propose a novel automated learning system for distributed active learning (AutoDAL) to address these challenges. First, automated graph-based semi-supervised learning is conducted by aggregating the proposed cost functions from different compute nodes in a distributed manner. Subsequently, automated active learning is addressed by jointly optimizing hyperparameters in both the classification and query selection stages leveraging the graph loss minimization and entropy regularization. Moreover, we propose an efficient distributed active learning algorithm which is scalable for big data by first partitioning the unlabeled data and replicating the labeled data to different worker nodes in the classification stage, and then aggregating the data in the controller in the query selection stage. The proposed AutoDAL algorithm is applied to multiple benchmark datasets and a real-world electrocardiogram (ECG) dataset for classification. We demonstrate that the proposed AutoDAL algorithm is capable of achieving significantly better performance compared to several state-of-the-art AutoML approaches and active learning algorithms.

Sampling Approach Matters: Active Learning for Robotic Language Acquisition

“…Unsupervised learning techniques, such as subspace clustering, have been shown to find influential points from a cluster [51]. A hybrid method that connects active learning and data programming [48] has shown improvements in the reduction of noisy data in large scale workspaces [15]. Similar to our work, active learning approaches [23], [63], [78] have been effective while training biased and highly varied datasets.…”

Distributed Active Partial Label Learning

“…To the best of our knowledge, the problem of fully decentralized AL has not been addressed. So, here, we only introduce several centralized AL methods [2,3,6] and distributed-like AL methods [7,9,10,11]. In recent years, several centralized AL methods have been proposed [2,3,6], including single-mode AL methods [3,6] and batch-mode methods [2].…”

“…However, in many practical applications, it is easy to obtain a massive number of unlabeled data is simple and inexpensive, but manually labeling each data sample is time-consuming and laborious [1]. To reduce the labeling effort and cost, active learning (AL), which can iteratively assign a class label to the valuable sample selected from an unlabeled data pool, has been developed [2][3][4][5][6][7][8][9][10][11][12]. Extensive experiments in [2][3][4][5][6][7][8][9][10][11][12] have shown AL can efficiently reduce the cost of label acquisition for training a high-precision model.…”

“…To reduce the labeling effort and cost, active learning (AL), which can iteratively assign a class label to the valuable sample selected from an unlabeled data pool, has been developed [2][3][4][5][6][7][8][9][10][11][12]. Extensive experiments in [2][3][4][5][6][7][8][9][10][11][12] have shown AL can efficiently reduce the cost of label acquisition for training a high-precision model.…”

“…Most studies on AL [2][3][4][5][6][7][8][9][10][11][12] utilized the representativeness criterion [4] and uncertainty criterion [2,5,12] as their selection criteria. For the former one, the samples that can effectively represent the whole dataset are preferentially selected.…”