Ranked batch-mode active learning

Cardoso, Thiago N. C.; Silva, Rodrigo M.; Canuto, Sérgio D.; Moro, Mirella M.; Gonçalves, Marcos André

doi:10.1016/j.ins.2016.10.037

Cited by 50 publications

(23 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare ITAL with a variety of baselines and competing methods, including SUD [33], TCAL [5], RBMAL [3], and the method of Brinker [2] referred to as "border div" in the following. All these native BMAL methods have been described in Section 2.…”

Section: Competitor Methodsmentioning

confidence: 99%

“…With regard to batch-mode active learning (BMAL), most existing methods employ some combination of the criteria uncertainty, diversity, and density: Brinker [2] proposes to select samples close to the decision boundary, while enforcing diversity by minimizing the maximum cosine similarity of samples within the batch. Similarly, "Sampling by Uncertainty and Density (SUD)" [33] selects samples maximizing the product of entropy and average cosine similarity to the nearest neighbors and "Ranked Batch-mode Active Learning (RBMAL)" [3] constructs a batch by successively adding samples with high uncertainty and low maximum similarity to any other already selected sample. "Triple Criteria Active Learning (TCAL)" [5], on the other hand, first selects a subset of uncertain samples near the decision boundary, divides them into k clusters, and chooses that sample from each cluster that has the minimum average distance to all other samples in the same cluster.…”

Section: Related Workmentioning

confidence: 99%

“…Because care has to be taken not only that all images in the batch of unlabeled samples selected for annotation are informative individually, but that they are also diverse compared to each other to avoid unnecessary redundant feedback. The majority of existing works on BMAL try to achieve this using a combination of several heuristics to simultaneously maximize the diversity within the batch and the uncertainty of the selected samples or their density in the dataset [2,33,12,5,32,3].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Information-Theoretic Active Learning for Content-Based Image Retrieval

Barz

Käding

Denzler

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We propose Information-Theoretic Active Learning (ITAL), a novel batch-mode active learning method for binary classification, and apply it for acquiring meaningful user feedback in the context of contentbased image retrieval. Instead of combining different heuristics such as uncertainty, diversity, or density, our method is based on maximizing the mutual information between the predicted relevance of the images and the expected user feedback regarding the selected batch. We propose suitable approximations to this computationally demanding problem and also integrate an explicit model of user behavior that accounts for possible incorrect labels and unnameable instances. Furthermore, our approach does not only take the structure of the data but also the expected model output change caused by the user feedback into account. In contrast to other methods, ITAL turns out to be highly flexible and provides state-of-the-art performance across various datasets, such as MIRFLICKR and ImageNet.

show abstract

Section: Competitor Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Information-Theoretic Active Learning for Content-Based Image Retrieval

Barz

Käding

Denzler

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Haertel, et al [31] proposed a parallel active learning method which can eliminate the wait time with minimal staleness. Thiago et al [32] introduce a ranked batch-mode active learning framework to reduce the manual labeling delays. However, these methods do not actually represent or reason about costs.…”

Section: Reduce Image Annotation Cost In Active Learningmentioning

confidence: 99%

Context Aware Image Annotation in Active Learning with Batch Mode

Sun

Loparo

2019

2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC)

View full text Add to dashboard Cite

Image annotation for active learning is labor-intensive. Various automatic and semi-automatic labeling methods are proposed to save the labeling cost, but a reduction in the number of labeled instances does not guarantee a reduction in cost because the queries that are most valuable to the learner may be the most difficult or ambiguous cases, and therefore the most expensive for an oracle to label accurately. In this paper, we try to solve this problem by using image metadata to offer the oracle more clues about the image during annotation process. We propose a Context Aware Image Annotation Framework (CAIAF) that uses image metadata as similarity metric to cluster images into groups for annotation. We also present useful metadata information as context for each image on the annotation interface. Experiments show that it reduces that annotation cost with CAIAF compared to the conventional framework, while maintaining a high classification performance.

show abstract

“…However, BMAL only uses an active selection strategy based on a single uncertainty index or diversity index when screening samples, which leads to considerable information redundancy in the labeled samples and unnecessary labeling costs. Cardoso [38] proposed the ranked batch-mode active learning (RBMAL) framework, which overcomes the limitations of traditional BMAL methods and generates an optimized ranked list to determine the priority of samples being labeled. Therefore, the RBMAL method has higher flexibility than the classic methods.…”

Section: Introductionmentioning

confidence: 99%

A Novel Query Strategy-Based Rank Batch-Mode Active Learning Method for High-Resolution Remote Sensing Image Classification

Luo

Zhou

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

An informative training set is necessary for ensuring the robust performance of the classification of very-high-resolution remote sensing (VHRRS) images, but labeling work is often difficult, expensive, and time-consuming. This makes active learning (AL) an important part of an image analysis framework. AL aims to efficiently build a representative and efficient library of training samples that are most informative for the underlying classification task, thereby minimizing the cost of obtaining labeled data. Based on ranked batch-mode active learning (RBMAL), this paper proposes a novel combined query strategy of spectral information divergence lowest confidence uncertainty sampling (SIDLC), called RBSIDLC. The base classifier of random forest (RF) is initialized by using a small initial training set, and each unlabeled sample is analyzed to obtain the classification uncertainty score. A spectral information divergence (SID) function is then used to calculate the similarity score, and according to the final score, the unlabeled samples are ranked in descending lists. The most “valuable” samples are selected according to ranked lists and then labeled by the analyst/expert (also called the oracle). Finally, these samples are added to the training set, and the RF is retrained for the next iteration. The whole procedure is iteratively implemented until a stopping criterion is met. The results indicate that RBSIDLC achieves high-precision extraction of urban land use information based on VHRRS; the accuracy of extraction for each land-use type is greater than 90%, and the overall accuracy (OA) is greater than 96%. After the SID replaces the Euclidean distance in the RBMAL algorithm, the RBSIDLC method greatly reduces the misclassification rate among different land types. Therefore, the similarity function based on SID performs better than that based on the Euclidean distance. In addition, the OA of RF classification is greater than 90%, suggesting that it is feasible to use RF to estimate the uncertainty score. Compared with the three single query strategies of other AL methods, sample labeling with the SIDLC combined query strategy yields a lower cost and higher quality, thus effectively reducing the misclassification rate of different land use types. For example, compared with the Batch_Based_Entropy (BBE) algorithm, RBSIDLC improves the precision of barren land extraction by 37% and that of vegetation by 14%. The 25 characteristics of different land use types screened by RF cross-validation (RFCV) combined with the permutation method exhibit an excellent separation degree, and the results provide the basis for VHRRS information extraction in urban land use settings based on RBSIDLC.

show abstract

Ranked batch-mode active learning

Cited by 50 publications

References 11 publications

Information-Theoretic Active Learning for Content-Based Image Retrieval

Information-Theoretic Active Learning for Content-Based Image Retrieval

Context Aware Image Annotation in Active Learning with Batch Mode

A Novel Query Strategy-Based Rank Batch-Mode Active Learning Method for High-Resolution Remote Sensing Image Classification

Contact Info

Product

Resources

About