modAL: A modular active learning framework for Python

Danka, Tivadar; Horváth, Péter

doi:10.48550/arxiv.1805.00979

Cited by 18 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For training and evaluating the ML models, Google Colab is used. In addition, the modAL framework [65], an active learning framework for python is used for manually labeling the Twitter users. It is a scikit-learn based platform that is modular, flexible and extensible.…”

Section: Experimental Results and Model Evaluationmentioning

confidence: 99%

Trust and Believe – Should We? Evaluating the Trustworthiness of Twitter Users

Khan

2021

View full text Add to dashboard Cite

Social networking and micro-blogging services, such as Twitter, play an important role in sharing digital information. Despite the popularity and usefulness of social media, there have been many instances where corrupted users found ways to abuse it, as for instance, through raising or lowering user's credibility. As a result, while social media facilitates an unprecedented ease of access to information, it also introduces a new challenge -that of ascertaining the credibility of shared information. Currently, there is no automated way of determining which news or users are credible and which are not. Hence, establishing a system that can measure the social media user's credibility has become an issue of great importance. Assigning a credibility score to a user has piqued the interest of not only the research community but also most of the big players on both sides -such as Facebook, on the side of industry, and political parties on the societal one. In this work, we created a model which, we hope, will ultimately facilitate and support the increase of trust in the social network communities. Our model collected data and analysed the behaviour of 50,000 politicians on Twitter. Influence score, based on several chosen features, was assigned to each evaluated user. Further, we classified the political Twitter users as either trusted or untrusted using random forest, multilayer perceptron, and support vector machine. An active learning model was used to classify any unlabelled ambiguous records from our dataset. Finally, to measure the performance of the proposed model, we used precision, recall, F1 score, and accuracy as the main evaluation metrics.

show abstract

Section: Experimental Results and Model Evaluationmentioning

confidence: 99%

Trust and Believe – Should We? Evaluating the Trustworthiness of Twitter Users

Khan

2021

View full text Add to dashboard Cite

show abstract

“…Once the initial labelled dataset is formed, the first model is trained and the iterative querying process begins. QBC is implemented in Python using the modAL framework [7] as the basic tool for selection of instances for annotation by a human expert. kNN is selected as the base classification method because it is simple and easy to implement, versatile, and is a non-parametric algorithm.…”

Section: Dataset 1 -Process Data Collected From a Simulated Ethanol P...mentioning

confidence: 99%

“…An initial model is formed by using the initial labelled dataset and then the iterative querying process is initiated. In these experiments, US and QBC are implemented for querying in Python using the modAL framework [7] and kNN is chosen as the base classifier. In the experiment using this dataset, three query procedures are used.…”

Section: Dataset 2 -Drug Consumption (Quantified) Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Addressing practical challenges in Active Learning via a hybrid query strategy

Agarwal,

Srivastava,

Martin-del-Campo

et al. 2021

Preprint

View full text Add to dashboard Cite

Active Learning (AL) is a powerful tool to address modern machine learning problems with significantly fewer labeled training instances. However, implementation of traditional AL methodologies in practical scenarios is accompanied by multiple challenges due to the inherent assumptions. There are several hindrances, such as unavailability of labels for the AL algorithm at the beginning; unreliable external source of labels during the querying process; or incompatible mechanisms to evaluate the performance of Active Learner. Inspired by these practical challenges, we present a hybrid query strategy-based AL framework that addresses three practical challenges simultaneously: cold-start, oracle uncertainty and performance evaluation of Active Learner in the absence of ground truth.While a pre-clustering approach is employed to address the cold-start problem, the uncertainty surrounding the expertise of labeler and confidence in the given labels is incorporated to handle oracle uncertainty. The heuristics obtained during the querying process serve as the fundamental premise for accessing the performance of Active Learner. The robustness of the proposed AL framework is evaluated across three different environments and industrial settings. The results demonstrate the capability of the proposed framework to tackle practical challenges during AL implementation in real-world scenarios. 1

show abstract

“…Algorithm 2 describes this process in more detail. The implementation of AL in the modAL package (Danka and Horvath, 2018) was used in our experiment.…”

Section: Active Learningmentioning

confidence: 99%

Unsupervised Instance Selection with Low-Label, Supervised Learning for Outlier Detection

Bradberry¹,

H.²,

Kent³

et al. 2021

Preprint

View full text Add to dashboard Cite

The laborious process of labeling data often bottlenecks projects that aim to leverage the power of supervised machine learning. Active Learning (AL) has been established as a technique to ameliorate this condition through an iterative framework that queries a human annotator for labels of instances with the most uncertain class assignment. Via this mechanism, AL produces a binary classifier trained on less labeled data but with little, if any, loss in predictive performance. Despite its advantages, AL can have difficulty with class-imbalanced datasets and results in an inefficient labeling process. To address these drawbacks, we investigate our unsupervised instance selection (UNISEL) technique followed by a Random Forest (RF) classifier on 10 outlier detection datasets under low-label conditions. These results are compared to AL performed on the same datasets. Further, we investigate the combination of UNISEL and AL. Results indicate that UNISEL followed by an RF performs comparably to AL with an RF and that the combination of UNISEL and AL demonstrates superior performance. The practical implications of these findings in terms of time savings and generalizability afforded by UNISEL are discussed.

show abstract

modAL: A modular active learning framework for Python

Cited by 18 publications

References 0 publications

Trust and Believe – Should We? Evaluating the Trustworthiness of Twitter Users

Trust and Believe – Should We? Evaluating the Trustworthiness of Twitter Users

Addressing practical challenges in Active Learning via a hybrid query strategy

Unsupervised Instance Selection with Low-Label, Supervised Learning for Outlier Detection

Contact Info

Product

Resources

About