Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers

Andersen, Jakob Smedegaard; Maalej, Walid

doi:10.18653/v1/2022.findings-acl.121

Cited by 7 publications

(2 citation statements)

References 21 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, solutions based on DistilBERT are inspired. Andersen and Maalej [ 26 ] proposed a framework for the efficient, in-operation moderation of classifier output to maximize the accuracy and increase the overall acceptance of text classifiers. Chang et al [ 27 ] developed a universal financial fraud awareness model to avoid these cases escalating to the level of crime.…”

Section: Related Workmentioning

confidence: 99%

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Tong¹,

Wang²,

Rui³

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Text classification plays an important role in many practical applications. In the real world, there are extremely small datasets. Most existing methods adopt pretrained neural network models to handle this kind of dataset. However, these methods are either difficult to deploy on mobile devices because of their large output size or cannot fully extract the deep semantic information between phrases and clauses. This paper proposes a multimodel-based deep learning framework for short-text multiclass classification with an imbalanced and extremely small dataset. Our framework mainly includes five layers: the encoder layer, the word-level LSTM network layer, the sentence-level LSTM network layer, the max-pooling layer, and the SoftMax layer. The encoder layer uses DistilBERT to obtain context-sensitive dynamic word vectors that are difficult to represent in traditional feature engineering methods. Since the transformer part of this layer is distilled, our framework is compressed. Then, we use the next two layers to extract deep semantic information. The output of the encoder layer is sent to a bidirectional LSTM network, and the feature matrix is extracted hierarchically through the LSTM at the word and sentence level to obtain the fine-grained semantic representation. After that, the max-pooling layer converts the feature matrix into a lower-dimensional matrix, preserving only the obvious features. Finally, the feature matrix is taken as the input of a fully connected SoftMax layer, which contains a function that can convert the predicted linear vector into the output value as the probability of the text in each classification. Extensive experiments on two public benchmarks demonstrate the effectiveness of our proposed approach on an extremely small dataset. It retains the state-of-the-art baseline performance in terms of precision, recall, accuracy, and F1 score, and through the model size, training time, and convergence epoch, we can conclude that our method can be deployed faster and lighter on mobile devices.

show abstract

Section: Related Workmentioning

confidence: 99%

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Tong¹,

Wang²,

Rui³

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…On the other hand, the teacher model inevitably generates some noisy labels that cause error accumulation (Wang et al 2021a). Some sample selection strategies (e.g., model confidence, uncertainty estimation) and consistency regularization mitigate the effect of noisy labels and alleviate the problem of confirmation bias (Do, Tran, and Venkatesh 2021;Cao et al 2021;Rizve et al 2021;Wang et al 2021b;Andersen and Maalej 2022). However, it is unclear how these methods can be applied to token-level classification.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty-Aware Self-Training for Low-Resource Neural Sequence Labeling

Wang

Huang

et al. 2023

AAAI

View full text Add to dashboard Cite

Neural sequence labeling (NSL) aims at assigning labels for input language tokens, which covers a broad range of applications, such as named entity recognition (NER) and slot filling, etc. However, the satisfying results achieved by traditional supervised-based approaches heavily depend on the large amounts of human annotation data, which may not be feasible in real-world scenarios due to data privacy and computation efficiency issues. This paper presents SeqUST, a novel uncertain-aware self-training framework for NSL to address the labeled data scarcity issue and to effectively utilize unlabeled data. Specifically, we incorporate Monte Carlo (MC) dropout in Bayesian neural network (BNN) to perform uncertainty estimation at the token level and then select reliable language tokens from unlabeled data based on the model confidence and certainty. A well-designed masked sequence labeling task with a noise-robust loss supports robust training, which aims to suppress the problem of noisy pseudo labels. In addition, we develop a Gaussian-based consistency regularization technique to further improve the model robustness on Gaussian-distributed perturbed representations. This effectively alleviates the over-fitting dilemma originating from pseudo-labeled augmented data. Extensive experiments over six benchmarks demonstrate that our SeqUST framework effectively improves the performance of self-training, and consistently outperforms strong baselines by a large margin in low-resource scenarios.

show abstract

More Sustainable Text Classification via Uncertainty Sampling and a Human-in-the-Loop

Andersen

Zukunft

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers

Cited by 7 publications

References 21 publications

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Uncertainty-Aware Self-Training for Low-Resource Neural Sequence Labeling

More Sustainable Text Classification via Uncertainty Sampling and a Human-in-the-Loop

Contact Info

Product

Resources

About