Robust and Efficient Uncertainty Aware Biosignal Classification via Early Exit Ensembles

Campbell, Alexander; Qendro, Lorena; Lió, Píetro; Mascolo, Cecilia

doi:10.1109/icassp43922.2022.9746330

Cited by 2 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The output of early exits ensemble can also be computed in a multitude of ways, e.g., as arithmetic mean of the predictions [19], [20], via geometric ensemble [11] or through voting strategies [12]. The effectiveness of early exits ensembles is not limited to image classification, as they've also recently been used for image captioning [22], natural language processing [12], for uncertainty quantification and biosignal classification [13], [20] and to improve robustness against adversarial attacks [19]. Moreover, early exits ensembles were employed to produce a teacher-free knowledge distillation technique by treating the aggregated predictions as the teacher predictions [23].…”

Section: Related Workmentioning

confidence: 99%

“…The experiments are conducted on a large set of ConvNets and datasets and both in a traditional training context and through the tuning of pre-trained architectures. Unlike the main reference works in the literature, which aim to reduce latency as much as possible without sacrificing model accuracy or to develop techniques for edge computing [11]- [13], this work aims to quantify and maximise the accuracy gain that the use of early exits' ensemble can provide.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Anticipate, Ensemble and Prune: Improving Convolutional Neural Networks via Aggregated Early Exits

Sarti¹,

Lomurno²,

Matteucci³

2023

Preprint

View full text Add to dashboard Cite

Today, artificial neural networks are the state of the art for solving a variety of complex tasks, especially in image classification. Such architectures consist of a sequence of stacked layers with the aim of extracting useful information and having it processed by a classifier to make accurate predictions. However, intermediate information within such models is often left unused. In other cases, such as in edge computing contexts, these architectures are divided into multiple partitions that are made functional by including early exits, i.e. intermediate classifiers, with the goal of reducing the computational and temporal load without extremely compromising the accuracy of the classifications. In this paper, we present Anticipate, Ensemble and Prune (AEP), a new training technique based on weighted ensembles of early exits, which aims at exploiting the information in the structure of networks to maximise their performance. Through a comprehensive set of experiments, we show how the use of this approach can yield average accuracy improvements of up to 15% over traditional training. In its hybrid-weighted configuration, AEP's internal pruning operation also allows reducing the number of parameters by up to 41%, lowering the number of multiplications and additions by 18% and the latency time to make inference by 16%. By using AEP, it is also possible to learn weights that allow early exits to achieve better accuracy values than those obtained from single-output reference models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Anticipate, Ensemble and Prune: Improving Convolutional Neural Networks via Aggregated Early Exits

Sarti¹,

Lomurno²,

Matteucci³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The uncertainty in the environment, referred to as aleatory uncertainty, originates from intrinsic stochasticity and persists even after the environment's model has been learned [4]. While incorporating and quantifying both uncertainties in supervised learning has been explored [5,6], this problem in RL is not yet well-understood [1,2]. Considering uncertainty for decision-making leads to a risk-sensitive policy, where being pessimistic or optimistic about aleatory uncertainty creates a risk-averse or risk-seeking policy, respectively.…”

Section: Introductionmentioning

confidence: 99%

A Unified Uncertainty-Aware Exploration: Combining Epistemic and Aleatory Uncertainty

Malekzadeh

Hou

Plataniotis

2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Exploration is a significant challenge in practical reinforcement learning (RL), and uncertainty-aware exploration that incorporates the quantification of epistemic and aleatory uncertainty has been recognized as an effective exploration strategy. However, capturing the combined effect of aleatory and epistemic uncertainty for decision-making is difficult. Existing works estimate aleatory and epistemic uncertainty separately and consider the composite uncertainty as an additive combination of the two. Nevertheless, the additive formulation leads to excessive risk-taking behavior, causing instability. In this paper, we propose an algorithm that clarifies the theoretical connection between aleatory and epistemic uncertainty, unifies aleatory and epistemic uncertainty estimation, and quantifies the combined effect of both uncertainties for a risk-sensitive exploration. Our method builds on a novel extension of distributional RL that estimates a parameterized return distribution whose parameters are random variables encoding epistemic uncertainty. Experimental results on tasks with exploration and risk challenges show that our method outperforms alternative approaches.

show abstract

Robust and Efficient Uncertainty Aware Biosignal Classification via Early Exit Ensembles

Cited by 2 publications

References 11 publications

Anticipate, Ensemble and Prune: Improving Convolutional Neural Networks via Aggregated Early Exits

Anticipate, Ensemble and Prune: Improving Convolutional Neural Networks via Aggregated Early Exits

A Unified Uncertainty-Aware Exploration: Combining Epistemic and Aleatory Uncertainty

Contact Info

Product

Resources

About