FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applications

Kwon, Young Dae; Chauhan, Jagmohan; Mascolo, Cecilia

doi:10.21437/interspeech.2021-1091

Cited by 11 publications

(4 citation statements)

References 21 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For other datasets (CIFAR-10, SVHN, GTSRB, GSC), we use variants of Mi-croNet architecture to construct pretrained models. To identify a high-performing and yet lightweight model to operate on embedded and mobile devices, we conduct a hyper-parameter search based on different variants of MicroNet (e.g., small, medium, large models), lightweight convolutional neural network (CNN) architectures [38], the number of convolutional filters. A basic convolutional layer consists of 3 × 3 convolution, batch normalization, and Rectified Linear Unit (ReLU).…”

Section: Performancementioning

confidence: 99%

YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers

Kwon¹,

Chauhan²,

Mascolo³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Internet of Things (IoT) systems provide large amounts of data on all aspects of human behavior. Machine learning techniques, especially deep neural networks (DNN), have shown promise in making sense of this data at a large scale. Also, the research community has worked to reduce the computational and resource demands of DNN to compute on low-resourced microcontrollers (MCUs). However, most of the current work in embedded deep learning focuses on solving a single task efficiently, while the multi-tasking nature and applications of IoT devices demand systems that can handle a diverse range of tasks (such as activity, gesture, voice, and context recognition) with input from a variety of sensors, simultaneously.In this paper, we propose YONO, a product quantization (PQ) based approach that compresses multiple heterogeneous models and enables in-memory model execution and model switching for dissimilar multi-task learning on MCUs. We first adopt PQ to learn codebooks that store weights of different models. Also, we propose a novel network optimization and heuristics to maximize the compression rate and minimize the accuracy loss. Then, we develop an online component of YONO for efficient model execution and switching between multiple tasks on an MCU at run time without relying on an external storage device.YONO shows remarkable performance as it can compress multiple heterogeneous models with negligible or no loss of accuracy up to 12.37×. Furthermore, YONO's online component enables an efficient execution (latency of 16-159 ms and energy consumption of 3.8-37.9 mJ per operation) and reduces model loading/switching latency and energy consumption by 93.3-94.5% and 93.9-95.0%, respectively, compared to external storage access. Interestingly, YONO can compress various architectures trained with datasets that were not shown during YONO's offline codebook learning phase showing the generalizability of our method. To summarize, YONO shows great potential and opens further doors to enable multi-task learning systems on extremely resource-constrained devices. CCS CONCEPTS• Computer systems organization → Embedded and cyberphysical systems.

show abstract

Section: Performancementioning

confidence: 99%

YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers

Kwon¹,

Chauhan²,

Mascolo³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The first group of approaches is a regularization-based method [22,43,45,55] where regularization terms are added to the loss function to minimize changes to important weights of a model for previous tasks to prevent forgetting. Another group of approaches is a replay-based method [28] where model parameters are updated for learning a representation by using training data of the currently available classes, which is different from replay with exemplars-based method [23,29,40] where updating the model requires training data from the new class and also few training samples from earlier classes.…”

Section: Continual Learningmentioning

confidence: 99%

Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications

Kwon¹,

Chauhan²,

Kumar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Continual learning approaches help deep neural network models adapt and learn incrementally by trying to solve catastrophic forgetting. However, whether these existing approaches, applied traditionally to image-based tasks, work with the same efficacy to the sequential time series data generated by mobile or embedded sensing systems remains an unanswered question.To address this void, we conduct the first comprehensive empirical study that quantifies the performance of three predominant continual learning schemes (i.e., regularization, replay, and replay with examples) on six datasets from three mobile and embedded sensing applications in a range of scenarios having different learning complexities. More specifically, we implement an end-to-end continual learning framework on edge devices. Then we investigate the generalizability, trade-offs between performance, storage, computational costs, and memory footprint of different continual learning methods.Our findings suggest that replay with exemplars-based schemes such as iCaRL has the best performance trade-offs, even in complex scenarios, at the expense of some storage space (few MBs) for training examples (1% to 5%). We also demonstrate for the first time that it is feasible and practical to run continual learning ondevice with a limited memory budget. In particular, the latency on two types of mobile and embedded devices suggests that both incremental learning time (few seconds -4 minutes) and training time (1 -75 minutes) across datasets are acceptable, as training could happen on the device when the embedded device is charging thereby ensuring complete data privacy. Finally, we present some guidelines for practitioners who want to apply a continual learning paradigm for mobile sensing tasks. CCS CONCEPTS• Human-centered costmputing → Ubiquitous and mobile computing; • Computing methodologies → Lifelong machine learning; • General and reference → Empirical studies.

show abstract

“…This leads to higher computational costs as the model expands and prohibits the utilization of compile-time optimizations on a fixed computation graph of the model. The last group of approaches among conventional CL includes rehearsal-based methods [7,8,26,49,63,67,74,81,98]. These prevent forgetting by replaying the saved rehearsal samples from earlier classes, typically leading to superior CL performance over the other methods at the cost of increased memory footprint.…”

Section: Introductionmentioning

confidence: 99%

LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms

Kwon,

Chauhan,

Jia

et al. 2023

Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems

View full text Add to dashboard Cite

Continual Learning (CL) allows applications such as user personalization and household robots to learn on the fly and adapt to context. This is an important feature when context, actions, and users change. However, enabling CL on resource-constrained embedded systems is challenging due to the limited labeled data, memory, and computing capacity.In this paper, we propose LifeLearner, a hardware-aware meta continual learning system that drastically optimizes system resources (lower memory, latency, energy consumption) while ensuring high accuracy. Specifically, we (1) exploit meta-learning and rehearsal strategies to explicitly cope with data scarcity issues and ensure high accuracy, (2) effectively combine lossless and lossy compression to significantly reduce the resource requirements of CL and rehearsal samples, and (3) developed hardware-aware system on embedded and IoT platforms considering the hardware characteristics.As a result, LifeLearner achieves near-optimal CL performance, falling short by only 2.8% on accuracy compared to an Oracle baseline. With respect to the state-of-the-art (SOTA) Meta CL method, LifeLearner drastically reduces the memory footprint (by 178.7×), end-to-end latency by 80.8-94.2%, and energy consumption by 80.9-94.2%. In addition, we successfully deployed LifeLearner on two edge devices and a microcontroller unit, thereby enabling efficient CL on resource-constrained platforms where it would be impractical to run SOTA methods and the far-reaching deployment of adaptable CL in a ubiquitous manner. Code is available at https://github.com/theyoungkwon/LifeLearner. CCS CONCEPTS• Computer systems organization → Embedded and cyber-physical systems; • Human-centered computing → Ubiquitous and mobile computing.

show abstract

FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applications

Cited by 11 publications

References 21 publications

YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers

YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers

Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications

LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms

Contact Info

Product

Resources

About