A Quantitative Review of Automated Neural Search and On-Device Learning for Tiny Devices

Pau, Danilo Pietro; Ambrose, Prem Kumar; Aymone, Fabrizio Maria

doi:10.3390/chips2020008

Cited by 5 publications

(1 citation statement)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance of the AI models, in fact, deteriorates as time passes since the last training cycle; phenomenon known as concept drift [16], hence mandates model's parameter updates from time to time. The prominent field of on-device learning (ODL) [17] allows for machine learning (ML) models deployed on edge devices to adapt to the continuously changing data statistics, which are collected by the sensors, and performing model's parameter training. This paper explores the application of novel learning methods, PEPITA and MEMPEPITA, to Transformer-based Large Language Models (LLMs) for ODL on edge devices.…”

Section: Introductionmentioning

confidence: 99%

Forward Learning of Large Language Models by Consumer Devices

Pau,

Aymone

2024

Electronics

Self Cite

View full text Add to dashboard Cite

Large Language Models achieve state of art performances on a broad variety of Natural Language Processing tasks. In the pervasive IoT era, their deployment on edge devices is more compelling than ever. However, their gigantic model footprint has hindered on-device learning applications which enable AI models to continuously learn and adapt to changes over time. Back-propagation, in use by the majority of deep learning frameworks, is computationally intensive and requires storing intermediate activations into memory to cope with the model’s weights update. Recently, “Forward-only algorithms” have been proposed since they are biologically plausible alternatives. By applying more “forward” passes, this class of algorithms can achieve memory reductions with respect to more naive forward-only approaches and by removing the need to store intermediate activations. This comes at the expense of increased computational complexity. This paper considered three Large Language Model: DistilBERT, GPT-3 Small and AlexaTM. It investigated quantitatively any improvements about memory usage and computational complexity brought by known approaches named PEPITA and MEMPEPITA with respect to backpropagation. For low number of tokens in context, and depending on the model, PEPITA increases marginally or reduces substantially arithmetic operations. On the other hand, for large number of tokens in context, PEPITA reduces computational complexity by 30% to 50%. MEMPEPITA increases PEPITA’s complexity by one third. About memory, PEPITA and backpropagation, require a comparable amount of memory to store activations, while MEMPEPITA reduces it by 50% to 94% with the benefits being more evident for architectures with a long sequence of blocks. In various real case scenarios, MEMPEPITA’s memory reduction was essential for meeting the tight memory requirements of 128 MB equipped edge consumer devices, which are commonly available as smartphone and industrial application multi processors.

show abstract

Section: Introductionmentioning

confidence: 99%

Forward Learning of Large Language Models by Consumer Devices

Pau,

Aymone

2024

Electronics

Self Cite

View full text Add to dashboard Cite

show abstract

A Machine Learning-Oriented Survey on Tiny Machine Learning

Capogrosso,

Cunico,

Cheng

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The emergence of Tiny Machine Learning (TinyML) has positively revolutionized the field of Artificial Intelligence by promoting the joint design of resource-constrained IoT hardware devices and their learning-based software architectures. TinyML carries an essential role within the fourth and fifth industrial revolutions in helping societies, economies, and individuals employ effective AI-infused computing technologies (e.g., smart cities, automotive, and medical robotics). Given its multidisciplinary nature, the field of TinyML has been approached from many different angles: this comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions. The survey is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological flow, allowing for a systematic and complete literature survey. In particular, firstly, we will examine the three different workflows for implementing a TinyML-based system, i.e., MLoriented, HW-oriented, and co-design. Secondly, we propose a taxonomy that covers the learning panorama under the TinyML lens, examining in detail the different families of model optimization and design, as well as the state-of-the-art learning techniques. Thirdly, this survey will present the distinct features of hardware devices and software tools that represent the current state-of-the-art for TinyML intelligent edge applications. Finally, we discuss the challenges and future directions.

show abstract

A Practical Framework for Designing and Deploying Tiny Deep Neural Networks on Microcontrollers

Zhuang,

Pau

2024

2024 IEEE International Conference on Consumer Electronics (ICCE)

View full text Add to dashboard Cite

A Quantitative Review of Automated Neural Search and On-Device Learning for Tiny Devices

Cited by 5 publications

References 34 publications

Forward Learning of Large Language Models by Consumer Devices

Forward Learning of Large Language Models by Consumer Devices

A Machine Learning-Oriented Survey on Tiny Machine Learning

A Practical Framework for Designing and Deploying Tiny Deep Neural Networks on Microcontrollers

Contact Info

Product

Resources

About