Advancements in On-Device Deep Neural Networks

Abstract

In recent years, rapid advancements in both hardware and software technologies have resulted in the ability to execute artificial intelligence (AI) algorithms on low-resource devices. The combination of high-speed, low-power electronic hardware and efficient AI algorithms is driving the emergence of on-device AI. Deep neural networks (DNNs) are highly effective AI algorithms used for identifying patterns in complex data. DNNs, however, contain many parameters and operations that make them computationally intensive to execute. Accordingly, DNNs are usually executed on high-resource backend processors. This causes an increase in data processing latency and energy expenditure. Therefore, modern strategies are being developed to facilitate the implementation of DNNs on devices with limited resources. This paper presents a detailed review of the current methods and structures that have been developed to deploy DNNs on devices with limited resources. Firstly, an overview of DNNs is presented. Next, the methods used to implement DNNs on resource-constrained devices are explained. Following this, the existing works reported in the literature on the execution of DNNs on low-resource devices are reviewed. The reviewed works are classified into three categories: software, hardware, and hardware/software co-design. Then, a discussion on the reviewed approaches is given, followed by a list of challenges and future prospects of on-device AI, together with its emerging applications.

Exploring the Potential of Distributed Computing Continuum Systems

“…Recently developed large language models (LLMs) are becoming popular due to their benefits, such as efficiency, customization, understanding different languages, and automating tasks. But LLMs require large computing resources; hence, all the data move to the cloud for computation [73][74][75][76]. Currently developed hybrid AI architectures can recommend different offload options based on factors like model complexity and query size to distribute processing among the cloud and other computing devices [77,78].…”