An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices

Abstract

Detecting and reacting to user behavior and ambient context are core elements of many emerging mobile sensing and Internet-of-Things (IoT) applications. However, extracting accurate inferences from raw sensor data is challenging within the noisy and complex environments where these systems are deployed. Deep Learning-is one of the most promising approaches for overcoming this challenge, and achieving more robust and reliable inference. Techniques developed within this rapidly evolving area of machine learning are now state-of-the-art for many inference tasks (such as, audio sensing and computer vision) commonly needed by IoT and wearable applications. But currently deep learning algorithms are seldom used in mobile/IoT class hardware because they often impose debilitating levels of system overhead (e.g., memory, computation and energy). Efforts to address this barrier to deep learning adoption are slowed by our lack of a systematic understanding of how these algorithms behave at inference time on resource constrained hardware. In this paper, we present the first-albeit preliminary-measurement study of common deep learning models (such as Convolutional Neural Networks and Deep Neural Networks) on representative mobile and embedded platforms. The aim of this investigation is to begin to build knowledge of the performance characteristics, resource requirements and the execution bottlenecks for deep learning models when being used to recognize categories of behavior and context. The results and insights of this study, lay an empirical foundation for the development of optimization methods and execution environments that enable deep learning to be more readily integrated into next-generation IoT, smartphones and wearable systems.

A First Look at Deep Learning Apps on Smartphones

“…The prior efforts on mobile DL can be mainly summarized into two categories. First, researchers have built numerous novel applications based on DL [46,72,73,80,87]. For example, MobileDeepPill [106] is a small-footprint mobile DL system that can accurately recognize unconstrained pill images.…”

Breathing-Based Authentication on Resource-Constrained IoT Devices using Recurrent Neural Networks

“…For example, even audio analysis and speaker identification tasks have been performed using CNN and DNNs. In general, CNNs are good at exploiting features defined on spatial data (e.g., images), whereas RNNs are more appropriate for identifying Hence, compared to CNNs, RNN based models require comparatively lower computational power and memory [3], [7]. As discussed earlier, BreathPrint proposed a technique to authenticate users based on their breathing acoustics on mobile and IoT devices.…”

A write‐improved low‐power 12T SRAM cell for wearable wireless sensor nodes

“…All of these sensors capture and analyze the run time data by temporarily storing it into a memory, and thus, SRAM has been the repetitive architecture of this data storage and occupies the major portion of the system area. 3 Therefore, the power consumed by SRAM plays the crucial role in overall power consumed by the system.…”