An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

Abstract

Deep learning (DL) is moving its step into a growing number of mobile software applications. These software applications, named as DL based mobile applications (abbreviated as mobile DL apps) integrate DL models trained using large-scale data with DL programs. A DL program encodes the structure of a desirable DL model and the process by which the model is trained using training data. Due to the increasing dependency of current mobile apps on DL, software engineering (SE) for mobile DL apps has become important. However, existing efforts in SE research community mainly focus on the development of DL models and extensively analyze faults in DL programs. In contrast, faults related to the deployment of DL models on mobile devices (named as deployment faults of mobile DL apps) have not been well studied. Since mobile DL apps have been used by billions of end users daily for various purposes including for safety-critical scenarios, characterizing their deployment faults is of enormous importance. To fill in the knowledge gap, this paper presents the first comprehensive study to date on the deployment faults of mobile DL apps. We identify 304 real deployment faults from Stack Overflow and GitHub, two commonly used data sources for studying software faults. Based on the identified faults, we construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault symptoms. Furthermore, we suggest actionable implications and research avenues that can potentially facilitate the deployment of DL models on mobile devices.

Understanding Performance Problems in Deep Learning Systems

“…Zhang et al [82] studied failures of deep learning jobs that are running on a remote, shared platform in Microsoft. Chen et al [8] investigated faults related to the deployment of deep learning models to mobile devices. Zhang et al [85] summarized five common training problems in deep learning systems, and developed a tool to automatically detect and repair training problems.…”

“…Zhang et al [82] identified GPU out of memory as a failure category of deep learning jobs. Chen et al [8] recognized memory and speed issues as two types of faults in the inference stage of deployment process. Wan et al [68] derived four performance-related API misuse patterns of cloud AI services.…”

“…A lot of efforts have been recently made to extensively investigate the characteristics (e.g., symptoms, root causes, fixes and taxonomy) of general bugs [26][27][28]86] and specific bugs [8,68,82,85] in DL systems. However, these studies are not specifically designed for PBs, and thus only capture some partial characteristics of PBs in DL systems.…”

Understanding Diffusion of Recurrent Innovations

“…Considerable effort in mobile app studies has been devoted to understanding the adoption of apps by mining the ratings, reviews, number of downloads, and other measures such as uninstalls [17,19,20,22,26,27] from app markets as indicators of users attitudes towards apps. Other research efforts focus on understanding the adoption of apps through in-app user behavior analysis, which including contextual data analysis [18], natural language data analysis [1,5,25], and so on [4,6,24,37,41]. However, there are some limitations from the perspective of innovation diffusion.…”

$C^3DRec$: Cloud-Client Cooperative Deep Learning for Temporal Recommendation in the Post-GDPR Era

“…To improve the accuracy of recommendation services on mobile devices, temporal recommendation techniques [20,38,40] have been proposed by leveraging temporal user behavior data generated on mobile devices [7,29,41], such as item clicks, dwell time, and revisitation frequency.…”