Towards multimodal deep learning for activity recognition on mobile devices

Radu, Valentin; Lane, Nicholas D.; Bhattacharya, Sourav; Mascolo, Cecilia; Marina, Mahesh K.; Kawsar, Fahim

doi:10.1145/2968219.2971461

Cited by 122 publications

(86 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The prior efforts on mobile DL can be mainly summarized into two categories. First, researchers have built numerous novel applications based on DL [46,72,73,80,87]. For example, MobileDeepPill [106] is a small-footprint mobile DL system that can accurately recognize unconstrained pill images.…”

Section: Related Workmentioning

confidence: 99%

A First Look at Deep Learning Apps on Smartphones

Liu

et al. 2019

The World Wide Web Conference

149

104

View full text Add to dashboard Cite

We are in the dawn of deep learning explosion for smartphones. To bridge the gap between research and practice, we present the first empirical study on 16,500 the most popular Android apps, demystifying how smartphone apps exploit deep learning in the wild. To this end, we build a new static tool that dissects apps and analyzes their deep learning functions. Our study answers threefold questions: what are the early adopter apps of deep learning, what do they use deep learning for, and how do their deep learning models look like. Our study has strong implications for app developers, smartphone vendors, and deep learning R&D. On one hand, our findings paint a promising picture of deep learning for smartphones, showing the prosperity of mobile deep learning frameworks as well as the prosperity of apps building their cores atop deep learning. On the other hand, our findings urge optimizations on deep learning models deployed on smartphones, protection of these models, and validation of research ideas on these models.

show abstract

Section: Related Workmentioning

confidence: 99%

A First Look at Deep Learning Apps on Smartphones

Liu

et al. 2019

The World Wide Web Conference

149

104

View full text Add to dashboard Cite

show abstract

“…Their goals are similar, while their learning processes are different. Both can be exploited to extract patterns from unlabeled mobile data, which may be subsequently employed for various supervised learning tasks, e.g., routing [186], mobile activity recognition [187], [188], periocular verification [189] and base station user number prediction [190].…”

Section: Auto-encodersmentioning

confidence: 99%

“…Analysis [17], [112], [235]- [266] [73], [97], [187], [267]- [291] Mobility Analysis [227], [292]- [310] User Localization [272], [273], [311]- [315] [111], [316]- [334] Wireless Sensor Networks [335]- [346], [346]- [356] Network Control [186], [293], [357]- [368] [234], [368]- [403] Network Security [185], [345], [404]- [419] [223], [420]- [429], [429]- [436] Signal Processing [378], [380], [437]- [444] [322], [445]- [458] Emerging Applications For each domain, we summarize work broadly in tabular form, providing readers with a general picture of individual topics. Most important works in each domain are discussed in more details in text.…”

Section: App-level Mobile Datamentioning

confidence: 99%

Deep Learning in Mobile and Wireless Networking: A Survey

Zhang

Patras

Haddadi

2019

IEEE Commun. Surv. Tutorials

1,335

838

View full text Add to dashboard Cite

The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure. Upcoming 5G systems are evolving to support exploding mobile traffic volumes, real-time extraction of fine-grained analytics, and agile management of network resources, so as to maximize user experience. Fulfilling these tasks is challenging, as mobile environments are increasingly complex, heterogeneous, and evolving. One potential solution is to resort to advanced machine learning techniques, in order to help manage the rise in data volumes and algorithm-driven applications. The recent success of deep learning underpins new and powerful tools that tackle problems in this space.In this paper we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas. We first briefly introduce essential background and state-of-theart in deep learning techniques with potential applications to networking. We then discuss several techniques and platforms that facilitate the efficient deployment of deep learning onto mobile systems. Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains. Drawing from our experience, we discuss how to tailor deep learning to mobile environments. We complete this survey by pinpointing current challenges and open future directions for research.

show abstract

“…Second, DL models can be reused for similar tasks, which makes HAR model construction more efficient. Different DL models such as deep neural networks [26,27], convolutional neural networks [10,28], autoencoders [11,29], restricted Boltzmann machines [12,30], and recurrent neural networks [31,32] have been applied in HAR. We refer readers to [8] for more details on DL-based HAR.…”

Section: Human Activity Recognitionmentioning

confidence: 99%

“…The capability to automatically extract high-level features makes DL methods largely alleviated from the drawbacks of conventional ML methods. There have been many DL-based HAR works proposed in recent years [10,11,12,13]. However, the training time and the amount of data required for DL systems are always much larger than that of traditional ML systems, and the large time and labor costs makes it difficult to build a large-scale labeled human activity dataset with high quality.…”

Section: Introductionmentioning

confidence: 99%

Few-shot learning-based human activity recognition

Feng

Duarte

2019

Expert Systems with Applications

View full text Add to dashboard Cite

Few-shot learning is a technique to learn a model with a very small amount of labeled training data by transferring knowledge from relevant tasks. In this paper, we propose a few-shot learning method for wearable sensor based human activity recognition, a technique that seeks high-level human activity knowledge from low-level sensor inputs. Due to the high costs to obtain human generated activity data and the ubiquitous similarities between activity modes, it can be more efficient to borrow information from existing activity recognition models than to collect more data to train a new model from scratch when only a few data are available for model training. The proposed few-shot human activity recognition method leverages a deep learning model for feature extraction and classification while knowledge transfer is performed in the manner of model parameter transfer. In order to alleviate negative transfer, we propose a metric to measure cross-domain class-wise relevance so that knowledge of higher relevance is assigned larger weights during knowledge transfer. Promising results in extensive experiments show the advantages of the proposed approach.

show abstract

Towards multimodal deep learning for activity recognition on mobile devices

Cited by 122 publications

References 3 publications

A First Look at Deep Learning Apps on Smartphones

A First Look at Deep Learning Apps on Smartphones

Deep Learning in Mobile and Wireless Networking: A Survey

Few-shot learning-based human activity recognition

Contact Info

Product

Resources

About