Compacting Deep Neural Networks for Internet of Things: Methods and Applications

Zhang, Ke; Ying, Hanbo; Dai, Hong-Ning; Li, Lin; Peng, Yuanyuan; Guo, Keyi; Yu, Hongfang

doi:10.1109/jiot.2021.3063497

Cited by 34 publications

(26 citation statements)

References 208 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of these techniques can be also utilized for modeling opponent agents based on the local information of the main agents [123]. Some knowledge distillation techniques such as pruning and low-rank decomposition can be also applied for original multi-agent systems to remove redundant information collected from distributed agents [124]. There are other abstraction techniques in the literature that reduce the number of local states by collapsing data values.…”

Section: Dimension Reduction and Filteringmentioning

confidence: 99%

A Survey of Adaptive Multi-Agent Networks and Their Applications in Smart Cities

Nezamoddini

Gholami

2022

Smart Cities

View full text Add to dashboard Cite

The world is moving toward a new connected world in which millions of intelligent processing devices communicate with each other to provide services in transportation, telecommunication, and power grids in the future’s smart cities. Distributed computing is considered one of the efficient platforms for processing and management of massive amounts of data collected by smart devices. This can be implemented by utilizing multi-agent systems (MASs) with multiple autonomous computational entities by memory and computation capabilities and the possibility of message-passing between them. These systems provide a dynamic and self-adaptive platform for managing distributed large-scale systems, such as the Internet-of-Things (IoTs). Despite, the potential applicability of MASs in smart cities, very few practical systems have been deployed using agent-oriented systems. This research surveys the existing techniques presented in the literature that can be utilized for implementing adaptive multi-agent networks in smart cities. The related literature is categorized based on the steps of designing and controlling these adaptive systems. These steps cover the techniques required to define, monitor, plan, and evaluate the performance of an autonomous MAS. At the end, the challenges and barriers for the utilization of these systems in current smart cities, and insights and directions for future research in this domain, are presented.

show abstract

Section: Dimension Reduction and Filteringmentioning

confidence: 99%

A Survey of Adaptive Multi-Agent Networks and Their Applications in Smart Cities

Nezamoddini

Gholami

2022

Smart Cities

View full text Add to dashboard Cite

show abstract

“…Various techniques have been proposed in the literature for compacting DNNs, see survey [13]. The conventional framework deals with scenarios where one has access to a highly-parameterized high-performance DNN, and aims at making it more compact.…”

Section: A Non-collaborative Edge Inferencementioning

confidence: 99%

“…For instance, the fact that a DNN is to be pruned or quantized can be accounted for in its training procedure, boosting the trained model to facilitate compaction. Furthermore, one can prefer deep architectures, such as convolutional networks with small kernels and shortcut connections [13], that are inherently more compact compared with conventional ones. An alternative strategy is to design DNNaided systems that utilize compact networks by incorporating statistical model-based domain knowledge and augmenting a suitable classic inference algorithm with trainable models, see, e.g., survey in [14].…”

Section: A Non-collaborative Edge Inferencementioning

confidence: 99%

Collaborative Inference for AI-Empowered IoT Devices

Shlezinger¹,

Bajić²

2022

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) technologies, and particularly deep learning systems, are traditionally the domain of largescale cloud servers, which have access to high computational and energy resources. Nonetheless, in Internet-of-Things (IoT) networks, the interface with the real-world is carried out using edge devices that are limited in hardware and can communicate. The conventional approach to provide AI processing to data collected by edge devices involves sending samples to the cloud, at the cost of latency, communication, connectivity, and privacy concerns. Consequently, recent years have witnessed a growing interest in enabling AI-aided inference on edge devices by leveraging their communication capabilities to establish collaborative inference. This article reviews candidate strategies for facilitating the transition of AI to IoT devices via collaboration. We identify the need to operate in different mobility and connectivity constraints as a motivating factor to consider multiple schemes, which can be roughly divided into methods where inference is done remotely, i.e., on the cloud, and those that infer on the edge. We identify the key characteristics of each strategy in terms of inference accuracy, communication latency, privacy, and connectivity requirements, providing a systematic comparison between existing approaches. We conclude by presenting future research challenges and opportunities arising from the concept of collaborative inference.

show abstract

“…Insufficient labeled data means that the training samples do not reflect the overall data well, resulting in poor generalization of the learning models. However, massive amounts of subjective text data are not naturally labeled with sentiment categories, and it is labor-intensive and financially demanding to process large-scale high-quality labeled data [ 5 ]. Semisupervised learning has emerged to improve the performance of models by leveraging costly unlabeled data.…”

Section: Introductionmentioning

confidence: 99%

Construction and Research on Chinese Semantic Mapping Based on Linguistic Features and Sparse Self-Learning Neural Networks

Zhang

Chao

Huang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In this paper, we adopt the algorithms of linguistic feature Rong and sparse self-learning neural network to conduct an in-depth study and analysis of Chinese semantic mapping, which complements the emotion semantic representation ability of traditional word embedding and fully explores the emotion semantic information contained in the text in the task preprocessing stage. We incorporate various semantic features such as lexical information and location information to make the model have richer emotion semantic expression, and the model also uses an attention mechanism to allow various features to interact and abstract deeper contextual internal semantic associations to improve the model's sentiment classification performance. Finally, experiments are conducted on two publicly available English sentiment classification corpora, and the results prove that the model outperforms other comparison models and effectively improves the sentiment classification performance. The model uses deep memory networks and capsule networks to construct a transfer learning framework and effectively leverages the transfer learning properties of capsule networks to transfer knowledge embedded in large-scale labeled data from similar domains to the target domain, improving the classification performance on small data sets. The use of multidimensional combined features compensates for the lack of a one-dimensional feature attention mechanism, while multiple domain category-based attention computation layers are superimposed to obtain deeper domain-specific sentiment feature information. Based on the segmented convolutional neural network, the model first introduces the dependent subtree of relational attributes to obtain the position weights of each word in the sentence, then introduces domain ontology knowledge in the output layer to constrain the extraction results, and conducts experimental comparison through the data set to verify the validity of the model, which ensures the accuracy of the network term entity and relational attribute recognition extraction and makes the knowledge map constructed in this paper. It ensures the accuracy of the extraction rate of the web term entities and relationship attributes and makes the knowledge map constructed in this paper more factual.

show abstract

Compacting Deep Neural Networks for Internet of Things: Methods and Applications

Cited by 34 publications

References 208 publications

A Survey of Adaptive Multi-Agent Networks and Their Applications in Smart Cities

A Survey of Adaptive Multi-Agent Networks and Their Applications in Smart Cities

Collaborative Inference for AI-Empowered IoT Devices

Construction and Research on Chinese Semantic Mapping Based on Linguistic Features and Sparse Self-Learning Neural Networks

Contact Info

Product

Resources

About