No abstract

No abstract

Improvements in wireless sensor network (WSN) technology have resulted in a large number of applications. WSNs have been mainly used for monitoring applications, but they are also applicable to target tracking, health care, and monitoring with multimedia data. Nodes are generally deployed in environments where the exhausted batteries of sensor nodes are difficult to charge or replace. The primary goal of communication protocols in WSNs is to maximize energy efficiency in order to prolong network lifetime. In this paper, various medium access control (MAC) protocols for synchronous/asynchronous and single/multi-channel WSNs are investigated. Single-channel MAC protocols are categorized into synchronous and asynchronous approaches, and the advantages and disadvantages of each protocol are presented. The different features required in multi-channel WSNs compared to single-channel WSNs are also investigated, and surveys on multi-channel MAC protocols proposed for WSNs are provided. Then, existing broadcast schemes in such MAC protocols and efficient multi-hop broadcast protocols proposed for WSNs are provided. The limitations and challenges in many communication protocols according to this survey are pointed out, which will help future researches on the design of communication protocols for WSNs.

Top-performing Model-Based Reinforcement Learning (MBRL) agents, such as DREAMER, learn the world model by reconstructing the image observations. Hence, they often fail to discard task-irrelevant details and struggle to handle visual distractions. To address this issue, previous work has proposed to contrastively learn the world model, but the performance tends to be inferior in the absence of distractions. In this paper, we seek to enhance robustness to distractions for MBRL agents. Specifically, we consider incorporating prototypical representations, which have yielded more accurate and robust results than contrastive approaches in computer vision. However, it remains elusive how prototypical representations can benefit temporal dynamics learning in MBRL, since they treat each image independently without capturing temporal structures. To this end, we propose to learn the prototypes from the recurrent states of the world model, thereby distilling temporal structures from past observations and actions into the prototypes. The resulting model, DREAMERPRO, successfully combines DREAMER with prototypes, making large performance gains on the DeepMind Control suite both in the standard setting and when there are complex background distractions. Code available at https://github.com/fdeng18/dreamer-pro.

No abstract

No abstract

Deep reinforcement learning (DRL) is a promising approach for developing control policies by learning how to perform tasks. Edge devices are required to control their actions by exploiting DRL to solve tasks autonomously in various applications such as smart manufacturing and autonomous driving. However, the resource limitations of edge devices make it unfeasible for them to train their policies from scratch. It is also impractical for such an edge device to use the policy with a large number of layers and parameters, which is pre-trained by a centralized cloud infrastructure with high computational power. In this paper, we propose a method, on-device DRL with distillation (OD3), to efficiently transfer distilled knowledge of how to behave for on-device DRL in resource-constrained edge computing systems. Our proposed method makes it possible to simultaneously perform knowledge transfer and policy model compression in a single training process on edge devices with considering their limited resource budgets. The novelty of our method is to apply a knowledge distillation approach to DRL based edge device control in integrated edge cloud environments. We analyze the performance of the proposed method by implementing it on a commercial embedded system-on-module equipped with limited hardware resources. The experimental results show that 1) edge policy training with the proposed method achieves near-cloud-performance in terms of average rewards, although the size of the edge policy network is significantly smaller compared to that of the cloud policy network and 2) the training time elapsed for edge policy training with our method is reduced significantly compared to edge policy training from scratch.INDEX TERMS Deep reinforcement learning, edge computing, edge AI, knowledge transfer, policy model compression, on-device training.

The traditional Internet of Things (IoT) paradigm has evolved towards intelligent IoT applications which exploit knowledge produced by IoT devices using artificial intelligence techniques. Knowledge sharing between IoT devices is a challenging issue in this trend. In this paper, we propose a Knowledge of Things (KoT) framework which enables sharing self-taught knowledge between IoT devices which require similar or identical knowledge without help from the cloud. The proposed KoT framework allows an IoT device to effectively produce, cumulate, and share its self-taught knowledge with other devices at the edge in the vicinity. This framework can alleviate behavioral repetition in users and computational redundancy in systems in intelligent IoT applications. To demonstrate the feasibility of the proposed concept, we examine a smart home case study and build a prototype of the KoT framework-based smart home system. Experimental results show that the proposed KoT framework reduces the response time to use intelligent IoT devices from a user’s perspective and the power consumption for compuation from a system’s perspective.