Energy Efficient Two-Tier Data Dissemination Based on Q-Learning for Wireless Sensor Networks

Abstract

Green communication for different kinds of wireless networks has begun to receive significant research attention recently. Green communication focuses mainly on the issue of improving energy efficiency substantially. A wireless sensor network (WSN) consists of a large number of randomly and widely deployed sensor nodes, and these nodes themselves have the ability to wireless communicate, detect and process data. Sensor nodes can thus detect their surrounding environment, and transmit related data to a sink via wireless communication. This study proposes two two-tier data dissemination schemes based on Q-learning for wireless sensor networks. In the proposed schemes, a source node uses Q-learning to find the most energy efficient data dissemination path from the source node to the sink. The first scheme is called TTDD-QL, and the second scheme is called TTDD-QL-A which is an advanced version of TTDD-QL. In TTDD-QL, the reward is determined by the distance between the current dissemination node and the sink. In each iteration, the proposed scheme will update the Q values. After multiple learning iterations, the Q values are converged, and the data dissemination path is found according to the Q values. In TTDD-QL-A, the reward is determined not only by the distance between the current dissemination node and the sink but also by the remaining energy of the current dissemination node. Simulation results show that TTDD-QL and TTDD-QL-A can reduce sensor node energy consumption and extend the lifetime of the WSN.

A Safety Response Mechanism for an Autonomous Moving Robot in a Small Manufacturing Environment using Q-learning Algorithm and Speech Recognition

“…3.1 Q-learning algorithm to find obstacle free paths Q-learning is a reinforcement learning algorithm. One of its working principles is that the agent (the autonomous robot in our research) learns the best policy to adopt for a given scenario based on its interactions with the environment and the rewards gained [35]. According to its current state, a policy in Q-learning is how the agent chooses to behave, react, or make decisions at a given time.…”

“…The values in the Q table are computed based on a Bellman Equation [8]. Once the Q table is populated after several interactions, the agent uses the information in the Q table to choose the action giving the highest cumulative reward [35,49]. QL can be applied as a dynamic and incremental programming method to find the optimal strategy for a problem in a stepby-step learning mode.…”

“…The RL algorithm utilizes five main elements: the agent, the environment, the reward, the state, and the action. In its learning process, the agent performs several interactions with its environment by making some actions that will cause a change of state in the environment and result in a positive or a negative reward (penalty) [35]. The agent learns the impact of its actions based on the type of reward obtained.…”

An Experimental Safety Response Mechanism for an Autonomous Moving Robot in a Smart Manufacturing Environment Using Q-Learning Algorithm and Speech Recognition

“…The RL algorithm utilizes five main elements: the agent, the environment, the reward, the state, and the action. In its learning process, the agent performs several interactions with its environment by making some actions that will cause a change of state in the environment and result in a positive or a negative reward (penalty) [ 35 ]. Over the years, RL has been the subject of many kinds of research in various applications such as chemical reaction [ 36 ], resource management [ 37 ], traffic-light management [ 38 ], autonomous driving [ 39 ], dam management [ 40 ], surgery [ 41 ] and robotics [ 42 ].…”

Data-Driven Self-Learning Controller for Power-Aware Mobile Monitoring IoT Devices

“…QL algorithms can be used to iteratively change the MAC protocol parameters by a defined policy to achieve to a low energy state [32]. The TDMA-based adaptive task scheduling [33] method or two-tier data dissemination schemes based on Q-learning (TTDD-QL) [34] are energy efficient for wireless sensor networks (WSN). A cooperative energy-efficient model is presented in the article [35], where clustering, mobile sink deployment and variable sensing collaboratively improve the network lifetime.…”