tinyMAN: Lightweight Energy Manager using Reinforcement Learning for Energy Harvesting Wearable IoT Devices

Abstract

Advances in low-power electronics and machine learning techniques lead to many novel wearable IoT devices. These devices have limited battery capacity and computational power. Thus, energy harvesting from ambient sources is a promising solution to power these low-energy wearable devices. They need to manage the harvested energy optimally to achieve energy-neutral operation, which eliminates recharging requirements. Optimal energy management is a challenging task due to the dynamic nature of the harvested energy and the battery energy constraints of the target device. To address this challenge, we present a reinforcement learning based energy management framework, tinyMAN, for resource-constrained wearable IoT devices. The framework maximizes the utilization of the target device under dynamic energy harvesting patterns and battery constraints. Moreover, tinyMAN does not rely on forecasts of the harvested energy which makes it a prediction-free approach. We deployed tinyMAN on a wearable device prototype using TensorFlow Lite for Micro thanks to its small memory footprint of less than 100 KB. Our evaluations show that tinyMAN achieves less than 2.36 ms and 27.75 μJ while maintaining up to 45% higher utility compared to prior approaches.

Energy-Sustainable IoT Connectivity: Vision, Technological Enablers, Challenges, and Future Directions

“…Energy Management is a promising solution to enable i) energy-neutral operation (ENO) in always-on IoT devices utilizing EH [290] and ii) efficient allocation of PV energy from a single-panel or off-grid system to multiple tasks [291]. In this regard, TinyML models can forecast future EH values to help devise a proactive ENO strategy for IoT devices.…”

“…In this regard, TinyML models can forecast future EH values to help devise a proactive ENO strategy for IoT devices. Besides, when the number of EH samples becomes insufficient to elaborate a forecast on the incoming ambient energy, energy management strategies can still benefit from TinyML by using the current battery state and previous EH measurements [290]. Again, PV power prediction is necessary for proper energy management/distribution among tasks.…”

“…However, this may not always be the most effective way to balance performance and EC in all scenarios, unless the EER metrics already include some guarantee for QoS support and/or maximum EC. If not, minimizing EC while maintaining QoS requirements, e.g., [135], [137], [140], [210], [335], or maximizing performance given energy constraints, e.g., [131], [202], [205], [290], [336], [337], may be more important.…”

“…In general, real-time data aggregation, advanced predictions, and adjustable algorithms are needed to ensure accurate predictions and efficient energy allocation [356]. For instance, at a device level, an RL-based TinyML algorithm using past samples from the EH process and the current battery level may judiciously allocate energy for future operations [290]. The main limitation of the approach in [290] is that the TinyML algorithm is trained offline which ignores that unpredictable changes in EH patterns can lead to improper energy management strategies.…”