A Reinforcement Learning Framework for Dynamic Power Management of a Portable, Multi-camera Traffic Monitoring System

Khan, Umair Ali; Rinner, Bernhard

doi:10.1109/greencom.2012.85

Cited by 12 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare the performance of our proposed DPM technique with some of the techniques found in the literature, including fixed timeout, adaptive timeout [Douglis et al 1995], ML-ANN predictive policy based only on our workload estimation, exponential predictive [Hwang and Wu 2000], and online Timeout/N policy [Khan and Rinner 2012a] which learns timeout policies only for the idle state and takes wakeup decisions based on the queue occupancy. The comparison was performed on the same workload (workload 1).…”

Section: Exploring Power-performance Trade-offmentioning

confidence: 98%

“…We address this issue in our prior work [Khan and Rinner 2012a;Khan et al 2012] by proposing a model-free, RL-based DPM algorithm for nonstationary workloads. We use an ML-ANN-based workload estimator with backpropagation algorithm to provide estimated workload information to the learning algorithm.…”

Section: Machine Learning Policiesmentioning

confidence: 98%

“…-The convergence (speed) of the RL algorithm is an issue in many problems. However, this problem can be overcome by optimal design of state space, balanced selection of random versus deterministic actions, and/or multiple states update with a slightly increased overhead [Khan and Rinner 2012b]. …”

Section: Machine Learning Policiesmentioning

confidence: 99%

See 2 more Smart Citations

Online learning of timeout policies for dynamic power management

Khan

Rinner

2014

ACM Trans. Embed. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

Dynamic power management (DPM) refers to strategies which selectively change the operational states of a device during runtime to reduce the power consumption based on the past usage pattern, the current workload, and the given performance constraint. The power management problem becomes more challenging when the workload exhibits nonstationary behavior which may degrade the performance of any single or static DPM policy.This article presents a reinforcement learning (RL)-based DPM technique for optimal selection of timeout values in the different device states. Each timeout period determines how long the device will remain in a particular state before the transition decision is taken. The timeout selection is based on workload estimates derived from a Multilayer Artificial Neural Network (ML-ANN) and an objective function given by weighted performance and power parameters. Our DPM approach is further able to adapt the powerperformance weights online to meet user-specified power and performance constraints, respectively. We have completely implemented our DPM algorithm on our embedded traffic surveillance platform and performed long-term experiments using real traffic data to demonstrate the effectiveness of the DPM. Our results show that the proposed learning algorithm not only adequately explores the power-performance trade-off with nonstationary workload but can also successfully perform online adjustment of the trade-off parameter in order to meet the user-specified constraint. ACM Reference Format: U. A. Khan and B. Rinner. 2014. Online learning of timeout policies for dynamic power management. ACM Trans. Embedd. Comput.

show abstract

Section: Exploring Power-performance Trade-offmentioning

confidence: 98%

Section: Machine Learning Policiesmentioning

confidence: 98%

See 1 more Smart Citation

Online learning of timeout policies for dynamic power management

Khan

Rinner

2014

ACM Trans. Embed. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Q learning [8] is a technique which is often used to select these actions, even when the agent has no full knowledge about the reward and state transition functions. In each state the agent basically can choose from two kinds of behavior: either it can explore the state space or it can exploit the information already present in the Q values.…”

Section: System Modelmentioning

confidence: 99%

Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning

Khan

Rinner

2014

2014 IEEE International Conference on Communications Workshops (ICC)

Self Cite

View full text Add to dashboard Cite

Wireless sensor networks (WSN) are an attractive platform for cyber physical systems. A typical WSN application is composed of different tasks which need to be scheduled on each sensor node. However, the severe energy limitations pose a particular challenge for developing WSN applications, and the scheduling of tasks has typically a strong influence on the achievable performance and energy consumption. In this paper we propose a method for scheduling the tasks using cooperative reinforcement learning (RL) where each node determines the next task based on the observed application behavior. In this RL framework we can trade the application performance and the required energy consumption by a weighted reward function and can therefore achieve different energy/performance results of the overall application. By exchanging data among neighboring nodes we can further improve this energy/performance tradeoff. We evaluate our approach in an target tracking application. Our simulations show that cooperative approaches are superior to non-cooperative approaches for this kind of applications.

show abstract

“…Many studies have focused on resource efficiency for embedded smart cameras. Khan and Rinner [7] present a Reinforcement Learning (RL)-based Dynamic Power Management technique on smart cameras for traffic monitoring. Casares et al [2] decrease the energy consumption during object detection and tracking by estimating the region of interest and down sampling at hardware level.…”

Section: Introductionmentioning

confidence: 99%

Energy Efficient Tracking by Dynamic Voltage and Frequency Scaling on Android Smart Phones

Zheng

Özcan

Velipasalar

et al. 2014

Proceedings of the International Conference on Distributed Smart Cameras

View full text Add to dashboard Cite

Mobile smart phones and tablets provide not only high resolution cameras but also powerful processing capabilities. Yet, these devices consume more energy at the same time. Thus, energy efficiency and battery life have been extremely important topics for mobile devices. Dynamic Voltage and Frequency Scaling (DVFS) has been commonly used to reduce energy consumption for digital systems. In this paper, we propose a novel approach, which integrates DVFS to smart phones performing object tracking. If the processor is continuously run at the highest frequency, this would provide higher frame processing rate, but would consume significantly more energy. If the lowest CPU frequency is used, energy consumption would be decreased, but the frame processing rate would be low. In the proposed approach, instead of running at a certain set frequency, The CPU frequency is adjusted depending on the scene so that energy consumption is decreased without significantly lowering the frame processing rate. The tracking algorithm employed is based on the color-blob tracking provided by OpenCV. All the experiments have been performed on Android smart phones, both Google Nexus S and Samsung Galaxy S3. We measured the real-time energy consumption together with the frame per second processing rate as the metrics. The results show that by introducing DVFS, the proposed method provides 24.29% and 10.3% savings in energy consumption on Nexus S and Galaxy S3, respectively, compared to using the maximum frequency setting.

show abstract

A Reinforcement Learning Framework for Dynamic Power Management of a Portable, Multi-camera Traffic Monitoring System

Cited by 12 publications

References 13 publications

Online learning of timeout policies for dynamic power management

Online learning of timeout policies for dynamic power management

Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning

Energy Efficient Tracking by Dynamic Voltage and Frequency Scaling on Android Smart Phones

Contact Info

Product

Resources

About