Tracking the State of Large Dynamic Networks via Reinforcement Learning

Andrews, Matthew; Borst, Sem; Lee, Jeong-Ran; Martín-López, Enrique; Palyutina, Karina

doi:10.1109/infocom41043.2020.9155484

Cited by 4 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where = 4 is a constant, ℎ , () is the corresponding Rayleigh fading channel coeicient, which is constantly changing. Similar to [2,29], the channel condition on ℎ , () in time slot is unknown for the central controller. Let () and () be the transmit powers of the th sensor and the th relay in time slot , respectively.…”

Section: Channel Modelmentioning

confidence: 99%

“…where 2 represents the additive white Gaussian noise (AWGN) power at relay . Similarly, the transmission rate between the th sensor and the edge server in time slot is…”

Section: Channel Modelmentioning

confidence: 99%

“…Similar to [39], decode-and-forward (DF) is adopted for cooperative communication, which contains two phases 2 . In phase I, source sensor transmits task to the relay; in phase II, relay transmits the task to the edge [3].…”

Section: Channel Modelmentioning

confidence: 99%

See 2 more Smart Citations

EECDN: Energy-efficient Cooperative DNN Edge Inference in Wireless Sensor Networks

Chen

Yao

et al. 2022

ACM Trans. Internet Technol.

View full text Add to dashboard Cite

Multi-access edge computing (MEC) is emerging to improve the quality of experience of mobile devices including internet of things (IoT) sensors by offloading computing intensive tasks to MEC servers. Existing MEC-enabled cooperative computation offloading works focus on the optimization of total energy consumption, while fail to exploit multi-relay diversity and min-max fairness of energy consumption on participated sensors. We explore a typical wireless sensor network with multi-source, multi-relay and one edge server, where relay nodes can provide both cooperative communication and computation services. We divide the energy efficiency optimization problem into two sub-problems: one is to minimize the weighted average total energy consumption per time slot, and the other is to minimize the maximum weighted energy consumption. For the first sub-problem, we propose an optimal algorithm named as OTCA based on bipartite matching. For the second sub-problem, a greedy algorithm named as GAF is proposed with an approximation ratio of (1 + ϵ), where ϵ is a small positive constant. Extensive numerical results show that, OTCA outperforms the baseline algorithms by \(26.7\%\sim 77.4\% \) on the average total weighted energy consumption while GAF outperforms benchmark algorithms by \(30.7\%\sim 84.4\% \) . NS-3 simulation experiments comply with numerical results.

show abstract

Section: Channel Modelmentioning

confidence: 99%

“…where 2 represents the additive white Gaussian noise (AWGN) power at relay . Similarly, the transmission rate between the th sensor and the edge server in time slot is…”

Section: Channel Modelmentioning

confidence: 99%

See 1 more Smart Citation

EECDN: Energy-efficient Cooperative DNN Edge Inference in Wireless Sensor Networks

Chen

Yao

et al. 2022

ACM Trans. Internet Technol.

View full text Add to dashboard Cite

show abstract

“…1) Small, expensive and frequently changing (5%) 2) Large, inexpensive and infrequently changing (80%) 3) Moderately large, expensive and infrequently changing (10%) 4) Small, inexpensive and frequently changing (5%) Let λ i , i ∈ {1, 2, 3, 4} represents the actual rates at which each page changes in group i and let N i be the number of pages in group i. Note that these rate should satisfy [2] following condition,…”

Section: Numerical Examplesmentioning

confidence: 99%

Deep Reinforcement Learning for Web Crawling

Avrachenkov

Borkar

Patil

2021

2021 Seventh Indian Control Conference (ICC)

View full text Add to dashboard Cite

A search engine uses a web crawler to crawl the pages from the world wide web (WWW) and aims to maintain its local cache as fresh as possible. Unfortunately, the rates at which different pages change in WWW are highly nonuniform and also, unknown in many real-life scenarios. In addition, the finite available bandwidth and possible server restrictions on crawling frequency make it very difficult for the crawler to find the optimal scheduling policy that maximises the freshness of the local cache. We model this problem in a multi-armed restless bandits framework, where each arm represents a web page or an aggregate of statistically identical web pages. The objective is to find the scheduling policy that gives the exact indices of the pages to be crawled at a particular instance. We provide an online learning scheme using deep reinforcement learning (DRL) framework which learns the unknown page change dynamics on the fly along with the optimal crawling policy. Finally, we run numerical simulations to compare our approach with state-of-the-art algorithms such as static optimisation and Thompson sampling. We observe better performance for DRL.

show abstract

“…As we show in Section 3.5, it is also possible to view our second and third estimators as a Stochastic Gradient Descent (SGD) method and as an SGD method with heavy-ball momentum, respectively. Even though we present our results in the context of web crawling, our algorithms are equally applicable to the synchronization of databases [7] and the problem of network inventory management [20].…”

Section: Introductionmentioning

confidence: 99%