Deep Reinforcement Learning-based Beam Tracking from mmWave Antennas Installed on Overhead Messenger Wires

Shinzaki, Masao; Koda, Yusuke; Yamamoto, Koji; Nishio, Takayuki; Morikura, Masahiro; Chun-hsiang, Huang; Shirato, Yushi; Kita, Naoki

doi:10.1109/vtc2020-fall49728.2020.9348475

Cited by 4 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we discuss the importance of addressing the gap between training and test scenarios in learning-based mmWave beam-tracking, which was previously proposed in [9], [26]. Specifically, through numerical evaluations, we show that an increased gap between training and test scenarios deteriorates the received power in the learning-based beamtracking.…”

Section: Motivation Of Zero-shot Adaptationmentioning

confidence: 93%

“…In our previous works [9], [26], the installation of SBSs on overhead messenger wires was proposed to gain flexibility in physical deployments of SBSs and to ensure LOS connections between SBSs and gateway BSs. These installations pose challenges of beam-tracking because complicated wind-forced dynamics in on-wire SBSs require frequent beam training, and consequently, a large signaling overhead.…”

Section: B Beam-tracking Based With Non-rf Informationmentioning

confidence: 99%

“…We evaluated the received signal power when there is a gap in several environmental parameters between the training and test scenarios. The numerical evaluation of learning-based beam-tracking in [9], [26], which is subject to training and test gaps pertaining to the spring constant and total wire mass using the beam-tracking method proposed in [9], is shown in Fig. 3, where the cross mark represents the spring constant and total wire mass in the training.…”

Section: A Disadvantage Without Zero-shot Adaptationmentioning

confidence: 99%

“…To overcome these limitations, we deploy an SBS on overhead messenger wires near a gateway BS, as illustrated in Fig. 1 [9]. Overhead messenger wires can provide SBSs with both electric power and an Internet connection, which aids in overcoming the aforementioned limitations.…”

Section: Introductionmentioning

confidence: 99%

“…However, the wind-forced dynamics of SBSs can induce beam misalignments. To prevent beam misalignments, a deep reinforcement learning (RL)-based beam-tracking method was proposed in [9]. The background pertaining to the use of a deep RL-based beam-tracking method is discussed in detail in the subsequent section.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires through Robust Adversarial Reinforcement Learning

Shinzaki¹,

Koda²,

Yamamoto³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper discusses the opportunity of bringing the concept of zero-shot adaptation into learning-based millimeterwave (mmWave) communication systems, particularly in environments with unstable urban infrastructures. Here, zero-shot adaptation implies that a learning agent adapts to unseen scenarios during training without any adaptive fine-tuning. By considering learning-based beam-tracking of a mmWave node placed on an overhead messenger wire, we first discuss the importance of zero-shot adaptation. More specifically, we confirm that the gap between the values of wire tension and total wire mass in training and test scenarios deteriorates the beam-tracking performance in terms of the received power. Motivated by this discussion, we propose a robust beam-tracking method to adapt to a broad range of test scenarios in a zero-shot manner, i.e., without requiring any retraining to adapt the scenarios. The key idea is to leverage a recent, robust adversarial reinforcement learning technique, where such training and test gaps are regarded as disturbances from adversaries. In our case, a beam-tracking agent performs training competitively bases on an intelligent adversary who causes beam misalignments. Numerical evaluations confirm the feasibility of zero-shot adaptation by showing that the on-wire node achieves feasible beam-tracking performance without any adaptive fine-tuning in unseen scenarios.

show abstract

Section: Motivation Of Zero-shot Adaptationmentioning

confidence: 93%

Section: B Beam-tracking Based With Non-rf Informationmentioning

confidence: 99%

Section: A Disadvantage Without Zero-shot Adaptationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires through Robust Adversarial Reinforcement Learning

Shinzaki¹,

Koda²,

Yamamoto³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Millimeter Wave Communications on Overhead Messenger Wire: Deep Reinforcement Learning-Based Predictive Beam Tracking

Koda

Shinzaki

Yamamoto

et al. 2021

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires Through Robust Adversarial Reinforcement Learning

Shinzaki¹,

Koda²,

Yamamoto³

et al. 2022

IEEE Trans. Cogn. Commun. Netw.

Self Cite

View full text Add to dashboard Cite

Millimeter wave (mmWave) beam-tracking based on machine learning enables the development of accurate tracking policies while obviating the need to periodically solve beamoptimization problems. However, its applicability is still arguable when training-test gaps exist in terms of environmental parameters that affect the node dynamics. From this skeptical point of view, the contribution of this study is twofold. First, by considering an example scenario, we confirm that the training-test gap adversely affects the beam-tracking performance. More specifically, we consider nodes placed on overhead messenger wires, where the node dynamics are affected by several environmental parameters, e.g, the wire mass and tension. Although these are particular scenarios, they yield insight into the validation of the trainingtest gap problems. Second, we demonstrate the feasibility of zeroshot adaptation as a solution, where a learning agent adapts to environmental parameters unseen during training. This is achieved by leveraging a robust adversarial reinforcement learning (RARL) technique, where such training-and-test gaps are regarded as disturbances by adversaries that are jointly trained with a legitimate beam-tracking agent. Numerical evaluations demonstrate that the beam-tracking policy learned via RARL can be applied to a wide range of environmental parameters without severely degrading the received power.

show abstract

Deep Reinforcement Learning-based Beam Tracking from mmWave Antennas Installed on Overhead Messenger Wires

Cited by 4 publications

References 16 publications

Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires through Robust Adversarial Reinforcement Learning

Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires through Robust Adversarial Reinforcement Learning

Millimeter Wave Communications on Overhead Messenger Wire: Deep Reinforcement Learning-Based Predictive Beam Tracking

Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires Through Robust Adversarial Reinforcement Learning

Contact Info

Product

Resources

About