Temporal Graph Traversals Using Reinforcement Learning With Proximal Policy Optimization

Abstract

Graphs in real-world applications are dynamic both in terms of structures and inputs. Information discovery in such networks, which present dense and deeply connected patterns locally and sparsity globally can be time consuming and computationally costly. In this paper we address the shortest path query in spatio-temporal graphs which is a fundamental graph problem with numerous applications. In spatiotemporal graphs, shortest path query classical algorithms are insufficient or even flawed because information consistency can not be guaranteed between two timestamps and path recalculation is computationally costly. In this work, we address the complexity and dynamicity of the shortest path query in spatio-temporal graphs with a simple, yet effective model based on Reinforcement Learning with Proximal Policy Optimization. Our solution simplifies the problem by decomposing the spatio-temporal graph in two components: a static and a dynamic sub-graph. The static graph, known and immutable, is efficiently solved with A * algorithm. The sub-graphs interconnecting the static graph have unknown dynamics and we address such issue by estimating the unknown dynamic portion of the graph as a Markov Chain which correlates the observations of the agents in the environment and the path to be followed. We then derive an action policy through Proximal Policy Optimization to select the local optimal actions in the Markov Process that will lead to the shortest path, given the estimated system dynamics. We evaluate the system in a simulation environment constructed in Unity3D. In partially structured and unknown environments, with variable environment parameters we've obtained an efficiency 75% greater than the comparable deterministic solution.

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

“…The surge is partly due to the uptick of research in a field of ML, called Deep Learning (DL), where thousands (even billions) of neuronal parameters are trained to generalize on carrying out a particular task. Successful use of DL algorithms in healthcare [1]- [3], ophthalmology [4]- [6], developmental disorders [7]- [9], in autonomous robots and vehicles [10]- [12], in image processing classification and detection [13], [14], in speech and audio processing [15], [16], cyber-security [17], [18], and many more indicate the reach of DL algorithms in our daily lives. Easier access to high-performance compute nodes using cloud computing ecosystems, high-throughput AI accelerators to enhance performance, and access to big-data scale datasets and storage enables deep learning providers to research, test, and operate ML algorithms at scale in small edge devices [19], smartphones [20], and AI-based web-services using Application Programming Interfaces (APIs) for wider exposure to any applications.…”

Few-shot image classification algorithm based on attention mechanism and weight fusion

“…In recent years, the development of deep learning has been in full swing [ 1 ]. It has become a research hotspot in the field of artificial intelligence and has been widely used in computer vision [ 2 , 3 ], natural language processing [ 4 , 5 ], video analytics [ 6 , 7 ], and cyber security [ 8 , 9 ]. Deep learning is rapidly growing due to the support of big data and the improvement of computing power.…”

Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey

“…Fueled by the fact that new frameworks, libraries, and hardware resources are being improved and made available to the public and scientific community [24], [25], [26], Deep Neural networks (DNN) are being improved constantly and achieving new performance breakthroughs [27], [28], [29]. With the current maturity of DNN algorithms, its being applied in solving safety and security-critical problems [30], such as self-driving cars [31], [32], multi-agent aerial vehicle systems with face identification [33], robotics [34], [35], social engineering detection [36], network anomaly detection [37], deep packet inspection in networks [38]. DNN applications are already part of our day-to-day life (personal assistants [39], product recommendation [40], biometric identification [41]) and tend to occupy a bigger space as time passes.…”