ISRL: intelligent search by reinforcement learning in unstructured peer-to-peer networks

Abstract

Existing searches in unstructured peer-to-peer networks are either blind or informed based on simple heuristics. Blind schemes suffer from low query quality. Simple heuristics lack theoretical background to support the simulation results. In this paper, we propose an intelligent searching scheme, called ISRL (Intelligent Search by Reinforcement Learning), which systematically seeks the best route to desired files by reinforcement learning (RL). In artificial intelligence, RL has been proven to be able to learn the best sequence of actions to achieve a certain goal. To discover the best path to desired files, ISRL not only explores new paths by forwarding queries to randomly chosen neighbors, but also exploits the paths that have been discovered for reducing the cumulative query cost. We design three models of ISRL: a basic version for finding one desired file, MP-ISRL for finding at least k files, and C-ISRL for reducing maintenance overhead through clustering when there are many queries. ISRL outperforms existing searching approaches in unstructured peer-to-peer networks by achieving similar query quality with lower cumulative query cost. The experimental result confirms the performance improvement of ISRL.