AlphaGo Policy Network: A DCNN Accelerator on FPGA

Abstract

The game of GO has long been regarded as the most challenging game for artificial intelligence because of its enormous search space and the difficulty of evaluating its board positions. In early 2016, the defeat of Lee Sedol by AlphaGo became the milestone of artificial intelligence. AlphaGo's success lies in that it efficiently combines policy and value networks with Monte Carlo tree search (MCTS). And these deep convolutional neural networks (DCNNs) are trained by the combination of supervised learning and reinforcement learning. However, large convolution operations are computationally-intensive and typically require a powerful computing platform, for example, a graphics processing unit (GPU). Therefore, it is challenging to apply DCCNs in resource-limited embedded systems. Field programmable gate array (FPGA) is proposed to be an appropriate solution to implement real-time DCCNs models. However, the limited bandwidth and on-chip memory storage are the bottlenecks for DCCNs acceleration. In this paper, an AlphaGo Policy Network is designed, and efficient hardware architectures are proposed to accelerate the DCCN model. The accelerator can be fit into different FPGAs, providing the balancing between processing speed and hardware resources. As an example, the AlphaGo Policy Network is implemented on Xilinx design suite VCU118, and the results show that our implementation achieved a performance of 3036.32 GOPS and achieved up to 56x speedup compared to CPU and 22.4x speedup compared to GPU.

A discrete dual convolution operation

“…CNNs can also be used to design and implement other application systems by combining with other technologies. Examples of these include speech recognition software, machine translation software, game agent Q network [4] , go program Alphago [5] , and machine translation software. These advancements have ushered in a new era of artificial intelligence that is marked by unparalleled prosperity and wide-ranging influence.…”

“…The representatives of special models are NIN [21] . The representatives of reinforcement models are DQN [4] and AlphaGo [5] . In addition, 3D CNN [22] , which changes the input form of the CNN, is also included to process color images, even video images.…”

Hardware Rough Set Processor Parallel Architecture in FPGA for Finding Core in Big Datasets

“…In [6], a framework for integrating hardware and software optimizations for sparse CNNs is shown. It is worth adding, that [7] presented an AlphaGo Policy Network using DCNN accelerator on FPGA.…”

Electromagnetic situation analysis and judgment based on deep learning

“…The basic principle of AlphaGo Zero algorithm is to combine the neural network of DL with the Monte Carlo method of RL. The specific mechanism is to train the deep convolutional neural networks (DCNN) with RL [28,29]. Different from AlphaGo Fan and AlphaGo Lee, AlphaGo Zero algorithm can enhance its performance in Go through complete independent exploration and learning without any supervision or artificial data [27].…”