ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learning

Abstract

DNN accelerators provide efficiency by leveraging reuse of activations/weights/outputs during the DNN computations to reduce data movement from DRAM to the chip. The reuse is captured by the accelerator's dataflow. While there has been significant prior work in exploring and comparing various dataflows, the strategy for assigning on-chip hardware resources (i.e., compute and memory) given a dataflow that can optimize for performance/energy while meeting platform constraints of area/power for DNN(s) of interest is still relatively unexplored. The design-space of choices for balancing compute and memory explodes combinatorially, as we show in this work (e.g., as large as O(10 72 ) choices for running MobileNet-V2), making it infeasible to do manual-tuning via exhaustive searches. It is also difficult to come up with a specific heuristic given that different DNNs and layer types exhibit different amounts of reuse.In this paper, we propose an autonomous strategy called Con-fuciuX to find optimized HW resource assignments for a given model and dataflow style. ConfuciuX leverages a reinforcement learning method, REINFORCE, to guide the search process, leveraging a detailed HW performance cost model within the training loop to estimate rewards. We also augment the RL approach with a genetic algorithm for further fine-tuning. Con-fuciuX demonstrates the highest sample-efficiency for training compared to other techniques such as Bayesian optimization, genetic algorithm, simulated annealing, and other RL methods. It converges to the optimized hardware configuration 4.7 to 24 times faster than alternate techniques.

Search for Optimal Systolic Arrays: A Comprehensive Automated Exploration Framework and Lessons Learned

“…Iterative methods. Examples include simulated annealing [5], evolutionary search [43], Bayesian optimization [32], and reinforcement learning [19]. In this work, we examined several iterative search methods and chose evolutionary search given its high sample efficiency and search quality.…”

“…There is a plethora of previous works on performance tuning of systolic arrays [4,9,15,17,19,21,28,31,41]. Table 1 lists several recent works.…”

“…RL is a machine learning algorithm that can be used for hardware optimization. The previous work ConfuciuX [19] implemented a two-step search algorithm for tuning the dataflow architectures which employs RL as the first step to locate a good sub design space and utilizes evolutionary search to perform a more fine-grained search later to find the best design. We use the open-source implementation from ConfuciuX as the RL baseline [20].…”

“…Many previous works have attempted this challenging task by looking into different dimensions of the design space and proposing various auto-tuning methods [4,9,17,19,21,28,31,41]. However, after a thorough examination of the previous works, we identified several limitations that need to be addressed.…”

EF-Train: Enable Efficient On-device CNN Training on FPGA through Data Reshaping for Online Adaptation or Personalization

“…DRAM Access Issues for Current FPGA-based Inference Accelerators: Currently, FPGAs have been widely adopted in edge domains thanks to the well-developed FPGA-based inference accelerators [38,41]. Among the inference accelerators, many works [10,12,38] mainly focused on selecting optimal design parameters to improve the acceleration performance for individual Conv layers. Optimizing techniques such as loop tiling, loop unrolling are adopted by these works.…”