Learning to optimize halide with tree search and random programs

Abstract

We present a new algorithm to automatically schedule Halide programs for high-performance image processing and deep learning. We significantly improve upon the performance of previous methods, which considered a limited subset of schedules. We define a parameterization of possible schedules much larger than prior methods and use a variant of beam search to search over it. The search optimizes runtime predicted by a cost model based on a combination of new derived features and machine learning. We train the cost model by generating and featurizing hundreds of thousands of random programs and schedules. We show that this approach operates effectively with or without autotuning. It produces schedules which are on average almost twice as fast as the existing Halide autoscheduler without autotuning, or more than twice as fast with, and is the first automatic scheduling algorithm to significantly outperform human experts on average. CCS Concepts: • Computing methodologies → Image processing; • Software and its engineering → Domain specific languages.

Model-Based Warp Overlapped Tiling for Image Processing Programs on GPUs

“…We use a 3.4 GHz, quad-core Intel i5-4670 CPU with 16GB RAM and two GPUs (each experiment uses a single GPU): an NVIDIA GTX 1080Ti and an NVIDIA Tesla V100 (Table 1 lists their key specifications). For our benchmarks, we use six canonical image processing applications that have appeared in prior work [6,14,18,19,22]. Table 2 reports the number of stages and the size of the input image for each benchmark.…”

“…The final problem involves choosing tile and block sizes. We present an automatic fusion algorithm that considers key factors affecting the performance of GPU kernels which are not considered in previous work [6,17,18]: 1) number of global memory transactions, 2) achieved and theoretical occupancy, 3) GPU resource usage, and 4) fraction of overlapping computations.…”

“…These DSLs allow the programmer to write independent stages in a natural way, but still get high-performance code by applying key optimizations, including loop fusion and overlapped tiling. Loop fusion allows the program to exploit locality, and is performed on the basis of a schedule that is either specified by an expert [22,23] or automatically generated using heuristics [6,14,17,18]. After loop fusion, overlapped tiling [19,22,23] splits each stage into overlapping regions (known as tiles) that can be processed in parallel without synchronization with other tiles.…”

Fireiron

“…Auto-Tuning approaches including Halide's auto-tuners [1,23], OpenTuner [2], ATF [24], and program synthesis techniques such as SwizzleInventor [20] aim to automatically develop optimized code using design space exploration. We aim to automatically synthesize Fireiron strategies in the future but in its current version it is designed as a tool for human performance experts.…”

Machine Learning Systems are Stuck in a Rut

“…Recent research shows growing interest in automatic whole-program optimization techniques [18][19][20], but approaches are preliminary and typically focus on optimizing only one aspect of a program at a time. There is no doubt that multi-dimensional whole program optimization is a hard task, but we can perhaps take some hope from the recent success of hybrid search/learning approaches such as AlphaGo [21] that show promise in finding good solutions within huge combinatorial search spaces.…”