From relational verification to SIMD loop synthesis

Abstract

Existing pattern-based compiler technology is unable to effectively exploit the full potential of SIMD architectures. We present a new program synthesis based technique for auto-vectorizing performance critical innermost loops. Our synthesis technique is applicable to a wide range of loops, consistently produces performant SIMD code, and generates correctness proofs for the output code. The synthesis technique, which leverages existing work on relational verification methods, is a novel combination of deductive loop restructuring, synthesis condition generation and a new inductive synthesis algorithm for producing loop-free code fragments. The inductive synthesis algorithm wraps an optimized depth-first exploration of code sequences inside a CEGIS loop. Our technique is able to quickly produce SIMD implementations (up to 9 instructions in 0.12 seconds) for a wide range of fundamental looping structures. The resulting SIMD implementations outperform the original loops by 2.0×-3.7×.Motivating Example. Consider the program fragment in Figure 1, which consists of a loop that traverses an array of widget structs (of length len). The loop body checks if the values in the tag and score fields satisfy certain properties and if so returns 1 immediately. If no such widget is found then 0 is returned.

FrAngel: component-based synthesis with control structures

“…As noted in Section 1, many component-based synthesizers cannot handle control structures, especially loops [Feng et al 2017b;Gulwani et al 2011a;Jha et al 2010;Menon et al 2013]. Beyond component-based synthesis, some approaches in the wider program synthesis literature do handle loops [Bar-David and Taubenfeld 2003;Barthe et al 2013;Heule et al 2015;Qi et al 2012;Srivastava et al 2010], but many do not [Alur et al 2017;Balog et al 2017;Devlin et al 2017;Lau et al 2000;]. Hence, handling combinations of control structures is a distinguishing aspect of FrAngel.…”

“…Some synthesizers handle loops with techniques from program verification [Bar-David and Taubenfeld 2003;Barthe et al 2013;Srivastava et al 2010], but such approaches have only been demonstrated in low-level code (i.e., only using primitive operations and not library functions). Other approaches call components that loop internally [Feser et al 2015;Harris and Gulwani 2011;Qi et al 2012;Yaghmazadeh et al 2016] or use a DSL for specialized loops [Gulwani 2011;Perelman et al 2014], but these are only applicable in specific domains where the loops can be categorized into a few common types.…”

The three pillars of machine programming

“…As architectures continue to evolve and become more complex and reconfigurable, some of the responsibility for coping with this complexity will fall on the invention layer, either because it will have to discover algorithms that map well to the constraints imposed by the hardware, or in the case of architectures that include FPGAs, the invention layer may need to derive the hardware abstractions themselves that align for a given algorithm. There is already some precedent in using constraint-based synthesis to handle non-standard architectures, ranging from exploiting vector instructions [11], to synthesizing for complex low-power architectures [56], but significantly more research is needed for this problem to be fully addressed.…”

Approaching Symbolic Parallelization by Synthesis of Recurrence Decompositions

“…Trivially, the equivalence of final states entails equivalence of the outputs computed out of these states. array-max 1 ; given functions: 2 (define (f element current-max) (max element current-max)) 3 (define (array-max A) (foldl f -inf.0 A)) ; function needed for parallel decomposition: 6 (define (merge-max out) (array-max out))…”