Bringing together automatic differentiation and OpenMP

Multithreaded derivative computation with generated libraries

“…No specific change is needed for the Rapsodia code generator or the preparation of the f source code for overloading. With the Rapsodia library we saw results similar to those first presented in [11].…”

“…Given the ubiquity of multicore hardware, a major avenue for improving the efficiency of the derivative computation is its parallelization, even for numerical models that themselves are not parallelized. Several forays have already been made in this direction; see, for instance [11][12][13], most of which use OpenMP. In Section 4 we describe the problems we experienced with the use of OpenMP and an alternative implementation that employs a queue to asynchronously compute the derivatives with the help of either pthreads or the OpenPA library [14].…”

“…The opportunity to parallelize the outer loop has been well recognized, and various attempts have been made to exploit it, in particular with the help of OpenMP; see [11][12][13]. OpenMP is a convenient method to enable shared-memory parallelism.…”

“…The first is coarse-grained parallelism, which uses a single parallel section covering the entire execution of f, as was done in, for example [11]. The outer loop over the d directions is moved from inside the overloaded operators to the driver code (i.e., the code that calls f).…”

Automatic Generation of Parallel Code for Hessian Computations

“…The idea is based on the fact that, for large n, almost all the computational work is derivative computation, and the type of the operations for the derivative computations are always similar, e.g., vector linear combinations for first-order derivatives, which can be easily parallelized using appropriate OpenMP directives. In [11,12] two strategies for parallelizing the computation of first-order derivatives with OpenMP have been proposed. Both strategies can be implemented in software tools for AD, making such an AD tool capable of generating parallel differentiated code.…”