New Acyclic and Star Coloring Algorithms with Application to Computing Hessians

Gebremedhin, Assefaw H.; Tarafdar, Arijit; Manne, Fredrik; Pothen, Alex

doi:10.1137/050639879

Cited by 49 publications

(38 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By putting sparser rows near the bottom, we do not change the number of non-zeros in the lower triangle, but we should come close to minimizing the number of non-zeros in each row. Thus, we would expect the number of columns with non-zero elements in common rows to be smaller, and the intersection graph to be sparser (Gebremedhin et al 2007). …”

Section: Partitioning the Variablesmentioning

confidence: 99%

“…Coleman and Moré (1984) define a "triangular coloring" as a proper coloring with the additional condition that common neighbors of a vertex do not have the same color. A triangular coloring is a special case of an "cyclic coloring," in which any cycle in the graph uses at least three colors (Gebremedhin, Tarafdar, Manne, and Pothen 2007). …”

Section: Partitioning the Variablesmentioning

confidence: 99%

See 1 more Smart Citation

sparseHessianFD: An R Package for Estimating Sparse Hessian Matrices

Braun¹

2017

J. Stat. Soft.

View full text Add to dashboard Cite

Sparse Hessian matrices occur often in statistics, and their fast and accurate estimation can improve efficiency of numerical optimization and sampling algorithms. By exploiting the known sparsity pattern of a Hessian, methods in the sparseHessianFD package require many fewer function or gradient evaluations than would be required if the Hessian were treated as dense. The package implements established graph coloring and linear substitution algorithms that were previously unavailable to R users, and is most useful when other numerical, symbolic or algorithmic methods are impractical, inefficient or unavailable.Keywords: sparse Hessians, sparsity, computation of Hessians, graph coloring, finite differences, differentiation, complex step.The Hessian matrix of a log likelihood function or log posterior density function plays an important role in statistics. From a frequentist point of view, the inverse of the negative Hessian is the asymptotic covariance of the sampling distribution of a maximum likelihood estimator. In Bayesian analysis, when evaluated at the posterior mode, it is the covariance of a Gaussian approximation to the posterior distribution. More broadly, many numerical optimization algorithms require repeated computation, estimation or approximation of the Hessian or its inverse; see Nocedal and Wright (2006).The Hessian of an objective function with M variables has M 2 elements, of which M (M +1)/2 are unique. Thus, the storage requirements of the Hessian, and computational cost of many linear algebra operations on it, grow quadratically with the number of decision variables. For applications with hundreds of thousands of variables, computing the Hessian even once might not be practical under the given time, storage or processor constraints. Hierarchical models, in which each additional heterogeneous unit is associated with its own subset of variables, are particularly vulnerable to this curse of dimensionality However, for many problems, the Hessian is sparse, meaning that the proportion of "structural" zeros (matrix elements that are always zero, regardless of the value at which the Hessian

show abstract

Section: Partitioning the Variablesmentioning

confidence: 99%

Section: Partitioning the Variablesmentioning

confidence: 99%

sparseHessianFD: An R Package for Estimating Sparse Hessian Matrices

Braun¹

2017

J. Stat. Soft.

View full text Add to dashboard Cite

show abstract

“…Hence the graph needs to be colored in parallel. Finding a distance-k coloring using the fewest colors is an NP-hard problem [20], but greedy heuristics are effective in practice, as they run fast and provide solutions of acceptable quality [7,11,14]. They are, however, inherently sequential and thus challenging to parallelize.…”

mentioning

confidence: 99%

“…The literature on algorithmic graph theory features some work related to the distance-2 coloring problem [1,2,18,19]. Readers are referred to Section 11.4 of the paper [11] and Section 2.3 of the paper [14] for more pointers to theoretical work on distance-k and related coloring problems.…”

mentioning

confidence: 99%

“…Gebremedhin, Manne, and Pothen [13] have developed shared-memory parallel algorithms for distance-2 coloring. They have also provided a comprehensive review of the role of graph coloring in derivative computation in [11], and designed efficient serial algorithms for acyclic and star coloring (which are used in Hessian computation) in [14]. The literature on algorithmic graph theory features some work related to the distance-2 coloring problem [1,2,18,19].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Distributed-Memory Parallel Algorithms for Distance-2 Coloring and Related Problems in Derivative Computation

Bozdağ¹,

Çatalyürek²,

Gebremedhin³

et al. 2010

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

Abstract. The distance-2 graph coloring problem aims at partitioning the vertex set of a graph into the fewest sets consisting of vertices pairwise at distance greater than two from each other. Its applications include derivative computation in numerical optimization and channel assignment in radio networks. We present efficient, distributed-memory, parallel heuristic algorithms for this NPhard problem as well as for two related problems used in the computation of Jacobians and Hessians. Parallel speedup is achieved through graph partitioning, speculative (iterative) coloring, and a BSPlike organization of parallel computation. Results from experiments conducted on a PC cluster employing up to 96 processors and using large-size real-world as well as synthetically generated test graphs show that the algorithms are scalable. In terms of quality of solution, the algorithms perform remarkably well-the number of colors used by the parallel algorithms was observed to be very close to the number used by the sequential counterparts, which in turn are quite often near optimal. Moreover, the experimental results show that the parallel distance-2 coloring algorithm compares favorably with the alternative approach of solving the distance-2 coloring problem on a graph G by first constructing the square graph G 2 and then applying a parallel distance-1 coloring algorithm on G 2 . Implementations of the algorithms are made available via the Zoltan load-balancing library.

show abstract

An introduction to algorithmic differentiation

Gebremedhin

Walther

2019

WIREs Data Min & Knowl

Self Cite

View full text Add to dashboard Cite

Algorithmic differentiation (AD), also known as automatic differentiation, is a technology for accurate and efficient evaluation of derivatives of a function given as a computer model. The evaluations of such models are essential building blocks in numerous scientific computing and data analysis applications, including optimization, parameter identification, sensitivity analysis, uncertainty quantification, nonlinear equation solving, and integration of differential equations. We provide an introduction to AD and present its basic ideas and techniques, some of its most important results, the implementation paradigms it relies on, the connection it has to other domains including machine learning and parallel computing, and a few of the major open problems in the area. Topics we discuss include: forward mode and reverse mode of AD, higher‐order derivatives, operator overloading and source transformation, sparsity exploitation, checkpointing, cross‐country mode, and differentiating iterative processes. This article is categorized under: Algorithmic Development > Scalable Statistical Methods Technologies > Data Preprocessing

show abstract

New Acyclic and Star Coloring Algorithms with Application to Computing Hessians

Cited by 49 publications

References 21 publications

sparseHessianFD: An R Package for Estimating Sparse Hessian Matrices

sparseHessianFD: An R Package for Estimating Sparse Hessian Matrices

Distributed-Memory Parallel Algorithms for Distance-2 Coloring and Related Problems in Derivative Computation

An introduction to algorithmic differentiation

Contact Info

Product

Resources

About