Markov random fields (MRF's) have been widely used to model images in Bayesian frameworks for image reconstruction and restoration. Typically, these MRF models have parameters that allow the prior model to be adjusted for best performance. However, optimal estimation of these parameters(sometimes referred to as hyper parameters) is difficult in practice for two reasons: i) direct parameter estimation for MRF's is known to be mathematically and numerically challenging; ii)parameters can not be directly estimated because the true image cross section is unavailable.In this paper, we propose a computationally efficient scheme to address both these difficulties for a general class of MRF models,and we derive specific methods of parameter estimation for the MRF model known as generalized Gaussian MRF (GGMRF).The first section of the paper derives methods of direct estimation of scale and shape parameters for a general continuously valued MRF. For the GGMRF case, we show that the ML estimate of the scale parameter, sigma, has a simple closed-form solution, and we present an efficient scheme for computing the ML estimate of the shape parameter, p, by an off-line numerical computation of the dependence of the partition function on p.The second section of the paper presents a fast algorithm for computing ML parameter estimates when the true image is unavailable. To do this, we use the expectation maximization(EM) algorithm. We develop a fast simulation method to replace the E-step, and a method to improve parameter estimates when the simulations are terminated prior to convergence.Experimental results indicate that our fast algorithms substantially reduce computation and result in good scale estimates for real tomographic data sets.

Bayesian tomographic reconstruction algorithms generally require the efficient optimization of a functional of many variables. In this setting, as well as in many other optimization tasks, functional substitution (FS) has been widely applied to simplify each step of the iterative process. The function to be minimized is replaced locally by an approximation having a more easily manipulated form, e.g., quadratic, but which maintains sufficient similarity to descend the true functional while computing only the substitute. We provide two new applications of FS methods in iterative coordinate descent for Bayesian tomography. The first is a modification of our coordinate descent algorithm with one-dimensional (1-D) Newton-Raphson approximations to an alternative quadratic which allows convergence to be proven easily. In simulations, we find essentially no difference in convergence speed between the two techniques. We also present a new algorithm which exploits the FS method to allow parallel updates of arbitrary sets of pixels using computations similar to iterative coordinate descent. The theoretical potential speed up of parallel implementations is nearly linear with the number of processors if communication costs are neglected.

No abstract

A standard approach to solving inversion problems that involve many parameters uses gradient-based optimization to find the parameters that best match the data. I$-e will discuss enabling techniques that facilitate application of this approach to large-scale computational simulations. which are the only way to investigate many complex physical phenomena. Such simulations may not seem to lend themselves to calculation of the gradient with respect to numerous parameters. However, adjoint differentiation allows one to efficiently compute the gradient of an objective function with respect to all the variables of a simulation. LVhen combined with advanced gradient-based optimization algorithms. adjoint differentiation permits one to solve very large problems of optimization or parameter estimation. These techniques will be illustrated through the simulation of the time-dependent diffusion of infrared light through tissue. which has been used to perform optical tomography. The techniques discussed have a wide range of applicability to modeling including the optimization of models to achieve a desired design goal.Key words: simulation. inversion. reconstruction. adjoint differentiation. sensitivity analysis. optimization, model validation 1.Frequently a physical situation can only be described fully by a computational rnodel. We wish to address the general problem of finding the values of the parameters in such a model that best match a given set of data. This problem in often referred to as that of inversion. In data matching the objective function to be minimized is often the negative logarithm of the likelihood of the data given their predicted values, which yields the maximum likelihood (SIL) solution. Alternative approaches include regularized versions of maximum likelihood and Bayesian methods. in which the objective function is the minus-log-posterior, yielding the maximum a posteriori (hL4P) estimate.iVe confine ourselves to objective functions that depend on the parameters in a continuous and differentiable fashion. W e do not necessarily avoid problems for which the objective function possesses multiple minima. However, because the techniques that we present make use of gradients in the optimization process, they will work effectively only when one can easily find the basin of attraction for the global minimum. for example, by multiscale or multiresolution optimization.