A Dual Active-Set Algorithm for Regularized Monotonic Regression

Abstract

Monotonic (isotonic) regression is a powerful tool used for solving a wide range of important applied problems. One of its features, which poses a limitation on its use in some areas, is that it produces a piecewise constant fitted response. For smoothing the fitted response, we introduce a regularization term in the monotonic regression, formulated as a least distance problem with monotonicity constraints. The resulting smoothed monotonic regression is a convex quadratic optimization problem. We focus on the case, where the set of observations is completely (linearly) ordered. Our smoothed pool-adjacent-violators algorithm is designed for solving the regularized problem. It belongs to the class of dual active-set algorithms. We prove that it converges to the optimal solution in a finite number of iterations that does not exceed the problem size. One of its advantages is that the active set is progressively enlarging by including one or, typically, more constraints per iteration. This resulted in solving large-scale test problems in a few iterations, whereas the size of that problems was prohibitively too large for the conventional quadratic optimization solvers. Although the complexity of our algorithm grows quadratically with the problem size, we found its running time to grow almost linearly in our computational experiments.

A Dual Active-Set Algorithm for Regularized Slope-Constrained Monotonic Regression

“…Obviously, each of them is active if, and only if, the other one is active. According to [4,9], each iteration of the SPAV algorithm is related to choosing an active set ⊆ [1, − 1] and solving the corresponding subproblem Each block is characterized by its common value,…”

“…The algorithm, as described in [4,9], starts with any active set such that ⊆ * , where * is the active set for the optimal solutions in (6) and (9). The simplest of the valid choices is = ∅.…”

“…This poses a limitation of using the MR in some applications. To prevent from such too large values, a smoothing approach was proposed in [4,9]. It is based on introducing the penalty term…”

“…Given a data set, the penalty parameter can be chosen, as suggested in [9], by making use of Bayesian modeling and a cross-validation technique. Following [4,9], we denote…”

“…In [4,9], we developed an efficient dual active-set algorithm for solving the SMR problem. It was called smoothed pool adjacent violators (SPAV) algorithm.…”