A class of optimization problems motivated by rank estimators in robust regression

Abstract

A rank estimator in robust regression is a minimizer of a function which depends (in addition to other factors) on the ordering of residuals but not on their values. Here we focus on the optimization aspects of rank estimators. We distinguish two classes of functions: the class with a continuous and convex objective function (CCC), which covers the class of rank estimators known from statistics, and also another class (GEN), which is far more general. We propose efficient algorithms for both classes. For GEN we propose an enumerative algorithm that works in polynomial time as long as the number of regressors is O(1). The proposed algorithm utilizes the special structure of arrangements of hyperplanes that occur in our problem and is superior to other known algorithms in this area. For the continuous and convex case, we propose an unconditionally polynomial algorithm finding the exact minimizer, unlike the heuristic or approximate methods implemented in statistical packages.

Walks on hyperplane arrangements and optimization of piecewise linear functions

“…then WoA yields a polynomial method. This is a weak result compared to the polynomiality theorem from [3], where a (theoretically) efficient algorithm based on the ellipsoid method was designed. The crucial question is: Does WoA make sense when we already have an unconditionally polynomial method?…”

“…Does the simplex method with exponential worst-case complexity make sense once we have the ellipsoid method which works in polynomial time? The polynomial method for minimization of F from [3] is based on two procedures: the ellipsoid method for oracle-given polyhedra [8] and Diophantine approximation [10]. Both of these procedures, although polynomial in theory, are extremely hard to implement for numerical reasons: the computation involves rational numbers with huge bit sizes.…”

“…And the poor practical behavior of the ellipsoid method, well known from computational studies (see e.g. [6]), manifests itself in case of [3] as well. So it makes sense to have another method for minimization of F , possibly with superpolynomial worst-case bound, but with a potential to work better in many practical cases and with less serious numerical problems than those concerning the ellipsoid methods and Diophantine approximation.…”