End-to-End Learning for Prediction and Optimization with Gradient Boosting

Konishi, Tsuyoshi; Fukunaga, Takuro

doi:10.1007/978-3-030-67664-3_12

Cited by 3 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subsequent approaches applied implicit differentiation to the KKT optimality conditions of constrained problems directly , but only on special problem classes such as Quadratic Programs. Konishi and Fukunaga [2021] extend the method of , by modeling second-order derivatives of the optimization for training with gradient boosting methods. Donti et al [2017] uses the differentiable quadratic programming solver of to approximately differentiate general convex programs through quadratic surrogate problems.…”

Section: A Related Workmentioning

confidence: 99%

Folded Optimization for End-to-End Model-Based Learning

Kotary¹,

Dinh²,

Fioretto³

2023

Preprint

View full text Add to dashboard Cite

The integration of constrained optimization models as components in deep networks has led to promising advances in both these domains. A primary challenge in this setting is backpropagation through the optimization mapping, which typically lacks a closed form. A common approach is unrolling, which relies on automatic differentiation through the operations of an iterative solver. While flexible and general, unrolling can encounter accuracy and efficiency issues in practice. These issues can be avoided by differentiating the optimization mapping analytically, but current frameworks impose rigid requirements on the optimization problem's form. This paper provides theoretical insights into the backpropagation of unrolled optimizers, which lead to a system for generating equivalent but efficiently solvable analytical models. Additionally, it proposes a unifying view of unrolling and analytical differentiation through constrained optimization mappings. Experiments over various structured prediction and decision-focused learning tasks illustrate the potential of the approach both computationally and in terms of enhanced expressiveness.

show abstract

Section: A Related Workmentioning

confidence: 99%

Folded Optimization for End-to-End Model-Based Learning

Kotary¹,

Dinh²,

Fioretto³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The authors also consider a random forest [10] implementation, but do not consider a gradient boosting ensemble approach. Konishi and Fukunaga [27] consider gradient boosting problems under the SPO framework for a subset of optimization problems; namely with linear inequality constraints, but do not consider more general convex optimization problems. To our knowledge, dboost is the first 'smart' gradient boosting implementation that can support any convex optimization program that can be cast as convex quadratic cone program.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper we present dboost, a general purpose framework that combines the strength of gradient boosting ensembles with the SPO framework. Previous work [27] considers gradient boosting for integrated prediction and optimization problems but only considers a small subset of optimization problems with linear inequality constraints. In contrast, the dboost framework is capable of supporting any optimization problem that can be cast as a convex quadratic cone program (CQP); and thus supports linear, quadratic and second-order cone programming with general convex cone constraints.…”

Section: Introductionmentioning

confidence: 99%

Gradient boosting for convex cone predict and optimize problems

Butler¹,

Kwon²

2022

Preprint

View full text Add to dashboard Cite

Many problems in engineering and statistics involve both predictive forecasting and decision-based optimization. Traditionally, predictive models are optimized independently from the final decisionbased optimization problem. In contrast, a 'smart, predict then optimize' (SPO) framework optimizes prediction models to explicitly minimize the final downstream decision loss. In this paper we present dboost, a gradient boosting algorithm for training prediction model ensembles to minimize decision regret.The dboost framework supports any convex optimization program that can be cast as convex quadratic cone program and gradient boosting is performed by implicit differentiation of a custom fixed-point mapping. To our knowledge, the dboost framework is the first general purpose implementation of gradient boosting to predict and optimize problems. Experimental results comparing with state-of-the-art SPO methods show that dboost can further reduce out-ofsample decision regret.

show abstract

Gradient boosting for convex cone predict and optimize problems

Butler

Kwon

2023

Operations Research Letters

View full text Add to dashboard Cite

End-to-End Learning for Prediction and Optimization with Gradient Boosting

Cited by 3 publications

References 9 publications

Folded Optimization for End-to-End Model-Based Learning

Folded Optimization for End-to-End Model-Based Learning

Gradient boosting for convex cone predict and optimize problems

Gradient boosting for convex cone predict and optimize problems

Contact Info

Product

Resources

About